Objective: The objective of this study was to estimate patient compliance with oral and vaginal metronidazole treatment of bacterial vaginosis using personal digital assistants (PDAs) and paper diaries.
Goal: The goal of this study was to assess a novel compliance documentation approach.
Study: After each dose of intravaginal or oral metronidazole, 71 subjects recorded the time on a paper diary and answered questions on a PDA. All PDA entries were unknowingly time-date-stamped. Subjects returned for 2- and 6-week posttreatment examinations. Compliance was calculated using a repeated-measures multivariate analysis of variance (ANOVA).
Results: Mean patient compliance rates within the oral metronidazole group were greater with the paper diary compared with the PDA (68.3% and 50.0%, respectively, P = 0.001). The observed rate of compliance agreement for PDA versus paper diary was 69.0% (kappa = 0.4). The majority of noncompliant subjects reported they were compliant with the PDA and paper diary.
Conclusions: PDAs could more accurately document true compliance rates and could be reasonable instruments to assess compliance in intravaginal antimicrobial drug or contraceptive trials.
A study to estimate compliance with oral and vaginal metronidazole treatment of bacterial vaginosis found that personal digital assistants could more accurately represent true compliance rates compared with paper diaries.
From the Departments of *Obstetrics and Gynecology, †Family Medicine, and ‡Office of Biostatistics and Bioinformatics, Medical College of Georgia, Augusta, Georgia
Dr. Ferris has conducted clinical trials, served as a consultant, and received honoraria for speaking at medical education meetings from 3M Pharmaceuticals. The remaining authors have no perceived or potential conflicts.
Austin Lane Technologies, Denton, TX, was contracted to program customized software for the personal digital assistants (PDAs). The authors thank Dr. Sharon Hillier, Director of Reproductive Infectious Disease Research, Department of Obstetrics and Gynecology, Magee Women’s Hospital at the University of Pittsburgh, for performing the Gram stain analyses.
This investigator-initiated study was supported by an unrestricted research grant from 3M Pharmaceuticals, St. Paul, Minnesota.
Correspondence: Daron G. Ferris, MD, Medical College of Georgia, 1423 Harper Street, HH-100, Augusta, GA 30912. E-mail: firstname.lastname@example.org
Received for publication January 12, 2004, and accepted March 10, 2004.
BACTERIAL VAGINOSIS IS COMMON and has been associated with gynecologic and obstetric complications.1–4 Because of these potential risks and similar efficacy rates for various types of therapy, patient medication compliance for treatment of bacterial vaginosis becomes critically important.5 Noncompliance, a widespread phenomenon, creates significant problems for our healthcare system. Noncompliance can be attributed to inconvenient dosing frequency, cost of medication, route of delivery, side effects, dietary restrictions, ineffectiveness of treatment, patients’ perception of their illness, or other cultural and attitudinal factors.6,7
Many therapeutic trials lack a reliable compliance measure. Paper diaries, counting returned/unused medication, measuring systemic drug concentrations in plasma, low-dose chemical markers, and electronic medication event monitor systems have been used.8,9 The first 2 methods are fraught with potential for inaccuracy. The next 2 methods are invasive and comparatively more expensive. The latter electronic technology documents each time the cap of a medication bottle is opened. Although this high-tech instrument can document ingestion of oral medications using a time-date stamp, the microchip is not amenable for documenting use of intravaginal gels or creams.
Personal digital assistants (PDAs) have become enormously popular for professional and personal use. PDAs are small, portable electronic devices that have the capability of storing and transmitting data to a larger computer for further analysis.10 These small computers can also be programmed to secretly time-date stamp data entries. Such a device could be beneficial in assessing patient compliance with intravaginal products that do not require opening a bottle cap. The purpose of our study was to estimate women’s therapeutic compliance for the treatment of bacterial vaginosis using oral or vaginal metronidazole as documented by PDA technology and a standard paper diary.
Materials and Methods
Nonpregnant adult women with a clinical diagnosis of bacterial vaginosis between November 2001 and January 2003 were asked to participate in this Institutional Review Board-approved clinical trial at the Medical College of Georgia. Inclusion criteria were a newly diagnosed bacterial vaginosis infection, ability to see and read English, and willingness and ability to follow the protocol, including use and return of the PDA. Exclusion criteria included coexisting Trichomonas vaginitis or vulvovaginal candidiasis, pregnancy or lactation, age less than 18 years, allergy to metronidazole, or alcohol abuse.
Equipment and Materials
Two treatment regimens were considered, 500-mg metronidazole tablets orally twice daily for 7 days or intravaginal metronidazole gel (MetroGel Vaginal 0.75%, 3M Pharmaceuticals, St. Paul, MN) used nightly for 5 nights. One of these medications was randomly provided to each subject at no charge.
Three questionnaires (initial, end of treatment, and end of study) were designed for the study. The initial questionnaire included questions pertaining to the subject’s demographic information, symptoms, history of vaginitis, computer and PDA experience, and perceptions about her infection. An end-of-treatment questionnaire determined the subjects’ perception of her illness, presence or absence of associated symptoms, level of comfort using the PDA, medication dosing compliance, and reasons for her noncompliance, if applicable. The end-of-study questionnaire assessed the subjects’ overall satisfaction with her treatment, use of the PDA, and willingness to use either of these in the future.
Vaginal specimens were obtained for pH determination, 10% potassium hydroxide and saline wet prep, amine odor (sniff) test, and Gram stain (which were interpreted at the University of Pittsburgh using Nugent’s criteria).
Subjects used a paper medication diary to enter the date and time after each medication dose was taken. Each subject was also provided a PDA (Visor Deluxe Handspring, Inc., Mountain View, CA), which had previously been loaded with customized software (Austin Lane Technologies, Inc., Denton, TX). Before each PDA was distributed to a subject, all other nonessential software was rendered inaccessible, new batteries were inserted, and its proper functioning was assured.
Each subject was instructed to answer a set of 4 to 7 questions on the PDA immediately after every medication dose. Questions were designed to gain information on the progress of the subject’s symptoms, perception of her illness, and ease or difficulty in using the PDA. Unknowingly to the subject, answers to each question were time-date-stamped. Subjects taking oral metronidazole were given additional questions to answer because they had more total doses than the vaginal metronidazole group (14 doses compared with 5). Multiple distractible questions were included to minimize recognition of the masked study intent.
Eligible patients were told that the study was designed to determine the effectiveness of 2 FDA-approved treatment regimens for bacterial vaginosis. The true purpose, determination of patient compliance, was not disclosed. Subjects read and signed the Institutional Review Board-approved informed consent document and then completed the initial questionnaire. Vaginal specimens were collected for pH determination, 10% potassium hydroxide and saline wet prep, sniff test, and Gram stain. A positive clinical diagnosis of bacterial vaginosis was defined as the presence of 3 of 4 Amsel’s criteria. These diagnoses were later confirmed by a positive Nugent’s Gram stain score.5 Although women were enrolled using Amsel’s criteria, only Nugent’s scores were used to determine treatment failure or cure.
Subjects were randomized to drug using a random number table. Subjects also received instructions about drug use, and training for the paper medication diary and PDA. Each subject then completed a standardized set of training questions to ensure her familiarity with the use of the electronic device before leaving the clinic. Subjects were instructed to answer a set of questions on the PDA after every medication dose in addition to completing the paper medication diary.
Subjects returned for a posttreatment visit approximately 14 days after initiating therapy. At this visit, subjects were asked to return the PDA, paper medication diary, and unused drug. A repeat vaginal examination was performed and vaginal specimens obtained for test-of-cure analyses using the same criteria as described before. Medication dosing compliance was verified verbally and by an end-of-treatment questionnaire.
The final follow-up visit was completed approximately 6 weeks after treatment initiation. Subjects underwent a final vaginal examination for test-of-cure analysis using the same laboratory testing as mentioned previously. Subjects also answered a brief end-of-study questionnaire.
Compliance was calculated for each subject and was defined based on the specific randomized treatment group. For the oral metronidazole group, a subject was dose-compliant if the dose was taken every 12 ± 6 hours. A subject was dose-compliant in the vaginal metronidazole group if the dose was taken every 24 ± 12 hours. Using this data, a subject was defined as being overall compliant if ≥80% of all doses were taken within the specified time. A multivariate repeated-measures analysis of variance and retrospective power analysis was conducted using the Statistical Package for the Social Sciences (SPSS, version 11.0.1), as were all statistical analyses. The dependent variable was percentage-compliant. The within-subject (repeated-measures) factor was compliance measure (2 levels: PDA, paper diary). The between-subject factor was treatment (2 levels: vaginal metronidazole, oral metronidazole). Observed power was calculated for an α level of 0.05 at the observed effect size. Other continuous variables were tested using t tests (2-sample or paired). Categorical variables were tested using the chi-squared test or McNemar’s test. Agreement for categorical variables was assessed using Cohen’s kappa.
We enrolled and randomized 125 subjects into the study who were diagnosed clinically as having bacterial vaginosis by Amsel’s criteria. Complete data were available for 71 subjects. Fifty-four subjects were excluded for the following reasons: the initial Nugent’s Gram stain score was ≤6 (n = 37), and no complete data available for all 3 visits or subject lost to follow up (n = 17). Of the eligible women, 37 subjects were randomized to receive oral metronidazole and 34 subjects received vaginal metronidazole.
Demographic data were generally comparable for the evaluable vaginal and oral metronidazole groups (Table 1). However, there was a statistically significant difference (P = 0.04) about income, a tendency for higher household income reported by the oral metronidazole group.
Mean patient compliance rates for each treatment group were calculated using both PDA and paper diary data (Table 2). There were higher compliance rates recorded using the paper diary compared with the PDA. Significantly greater mean paper diary compared with PDA medication compliance was noted within the oral metronidazole cohort, 68.7% and 50.0%, respectively, (P = 0.001). Overall mean medication compliance was also significantly greater for subjects in the vaginal metronidazole cohort compared with the oral metronidazole cohort when stratified by PDA (P = 0.001) and paper diary (P = 0.001).
The results of the multivariate repeated-measures ANOVA for percentage compliance revealed no significant interaction between compliance measure and treatment (P = 0.337, power = 0.159). There was a significant main effect of compliance measure (P = 0.001, power = 0.996) and a significant main effect of treatment (P = 0.001, power = 0.997).
The observed rate of PDA versus paper diary compliance agreement was 69.0% (kappa = 0.4) as seen in Table 3. Interestingly, 41% of women were not compliant with the paper diary and 61% were not compliant with the PDA. Discordance was greater for women who were paper diary-compliant and PDA-noncompliant.
Of the 40 noncompliant subjects as documented by PDA technology, 37 subjects (92.5%) reported that they did answer the questions on the PDA every time they took their medicine. Similarly, 100% of the noncompliant subjects as documented by paper diary also reported compliance with answering the questions on the PDA with each medication dose. When asked if they entered the date and time on the paper diary after taking each medication dose, 95% of the noncompliant subjects based on PDA technology answered yes compared with 92.6% of the noncompliant subjects based on paper diary data. However, when asked if they took their medication at the correct time each day, 53.7% of the noncompliant subjects documented by the PDA reported yes as compared with 53.6% of the noncompliant subjects documented by the paper diary.
There were no statistically significant differences in the 2- or 6-week bacterial vaginosis cure rates for the oral and vaginal metronidazole groups.
Women treated for bacterial vaginosis, using 2 different routes of metronidazole delivery, claimed greater compliance rates using paper diary compared with the PDA. It is possible that aversion to using the PDA because of technophobia could explain this observed difference. Yet, although a lower compliance rate was seen with the PDA, these results could be more indicative of the true subject compliance rate. Whereas compliance measured by paper diary is self-reported, PDA compliance measures were recorded unknowingly. Such masked confirmation is virtually void of dishonest manipulations. The poor-to-fair agreement between compliance tools (kappa = 0.4) implies one method could erroneously inflate actual compliance. This could have been the case for the paper diary. Because of a fundamental difference in the way compliance was recorded with the 2 methods, the paper diary appeared to overestimate subject compliance whereas the PDA data could be more representative of what actually happened.
The majority (>90%) of noncompliant subjects reported that they were compliant regardless of the method of documentation. However, when asked specifically if they took their medication at the correct time each day, only 50% of noncompliant subjects reported that they were compliant. These data underscore a high rate of falsified compliance responses. Therefore, documentation of compliance by self-reporting, instead of by electronic monitoring, should always be suspect. Several studies have documented that an instrumented paper diary more accurately determines subject compliance compared with standard paper diaries.21 Photosensors built into the paper diary trigger an electronic record of the date and time of each diary opening and closing.
A higher compliance rate was observed with vaginal metronidazole as compared with oral metronidazole. The less frequent daily dosing regimen and fewer total treatment days of vaginal metronidazole likely explains this finding. A greater side effect profile previously reported for oral metronidazole could also contribute to a decrease in patient compliance. Although it is commonly assumed women prefer oral versus intravaginal treatment for bacterial vaginosis, our compliance results suggest otherwise. Previous research has also actually documented women’s preference for intravaginal bacterial vaginosis therapy.5
There were a few unanticipated complications with the PDA. We learned these devices can be easily damaged, and data can be lost because of battery failure. Yet, paper diaries are not perfect either. Paper diaries can be notoriously damaged or lost. Electronic monitoring methods do not prove dose ingestion or proper use. However, in this study, subjects were not aware that compliance was being tested. It would require more active intentions on the part of the subject to use the PDA at the appropriate dosing interval without taking the medication. The PDA provides a similar electronic surveillance system because the medication event monitor chips placed in the cap of oral medications. Because there is no similar way to electronically document intravaginal medication compliance, the PDA could be the best way to confirm compliance with intravaginal antimicrobial and contraceptive drug use during clinical trials.
Personal digital assistants, when used in clinical trials, are capable of presenting questions, recording responses, and transmitting stored data to a computer. These electronic diaries could offer some key advantages over paper diaries, including ease of data entry and enhanced data integrity because of internal validity checks and ability to time-date-stamp each entry.10 Previous studies have shown that subjects readily accept electronic diaries, and compliance with data collection is increased when using them. Missing data could be minimized by prompts that encourage a suitable response.11 Using PDAs eliminates timely transcription, coding and data reentry. Data are immediately available in an electronic format, which helps investigators facilitate trial completion. Personal digital assistants also have the capability of enhancing patient compliance by providing alarms to indicate when the next dose of medication is to be administered. All these advantages help reduce expenses and save money. Because of these attributes, PDAs have great potential for utilization in future clinical trials, particularly those involving nonoral medication interventions.