Introduction
Highly active antiretroviral therapy (HAART) has been clearly demonstrated to improve HIV-related clinical outcomes, including decreased opportunistic infections and prolonged life as well as improve surrogate markers such as HIV viral load and CD4 cell counts [1,2]. Unfortunately, not all individuals who are prescribed HAART benefit from the therapy [3,4]. Suboptimal adherence to these regimens has been postulated to be one of the main factors associated with decreased HIV suppression [5,6]. Furthermore, suboptimal adherence has been postulated to be an important mechanism for the emergence of resistant virus [7,8].
Recently published studies using a variety of techniques to assess adherence have demonstrated that greater adherence to therapy is associated with better HIV outcomes [9-13]. The techniques used in these studies have included self-reports [10,11], pill counts and drug levels [9], microelectronic monitors [12], or microelectronic monitors plus self-reports [13]. Microelectronic monitors have been used to measure adherence in a variety of settings and are considered to be one of the most valid of all the techniques [14-20].
As the various highly active agents have different pharmacokinetics and pharmacodynamics, it may be inappropriate to compare adherence in subjects on different highly active agents. Furthermore, adherence behavior in experimental studies is likely to be different from the clinic setting. Therefore, we chose to study subjects newly starting a single protease inhibitor in a clinic setting to determine whether adherence was different between those who did and did not achieve viral suppression to below the limit of detection (`undetectable'). Nelfinavir was chosen because, at the time this study began, it was the most commonly prescribed highly active agent in protease inhibitor naive subjects at the Philadelphia HIV clinic sites from which we recruit study subjects.
Several mathematical ways of summarizing an individual's pill-taking behavior exist [21]. Therefore, we also designed the study to determine which method best discriminates between the groups that do and do not achieve virological suppression. Secondary objectives were to determine the relation between adherence and the magnitude of decrease in viral load and increase in CD4 cell count over time.
Methods
Study design
We conducted a prospective observational cohort study of adherence to HAART at the University of Pennsylvania Medical Center between February 1998 and November 1999. All medical management was left to the discretion of the care providers and data were not shared with the study subjects or their physicians until the subject had completed the study. Study visits occurred at time 0, 1 month, and every month thereafter for a total of 4 months and were conducted at the General Clinical Research Center of the University of Pennsylvania Medical Center.
The primary outcome was viral suppression to undetectable levels using the VERSANT HIV-1 RNA 3.0 assay (bDNA) (Bayer Corporation, Berkeley, California, USA) (i.e. < 50 copies/ml; ) at the final visit. Secondary outcomes included magnitude of change in the log10 viral load and CD4 cell count from baseline to final visit. The endpoint of 4 months was chosen as most individuals who achieve undetectable viral loads will have done so by this time point [1]. Subjects were compensated with US$ 10 in cash at the time of each visit and a check for US$ 120 at the end of the study, for a total of US$ 180 if the study was completed.
Study population
Entry criteria included being naive to protease inhibitors, having an initial plasma viral load of > 10 000 copies/ml, and being newly started on a HAART regimen including nelfinavir, dosed either 750 mg three times daily or 1250 mg twice daily, in combination with two nucleoside analogue reverse transcriptase inhibitors (NRTI). At least one of the NRTIs had to be new to the patient, per standard treatment guidelines [22]. We excluded subjects with previous protease inhibitor treatment to decrease the possibility of subjects entering with protease inhibitor-resistant virus. We enrolled subjects using hydroxyurea for augmenting didanosine, but excluded subjects on dual protease inhibitor regimens and non-nucleoside analogue reverse transcriptase inhibitor regimens. We excluded these subjects because we felt that these regimens might differ from the single protease inhibitor regimens in the magnitude of adherence needed for viral suppression. We excluded subjects with viral loads < 10 000 copies/ml to limit the possibility of subjects achieving maximal suppression in the absence of adherence to the protease inhibitor (i.e. by adhering only to the NRTIs). Subjects were excluded if they could not provide informed consent or lived in a medical facility in which medications were dispensed for them. We did not exclude the mentally ill or active substance abusers if they were deemed to have the capacity to provide informed consent.
We recruited subjects from the Hospital of the University of Pennsylvania, the Presbyterian Medical Center, and the Veterans' Administration Medical Center, in Philadelphia, Pensylvania. The HIV-specialist pharmacists at these sites were asked to provide one of the investigators (R.G.) with the names of individuals who were newly starting nelfinavir-containing HAART regimens. In addition, subjects were referred by HIV providers directly if they were not seen by the pharmacists. HIV providers at other medical centers in Philadelphia who had links to HIV research programs at the University of Pennsylvania were also requested to refer their patients to the study.
Measurement of adherence
Adherence was tracked continuously using microelectronic monitors on the nelfinavir bottle (MEMS®; APREX Corporation, Menlo Park, California, USA). The data recorded consisted of the date and time of each pill bottle opening as well as the serial number of the pill cap, to ensure that each subject continued to use his/her own cap. Data were downloaded to a computer using APREX Corporation proprietary hardware and software. Subjects were informed that adherence was being tracked and were instructed not to take out any more pills than the ones they were scheduled to take at each bottle opening. They were also instructed not to open the bottles unless they were scheduled to take a dose. The system 'filters out' bottle-opening events recorded within 15 min of a previous event, so these events were excluded. In addition, events recorded at the time of MEMS cap transfers at the end of the month from the old (empty) bottle to the new (full) bottle were excluded, except when the transfer occurred at the time the last dose was taken from the old bottle.
The pill-taking data were obtained exclusively from the MEMS caps. Adherence was summarized over the time of observation using several a priori-defined adherence variables. The primary variable was the percentage of prescribed doses taken over the 4 months and was calculated by dividing the observed pill-taking events by the number of doses prescribed for the study period. The percentage of days with the optimal number of doses (`percentage optimal days') was also calculated. We defined a day as extending from 0200 to 0159 h to allow for late night pill taking. An optimal day was defined as one in which at least the prescribed number of doses was taken. To calculate the percentage optimal days, the number of optimal days was divided by the number of days of observation. Another variable was the coefficient of variation, which was calculated by the following formula: (standard deviation of the time between doses/mean time between doses) × 100%. Finally, the duration of the maximal interval between doses was also determined for each subject.
In addition, we a priori defined a pill-taking gap as 3 days between any two doses, based on the data of Vanhove et al. [23]. We also explored 7-day 'drug holidays' as a measure of adherence.
Analysis
Descriptive analyses of the baseline demographics of the population were performed. Categorical variables were summarized using percentages and compared using the S2 test or Fisher's exact test and continuous variables were summarized using medians and ranges and compared using the Wilcoxon rank sum test. The adherence variables were compared between the detectable and undetectable groups using the Wilcoxon rank sum test. Spearman's correlation coefficient was calculated for the association between the adherence variables and changes in viral load and CD4 cell count.
Receiver operator characteristic (ROC) curves were used to compare the discriminatory ability of the different adherence summary variables in predicting undetectable viral load. The ROC curve is a graphical technique for assessing the ability of a variable to discriminate between desirable and undesirable outcomes (e.g., detectable versus undetectable viral load) [24]. For each variable, the sensitivity and specificity of the outcome for each data point of that variable is calculated and then, sensitivity (y axis) is plotted against 1 - specificity (x axis) for each point. The area under the ROC curve is the probability that the variable can correctly discriminate between subjects who do versus do not achieve the outcome (i.e. undetectable viral load) [25]. We therefore compared the discriminatory ability of the four adherence variables considered by testing for differences in the areas under the respective ROC curves [26].
In separate analyses, we grouped subjects by quartile of percentage of prescribed doses taken. We also grouped the subjects in five categories of percentage of prescribed doses taken as defined by Paterson et al. [12], including ≥ 95%, between 90 and 94.9%, between 80 and 89.9%, between 70 and 79.9%, and < 70%. We tested the trend in the proportion of subjects achieving undetectable viral loads across these groups for both grouping methods.
We compared the proportion of individuals achieving a 1 log10 decrease in viral load at the end of the first month between the group that was undetectable at 4 months and the detectables using the S2 test. We also compared adherence between the detectable and undetectable groups by month for each of the first 3 months of observation. The percentage of prescribed doses taken was calculated for each individual for each month separately and the adherence was compared between the groups for each month using the Wilcoxon rank sum test. The data from the last month of follow-up was not analyzed because of lack of uniformity of follow-up (i.e. more or less than 30 days per individual) related to scheduling issues.
Potential confounders of the relation between adherence and HIV outcomes were identified and included age, race, sex, HIV risk group, being naive to all antiretrovirals, and nelfinavir dosing frequency. We controlled for potential confounders using multiple logistic regression [27] for the dichotomous outcomes (e.g., undetectable viral load) and median regression [28] for the continuous outcomes (e.g., change in viral load from baseline). In addition, we explored whether any differences existed between the 'twice daily doses' and 'three times daily' groups in the relation between these summary variables and the outcomes (i.e. effect modification by dosing frequency) using interaction terms in the logistic regression models. However, given the low power of these tests [29], we also visually examined the data for evidence of effect modification. Dropouts were excluded from all analyses of the relation between adherence and outcome.
For adherence comparisons between the detectable and undetectable groups, P values were a priori specified to be one-sided, as it was biologically implausible that adherence would be better in subjects who failed to achieve maximal suppression. For all other comparisons, P values were two-sided.
A sample size of 40 subjects was targeted to yield 80% power to detect a 30% difference in the percentage of prescribed doses taken between the detectable and undetectable groups. This calculation was performed using the Power software program [30] and based on a variance estimate from a prior study of adherence to antiretroviral therapy using microelectronic monitors [20].
We used STATA 6.0 statistical software package to perform all data analyses (STATA Corp., College Station, Texas, USA).
Approval for the study was obtained prior to initiation from the Committee on Research Involving Human Beings (IRB) of both the University of Pennsylvania and the Philadelphia Veteran's Administration Medical Center.
Results
Between February 1998 and July 1999, 58 subjects were recruited into the study. Of this initial cohort, 16 (27%) subjects dropped out before reaching study completion. One additional subject was mistakenly terminated from the study after the 3 month visit and was not included in the analyses. The most common reason for dropping out was defaulting from therapy (10 of 16 subjects, 63%), defined as self-discontinuation of antiretroviral therapy against providers' advice. Of the 10 subjects defaulting, five did so after the initial visit, two defaulted after month 1, two after month 2, and one after month 3. The other reasons included two subjects losing the MEMS cap, two subjects deciding not to participate after enrolling, one subject changing protease inhibitor during the study, and one subject dying. Dropouts were similar to those who completed the study with respect to sex, race, site of recruitment, being treatment naive, initial viral loads and initial CD4 cell counts (Table 1).
Twenty-five subjects (61%) achieved undetectable viral loads. The treatment characteristics included 19 (44%) being treatment experienced, of whom 12 (63%) started two NRTIs to which they were naive at the time of enrollment, 10 (24%) were started on didanosine plus hydroxyurea and another NRTI. No one received abacavir. There were no significant differences in these treatment characteristics between the detectable and undetectable groups (all P values > 0.50).
All four adherence variables tested (percentage of prescribed doses taken, percentage optimal days, coefficient of variation and maximal interval between doses) demonstrated differences between the detectable and undetectable groups (Table 2). No confounders were identified in the multivariable analyses. There was no evidence of effect modification on the relation between pill taking and virological outcome by age, sex, race, HIV risk group, being naive to therapy, or frequency of dosing of nelfinavir.
Pill-taking gaps were also different between the detectable and undetectable groups. The undetectable group had a median of zero 3-day gaps (range 0-6) whereas the detectable group had a median of one 3-day gap (range 0-8, P < 0.02). Similarly, only one in 25 (4%) individuals in the undetectable group had a 7-day gap, whereas seven of 16 (44%) individuals in the detectable group had such a 'drug holiday' (P < 0.002). Of these seven individuals, four had multiple week-long 'drug holidays'.
We analyzed the relation between adherence and virological outcome in two ways, by grouping the subjects into categories defined by Paterson et al. and by quartiles of adherence. The results were similar with either method and only the results of the former method are presented in Figure 1. Although a classic dose-response relationship is not present, and the number of subjects in each subcategory were small, there is a significant trend in the proportion of individuals who achieve undetectable viral load as the adherence category increases (test for linear trend, P = 0.01). There was also a statistically significant trend for greater viral suppression with increases in adherence by category of percentage of prescribed doses (test for linear trend, P = 0.02).
The ability of each of the four adherence variables to discriminate between detectables and undetectables was compared using ROC curves (Fig. 2). There were no statistically significant differences among the four variables with respect to discriminatory ability (P = 0.86), although coefficient of variation had the greatest area under the curve (0.73).
As there were no differences among the adherence variables in discriminatory ability, we focused the remainder of the analyses on the percentage of prescribed doses taken. There was a positive correlation between percentage of prescribed doses taken and log10 decrease in viral load from baseline (Spearman's ρ = 0.38, P < 0.01) and increase in CD4 cell count (Spearman's ρ = 0.25, P = 0.06). The increase in CD4 cell count by adherence category was 128 × 106 cells/l for ≥ 95% adherence, 124 × 106 cells/l for 90-94.9% adherence, 76 × 106 cells/l for 80-89.9% adherence, 17 × 106 cells/l for 70-79.9% adherence, and 63 × 106 cells/l for < 70% adherence. There were no confounders of these relations.
One of the measures used clinically to detect early poor adherence is response to therapy, with approximately 1 log decrease in viral load considered to be reflective of appropriate response after 1 month of therapy [1]. We compared the proportion of individuals experiencing a 1 log or greater drop in viral load to determine whether these early responses differed between the two groups. All but two subjects experienced a 1 log decrease in viral load after 1 month; one (6%) in the detectable group and one (4%) in the undetectable group (P = 0.63). Thus, this measure at this early stage in therapy is unable to distinguish between the subjects who ultimately go on to undetectable viral loads and those who do not.
Given the minimal difference in the proportion with an appropriate viral response at 1 month, we explored the data to determine the time at which poor adherence ensued and the groups diverged in their adherence levels. The percentage of prescribed doses taken did not differ between the groups over the first month (Fig. 3). However, over the second month of therapy, adherence was found to be different and this difference persisted into the third month. Data from the final month were not evaluated because the amount of follow-up time differed between individuals.
Discussion
These data demonstrate that adherence markedly differs between subjects who do and do not achieve viral suppression to below the limit of detection using the latest generation viral load assay. The results show that the viral suppression goes hand-in-hand with CD4 cell count increases, although the impact of adherence on change in CD4 cell count was of a lesser magnitude than the impact on viral suppression. The differences were present as defined by several different adherence variables, but none performed substantially better than any of the others. Therefore, percentage of prescribed doses taken may be the most appropriate choice for describing adherence in this setting as it is the easiest to conceptualize. However, as the degree or pattern of adherence that results in lack of viral suppression may be different than that which results in the emergence of viral resistance, it is possible that these variables will operate differently when studying the relation between adherence and resistance.
A major finding of this study is the fact that in subjects who do not fully suppress, the difference in adherence between the groups does not manifest until the second month of therapy. Practice guidelines recommend that subjects have viral loads monitored at 1 month after initiating a new regimen based on viral kinetics [31]. If viral load is not decreased by at least 1 log or more, then providers are recommended to consider the presence of resistant virus and/or suboptimal adherence. However, although our sample size is limited, our data suggest that there is no difference in very early adherence between subjects who ultimately go on to viral suppression and those who do not. Therefore, providers should avoid having a false sense of security that their patients will not have adherence problems if their viral load is appropriately decreased after 1 month. The presence of a 1 log or greater decrease at 1 month only suggests that the subject does not have resistant virus and that they are adhering early on. However, our results suggest these individuals may have adherence problems in the near future (e.g. by the end of the second month).
As adherence problems did not arise until after the first month of therapy, we recommend monthly visits immediately after initiating antiretroviral therapy to allow physicians time for non-judgmental inquiry about adherence. It is important that these inquiries be undertaken at the time when adherence is likely to wane (i.e. after the first month). As these data suggest that the end of the first month is a period of high risk for waning adherence, investigators designing adherence interventions should strongly consider focusing their strategies on this time window as it may be the time at which the greatest impact occurs. In addition, because the median adherence was above 90% in both groups after the first month, inadequate viral suppression at 1 month of therapy may be more likely to be caused by resistance than poor adherence. Therefore, resistance testing should be strongly considered even after only 1 month of therapy if the subject denies poor adherence.
The findings of this study confirm the importance of adherence on HIV outcome as described recently in a different cohort [12]. However, our results do not agree with the conclusion by Paterson et al. that > 95% adherence is required for substantially better virological outcomes. In contrast, we found a greater proportion of individuals with undetectable viral loads in the range of 80-95% adherence. In fact, there is a suggestion of a threshold effect at 80% rather than 95% adherence although it must be recognized that relatively few subjects in our study took less than 80% of their doses. The reason for this discrepancy may be due to any of several differences in study design and study population. Our study included patients who were newly starting their first protease inhibitor rather than including subjects at various stages of treatment. We included subjects on only a single protease inhibitor, nelfinavir, rather than a variety of protease inhibitors. Our definition of 'undetectable' was < 50 copies/ml rather than < 400 copies/ml. Finally, our follow-up lasted 4 months rather than a median of 6 months. Further work in this area is needed to clarify the discrepancies between the studies.
An important point to note is that more adherence may not always be better than less adherence. In fact, this may differ depending on how much an individual is adhering initially, and how much adherence is necessary for resistant virus to emerge. Although the extremely poor adherers will not achieve suppression, they may not develop resistant virus if their drug exposure is minimal. However, if these individuals increase their adherence by only a modest amount in response to an intervention, they may achieve a greater degree of viral suppression early, but may also be more likely to develop resistant virus. Thus, we need to determine if a threshold amount of adherence is necessary to achieve both a significant immunological benefit as well as forestall the emergence of resistance. If this threshold is found, it should be the target amount of adherence for intervention studies.
This study has several limitations. The small sample size prevented us from drawing conclusions about a possible threshold effect of less than 80% adherence resulting in a substantial drop-off in the proportion of subjects with undetectable viral load (Fig. 1). This same limitation may have prevented us from determining whether one of the adherence variables is a better discriminator of achieving undetectable viral load than the others.
Selection bias may have been present for two reasons. First, this study involved volunteers who agreed to using MEMS caps, have extra clinic visits and more frequent blood sampling in order to participate. If these volunteers were more motivated to adhere than non-participants, then this study underestimates the rate of poor adherence. The monetary compensation may have mitigated this volunteer effect. Participants who were motivated by the compensation, rather than the desire to contribute to scientific advancement, were probably no more likely to adhere than the target population. Second, study dropouts mean that our conclusions are based on a subset of the participants. It is probable that subjects who dropped out had worse adherence than those who remained within the study. This selection would bias our results toward the null as the data on the less adherent patients would have magnified the differences in pill taking between the detectables and undetectables. Despite the potential selection bias, we found large differences between the groups suggesting that the impact of adherence is likely to be even greater than we estimate from these data.
The study is also potentially affected by the inherent limitations of using MEMS in measuring adherence. First, these devices only measure a surrogate of pill taking (i.e. pill bottle opening). One cannot be certain that the correct number of pills were taken at each dose or that individuals did not either take out more doses for later use without opening the bottle or conversely, opened the bottle without taking the dose. However, if study subjects were 'gaming' the system, the results would be biased toward a lack of effect of adherence. Given the dramatic differences we observed, it is unlikely that this potential limitation was present, but if it were, the findings would be even stronger. Second, we only monitored a single drug in the regimen due to the impracticality of having multiple MEMS on an individual's bottles. Therefore, we are only able to make conclusions about adherence to nelfinavir and not the other drugs in the regimen. Of note, however, is a study which demonstrated in the renal transplant setting that adherence to one drug in a regimen is strongly correlated to adherence to other drugs in the regimen [32]. This phenomenon may mitigate the limitation of only monitoring a single drug.
Studies of adherence behavior are always potentially limited by the possibility that the behavior being studied is modified by the very fact that it is being observed. In this case, we believe that subjects who participated in the study were more likely than non-participants to adhere as the study was voluntary. This effect would tend to dampen any differences we might find. Thus, the real differences between the subjects may have been even more pronounced than we found.
This study was also limited by its narrow focus on the relation between early adherence and virological suppression in individuals newly starting on HAART. Given that long-term adherence is currently the desired approach, these results may not be generalizable to the setting of maintenance therapy. For example, after an individual has achieved an undetectable viral load, the amount of adherence required to maintain that level of suppression may differ from the amount required to achieve it. This will be an important area to explore in the future.
Finally, although several groups are currently undertaking research into identifiable and potentially modifiable behavioral factors associated with adherence to HAART [10,11,33,34], more work needs to be done to further elucidate the degree of adherence that should be considered 'optimal' to serve as the outcome in studies of behavioral risk factors. The type of data presented here can be used to set cut points for the amount of pill taking that should be considered 'good adherence' in both prospective observational and experimental studies. Furthermore, as these microelectronic monitors are able to measure the adherence differences between subjects who do and do not achieve the desired virological outcomes, they are likely to be able to measure whether interventions increase adherence.
Acknowledgements
We would like to thank the University of Pennsylvania Infectious Diseases Clinical Trials Unit nurses for their hard work in collecting the data and following the study subjects and the providers for referring their patients to this project. We are very grateful to the PENN Center for AIDS Research for its support. Finally, we want to thank the study subjects for their participation.
References
1. Anon. Report of the NIH Panel to define principles of therapy of HIV infection. Ann Intern Med 1998, 128: 1057-1078.
2. Carpenter CC, Cooper DA, Fischl MA. et al. Antiretroviral therapy in adults: updated recommendations of the International AIDS Society-USA Panel. JAMA 2000, 283: 381-390.
3. Deeks SG, Hecht FM, Swanson M. et al. HIV RNA and CD4 cell count response to protease inhibitor therapy in an urban AIDS clinic: response to both initial and salvage therapy. AIDS 1999, 13: F35-F43.
4. Lucas GM, Chaisson RE, Moore RD. Highly active antiretroviral therapy in a large urban clinic: risk factors for virologic failure and adverse drug reactions. Ann Intern Med 1999, 131: 81-87.
5. Chesney MA, Ickovics J, Hecht FM, Sikipa G, Rabkin J. Adherence: a necessity for successful HIV combination therapy. AIDS. 1999, 13: S271-S278.
6. Ickovics JR, Meisler AW. Adherence in AIDS clinical trials: a framework for clinical research and clinical care. J Clin Epidemiol 1997, 50: 385-391.
7. Hirsch MS, Conway B, D'Aquila RT. et al. Antiretroviral drug resistance testing in adults with HIV infection: implications for clinical management. International AIDS Society-USA Panel. JAMA 1998, 279: 1984-1991.
8. Wainberg MA, Friedland G. Public health implications of antiretroviral therapy and HIV drug resistance. JAMA 1998, 279: 1977-1983.
9. Descamps D, Flandre P, Calvez V. et al. Mechanisms of virologic failure in previously untreated HIV-infected patients from a trial of induction-maintenance therapy. Trilege (Agence Nationale de Recherches sur le SIDA 072) Study Team). JAMA 2000, 283: 205-211.
10. Gifford AL, Bormann JE, Shively MJ, Wright BC, Richman DD, Bozzette SA. Predictors of self-reported adherence and plasma HIV concentrations in patients on multidrug antiretroviral regimens. J Acquir Immune Defic Syndr 2000, 23: 386-395.
11. Haubrich RH, Little SJ, Currier JS. et al. The value of patient-reported adherence to antiretroviral therapy in predicting virologic and immunologic response. California Collaborative Treatment Group. AIDS 1999, 13: 1099-1107.
12. Paterson DL, Swindells S, Mohr J. et al. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med 2000, 133: 21-30.
13. Bangsberg DR, Hecht FM, Charlebois ED. et al. Adherence to protease inhibitors, HIV-1 viral load, and development of drug resistance in an indigent population. AIDS 2000, 14: 357-366.
14. Straka RJ, Fish JT, Benson SR, Suh JT. Magnitude and nature of noncompliance with treatment using isosorbide dinitrate in patients with ischemic heart disease. J Clin Pharm 1996, 36: 587-594.
15. Brun J. Patient compliance with once-daily and twice-daily oral formulations of 5-isosorbide mononitrate: a comparative study. J Intern Med Res 1994, 22: 266-272.
16. Cramer JA. Microelectronic systems for monitoring and enhancing patient compliance with medication regimens. Drugs 1995, 49: 321-327.
17. Cramer J, Vachon L, Desforges C, Sussman NM. Dose frequency and dose interval compliance with multiple antiepileptic medications during a controlled clinical trial. Epilepsia 1995, 36: 1111-1117.
18. Matsui D, Hermann C, Klein J, Berkovitch M, Olivieri N, Koren G. Critical comparison of novel and existing methods of compliance assessment during a clinical trial of an oral iron chelator. J Clin Pharmacol 1994, 34: 944-949.
19. Olivieri NF, Matsui D, Hermann C, Koren G. Compliance assessed by the Medication Event Monitoring System. Arch Dis Child 1991, 66: 1399-1402.
20. Wall TL, Sorensen JL, Batki SL, Delucchi KL, London JA, Chesney MA. Adherence to zidovudine (AZT) among HIV-infected methadone patients: a pilot study of supervised therapy and dispensing compared to usual care. Drug & Alcohol Dependence 1995, 37: 261-269.
21. Vrijens B, Goetghebeur E. Comparing compliance patterns between randomized treatments. Control Clin Trials 1997, 18: 187-203.
22. Carpenter CC, Fischl MA, Hammer SM. et al. Antiretroviral therapy for HIV infection in 1997. Updated recommendations of the International AIDS Society-USA panel. JAMA 1997, 277: 1962-1969.
23. Vanhove GF, Schapiro JM, Winters MA, Merigan TC, Blaschke TF. Patient compliance and drug failure in protease inhibitor monotherapy. JAMA 1996, 276: 1955-1956.
24. Last JM. A Dictionary of Epidemiology. Third ed. New York: Oxford University Press; 1995.
25. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143: 29-36.
26. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983, 148: 839-843.
27. Armitage P, Berry G. Statistical Methods in Medical Research. Second ed. Oxford: Blackwell Scientific Publications; 1987.
28. Dallal GE, Rousseeuw PJ. LMSMVE: a program for least median of squares regression and robust distances. Computers Biomed Res 1992, 25: 384-391.
29. Greenland S. Test for interaction in epidemiologic studies: A review and a study of power. Stat in Med 1983, 2: 243-251.
30. Dupont W, Plummer Jr. W. Power and sample size calculations: a review and computer program. Control Clin Trials 1990, 11: 116-128.
31. Perelson AS, Essunger P, Cao Y. et al. Decay characteristics of HIV-1-infected compartments during combination therapy. Nature 1997, 387: 188-191.
32. Feldman HI, Hackett M, Bilker W, Strom BL. Potential utility of electronic drug compliance monitoring in measures of adverse outcomes associated with immunosuppressive agents. Pharmacoepidemiol Drug Safety 1999, 8: 1-14.
33. Catz SL, Kelly JA, Bogart LM, Benotsch EG, McAuliffe TL. Patterns, correlates, and barriers to medication adherence among persons prescribed new treatments for HIV disease. Health Psychol 2000, 19: 124-133.
34. Chesney MA, Ickovics JR, Chambers DB. et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care. 2000, 12: 255-266.
© 2001 Lippincott Williams & Wilkins, Inc.