JAIDS Journal of Acquired Immune Deficiency Syndromes:
Heterogeneity Among Studies in Rates of Decline of Antiretroviral Therapy Adherence Over Time: Results From the Multisite Adherence Collaboration on HIV 14 Study
Wilson, Ira B. MD, MSc*; Bangsberg, David R. MD, MPH†; Shen, Jie PhD‡; Simoni, Jane M. PhD§; Reynolds, Nancy R. PhD‖; Goggin, Kathy PhD¶; Gross, Robert MD, MSCE#; Arnsten, Julia H. MD, MPH**; Remien, Robert H. PhD††; Erlen, Judith A. PhD‡‡; Liu, Honghu PhD*; for the Multisite Adherence Collaboration on HIV 14 Investigators
*Department of Health Services, Policy & Practice, Brown University School of Public Health, Brown University, Providence, RI;
†Massachusetts General Hospital, Center for Global Health; Ragon Institute of MGH, MIT, and Harvard; Harvard Medical School, Boston, MA;
‡Division of Public Health & Community Dentistry, University of California at Los Angeles School of Dentistry, University of California at Los Angeles, Los Angeles, CA;
§Department of Psychology, University of Washington, Seattle, WA;
‖School of Nursing, Yale University, New Haven, CT;
¶Division of Health Services and Outcomes Research, Children's Mercy Hospital and Clinics, University of Missouri – Kansas City, Kansas City, MO;
#Department of Medicine (Infectious Diseases), Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA;
**Department of Medicine, Albert Einstein College of Medicine, Montifiore Hospital, Bronx, NY;
††Department of Psychiatry, Columbia University, New York City, NY; and
‡‡Department of Health and Community Systems, University of Pittsburgh School of Nursing, Pittsburgh, PA.
Correspondence to: Ira B. Wilson, MD, MSc, Department of Health Services, Policy & Practice, Brown University School of Public Health, Brown University, G-121-7, 121 South Main St, Providence, RI 02912 (e-mail: firstname.lastname@example.org).
Supported by the Multisite Adherence Collaboration in HIV (MACH14) grant R01MH078773 from the National Institute of Mental Health, Office on AIDS. The original grants of individual participating studies are R01DA11869, MH54907, 2R01NR04749, R01NR04749, R01MH068197, R01 DA13826, K23MH01862, MH01584, R01 AI41413, R01 MH61173, NIH/NIAID AI38858, AI069419, K02 DA017277, R01DA15215, National Institute of Mental Health P01 MH49548, MH58986, RO1MH61695, CC99-SD-003, CC02-SD-003, and R01DA015679. The content of the paper is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
The authors have no conflicts of interest to disclose.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (www.jaids.com).
Received February 26, 2013
Accepted October 01, 2013
Objective: To use electronic drug monitoring to determine if adherence to HIV antiretroviral therapy (ART) changes over time, whether changes are linear, and how the declines vary by study.
Design: We conducted a longitudinal study of pooled data from 11 different studies of HIV-infected adults using ART. The main outcome was ART adherence (percent of prescribed doses taken) measured by electronic drug monitoring. We modeled and compared changes in adherence over time using repeated measures linear mixed effects models and generalized additive mixed models (GAMMs). Indicator variables were used to examine the impact of individual studies, and the variation across studies was evaluated using study-specific parameter estimates calculated by using interaction terms of study and time.
Results: The mean age of the subjects was 41 years, 35% were female, most had high school education or less, and 46% were African American. In GAMMs, adherence declined over time. The GAMMs further suggested that the decline was nonlinear, and in both sets of models, there was considerable study-to-study variability in how adherence changed over time.
Limitations: Findings may not be generalizable to non-US populations or to patients not in clinical studies.
Conclusions: Although overall ART adherence declined with time, not all studies showed declines, and a number of patterns of change were seen. Studies that identify clinical and organizational factors associated with these different patterns are needed. Models of changes in adherence with time should take account of possible nonlinear effects.
There is debate about the natural history of adherence to HIV antiretroviral therapy (ART) over time. Most studies have shown that ART adherence declines with time,1–9 but some suggest that adherence is stable over time,10–12 and still others show improvements over time.13–17 There are a number of potential explanations for these discrepant findings, including differences in patient populations, country, and/or setting, whether study populations are ART naive versus experienced at enrollment and whether the studies were in the context of interventions, methods used to assess adherence, and analytic approaches (eg, methods to account for missing data and losses to follow-up).
To better understand the natural history of ART adherence in the United States, we used data from the Multisite Adherence Collaboration on HIV study (MACH14). MACH14 conducts pooled analyses with individual subject data from 16 studies drawn from 14 different research groups in 12 states, all of which used electronic data monitoring (EDM). Three main study questions were addressed by this research. First, what is the natural history of changes in antiretroviral (ARV) adherence over time? Second, are these changes linear? Third, do these changes differ by study?
MACH14 is a multisite collaboration. Each of the 16 studies that make up this collaboration was approved by that institution’s institutional review board.
Studies and Patient Selection
MACH14 included both observational (n = 4) and intervention (n = 12) studies. The process is described in detail elsewhere.18–21 To be included in this analysis, monitoring had to be continuous and data on whether patients were ART naive had to be available. This excluded 4 studies that assessed adherence only in the weeks before a study visit and 1 study without ART naive information, leaving 11 eligible studies. We use the term “study” and not “site” to avoid confusion between the sites where the study took place and the sites where patients received HIV clinical care, which was not necessarily the same for the studies that recruited patients from multiple care sites.
Because we were interested in the natural history of changes in ART adherence over time, we only included control patients in adherence intervention trials. Of the 1456 patients in the 11 eligible studies, 916 were either in observational studies or in control arms of intervention studies and formed the analytic sample. The number of patients, intervention status, and length of follow-up for each of the 11 included studies are shown in Table S1 (see Supplemental Digital Content, http://links.lww.com/QAI/A475). Patients were followed for up to 12 months.
We defined the dependent variable as ART adherence. For each patient, the time of observation was divided into 1-month periods. Adherence was operationalized as the number of observed openings divided by the number of prescribed doses. Typically 1 ARV was monitored using EDM, but when more than 1 was monitored, we calculated an average from all monitored medications over the month. Adherence was summarized by month across the whole study population and also by study.
The major independent variable was time measured in months. Up to 12 months of observations were assessed for each study. Studies varied in the amount of time observed. The shortest study followed patients for 3 months, 1 study followed patients for 4 months, 2 studies followed patients for 6 months, 2 studies followed patients for 9 months, and 6 studies followed patients for 12 months.
Covariates included age in years, gender, education (high school graduate or less vs. more than high school education), race (white, African American, Hispanic, or others), ARV regimen [classified as nonnucleoside reverse transcriptase inhibitor (NNRTI) based, protease inhibitor (PI) based, boosted PI based, or others], history of substance abuse (yes/no), and whether the patient was ART naive (yes/no)].
We plotted adherence by month and then adherence by month for each individual study to visualize between study differences. Because these graphs suggested that there were considerable between study differences and also that the relationship might be nonlinear, next we modeled the relationship between adherence and time using a cubic spline model with 12 knots. To estimate the possible underlying nonlinear relationships, we used the generalized additive mixed model (GAMM) that is an extension of generalized linear mixed models. GAMM relaxes the assumptions of normality and linearity inherent in linear regression and allows the parametric fixed effects to be modeled nonparametrically using additive smooth functions. The flexibility of nonparametric regression for the continuous predictors coupled with linear models for predictors provides a way to uncover structure within the data that may be missed using linear assumptions. GAMM modeling was carried out using the R Statistical Language and the MGCV Package.22 The model allows the assessment of the nonlinearity of trend in the adherence curve, which is shown by the effective degrees of freedom (EDF) term from the model. An EDF of 1.0 denotes linearity, and values greater than 1.0 indicate nonlinearity. The higher the EDF, the more nonlinear is the smoothing spline. The model also allows for assessment of the directional trend of the curve, that is, whether it is downward, flat, or upward trending.
We used a manual forward selection approach to select covariates, starting with a model that included study and a study by time interaction, and then sequentially testing the following variables: race, calendar year, regimen, ART naive, and ART naive by regimen. In each case, we used the likelihood ratio test to determine whether the added variable contributed significantly (P < 0.05) to the model. For model diagnostics, we assessed (1) normality (the QQ plot and the histogram of residuals), (2) homogeneity (residuals vs. predictor plot and residuals vs. fitted values plot, also called the linear predictor plot for the Gaussian distribution with identity link), and (3) model fitting (fitted values vs. observed values plot).
Formal modeling of missing data in longitudinal studies is complex.23 We were interested in determining whether those who dropped out of the 11 studies in our analysis were different from those who did not drop out or, more formally, whether they were missing not at random.24 To test this, we estimated a Cox proportional hazards model with dropout as the dependent variable. Note that “dropout” here refers to study dropout not treatment dropout (ie, ARV nonpersistence). The principal independent variable in the model was medication adherence. For those who did not drop out, we used adherence in the last study month, and for those who did drop out, we used adherence in the month before dropout. Covariates included the same variables that were used in the models in which adherence was the dependent variable. This approach allows us to determine whether adherence before dropout predicted drop out.
Of the 916 patients studied, mean age was 41 years and 35% were female (Table 1). Twenty-seven percent were white, 46% African American, 21% were Hispanic, and 6% were classified as others. Eight-eight percent had a high school education or less. Seventy-seven percent had a history of substance abuse. In approximately 27%, an NNRTI was the monitored ARV; in 17%, it was a boosted PI; in 28%, it was a PI; and in 28%, it was an ARV classified as others.
Changes in Adherence Over Time
Unadjusted changes in adherence by month are shown in Figure 1 (all 11 studies aggregated) and Figure 2 (each study individually). The base GAMM model that included only the spline terms (12 knots) is shown in Figure 3. The EDF of the smoothed term for month in this model was 4.63, signifying high nonlinearity. The model shows that the curve is down sloping (F = 7.3, P < 0.0001). The R2 of this base model was <0.01.
Variables that were statistically significant in the final multivariable GAMM model included study, the study by time interaction, race, and calendar year. The R2 of this final model was 0.14. These results are shown graphically in Figure 4, where the fitted adherence values for each study are plotted against time. The scale of both axes varies for each plot to allow appreciation of the study-by-study variation. For 3 of the studies (7,10, and 12), the EDF was 1.0, denoting a linear relationship between adherence and time. For 2 of these studies (7 and 12), there was also a significant downward trend (P < 0.0001), but for study 10, adherence did not decline (P > 0.05). The relationship of adherence to time was nonlinear for the remaining studies. For 6 of these (studies 1, 2, 8, 11, 13, and 15), there was a significant downward trend (P < 0.05), and for 2 of these studies (5 and 6), there was no significant change in adherence over time (P > 0.05). Model diagnostics, including normality, heterogeneity, and fit, were acceptable. In the final model, African Americans had significantly worse adherence compared with whites (7.9 points, P = 0.0013). Calendar year was also significantly associated with adherence. Compared with year 1998, adherence was significantly better for years 1999 through 2006 and 2008 but was not significantly different for 2007.
We also tested a model that included an indicator variable for whether the study was a randomized trial or an observational study. The P value for the indicator was 0.72.
Bias and Missingness Analysis
In a Cox proportional hazards model predicting study dropout that controlled for ARV regimen and study, adherence was not associated with dropout (hazard ratio: 1.0, 95% CI: 0.74–1.33, P = 0.97).
There were 3 main findings from this research. First, across the 11 studies we studied, adherence declined with time, but this overall effect hides the fact that there was heterogeneity among studies in this decline. Second, the relationship between adherence and time was nonlinear both overall and in the majority of individual studies. Third, covariate adjustment did not eliminate this heterogeneity.
Many of the previously published studies that address changes in adherence over time included patients from multiple clinical care sites.3,5,7,9,10,13,14,17 Only 2 of these studies included site as a variable in analyses. Maqutu et al13 studied 2 sites in South Africa and found that the rural site had adherence levels that were approximately 20 percentage points lower at study initiation than the urban site. They also found that the rate of increase in adherence in the rural site was significantly higher than that seen in the urban site (adjusted odds ratio for the rate of change was 1.06, P < 0.004), such that both sites had approximately 90% adherence after 18 months of follow-up. Muyingo et al14 studied 4 sites in Uganda and Zimbabwe. Although they did not present data on changes over time by site, 2 of the sites had significantly higher (P < 0.001) baseline adherence than the reference site.
Our data in combination with these other 2 studies that examined adherence levels by site suggest that aggregate data may often hide important study-level variation. This is not surprising. The 11 studies that we analyzed included both observational and intervention studies, and we have only limited data on the nature of adherence support for patients in these studies. Although aggregated data are important for policy discussions, quality improvement requires care site-specific data. That is, because historical factors, available resources, structural aspects of the delivery system, patient characteristics, and even provider incentives can differ widely from site to site, analyses of correlates or predictors of adherence and changes in adherence over time may yield site-specific answers. We cannot know from these data why adherence in some studies declined with time and other studies did not. Answering this important question would require a study that collected a more extensive list of potential explanatory variables and that identified and characterized the care sites from which patients were recruited.
Overall, the relationship between adherence and time was nonlinear, and it was nonlinear for most of the studies. Existing studies use a variety of approaches to understanding changes in adherence with time, including descriptive measures,5–8,11,12 logistic regression, often using generalized estimating equations to account for repeated measures,1–3,9,10,13–17 and survival analysis.4 Generalized estimating equations work on the population level, but their estimates of associations in smaller data sets, are inefficient and sometimes inconsistent. Generalized linear mixed models can model the adherence change on an individual level, but GAMMs allow flexible and nonlinear specification of the dependence of the response variable on a set of temporal and/or spatial covariates without having to specify the model in terms of detailed parametric relationships, which made them more useful for the purposes of this analysis.
Another advantage of using models that capture nonlinearity is that we can address other hypotheses about how adherence as measured by EDM changes over time. For example, another possible interpretation for the decline in adherence that we saw is that in studies that use EDM, there may be a Hawthorne effect. That is, participants who start using EDM have higher adherence than they would otherwise have by virtue of the fact that they know that their adherence is being measured or observed. Some studies support the existence of such an effect,25–28 but others do not.29 A pattern of adherence in which there was an increase in adherence followed by a decline (eg, studies 6 and 8; Fig 4) or steeper declines in adherence in the first several months of treatment than what was seen later (studies 5, 13, and 15; Fig 4) would support the existence of Hawthorne effects. Thus, our data provide only limited support for the assertion that Hawthorne effects are necessarily seen when EDM is used.
We were not surprised that adjustment for race and other covariates did not eliminate differences between sites. The finding that adherence was lower in African Americans than whites has been demonstrated in multiple other reports30–35 including one from the MACH14 cohort.21
There is variability in how previous researchers have dealt with loss to follow-up. Some studies either do not mention loss to follow-up or note that those lost to follow-up were excluded from analyses.1–4,7,9,10,12,14,15,17 Others presented descriptive information about rates of loss to follow-up or missing data, in some way compare the full sample of those with missing data with those who had complete data or no loss to follow-up.5,6,8,11,16 Only 1 study explicitly analyzed loss to follow-up.13 In our study, and also in Maqutu et al,13 analyses suggested that missing adherence values were missing completely at random and therefore do not bias analyses, but this is a strong assumption, and we recommend that longitudinal studies of adherence formally examine the potential impact of loss to follow-up on their findings.
There were several study limitations. Regarding internal validity, we could not adjust for time on ART before study entry, which, if it varied systematically between studies, could explain some of the observed variation. We also could not adjust for potentially important patient-level covariates including depression, income, social support, and patient–provider relationship quality because these variables were not measured in all studies. Regarding generalizability, we studied a predominately treatment-experienced cohort cared for in the United States and included only patients who agreed to use EDM. Thus, our findings may not be generalizable to other populations or care settings. The studies we included were conducted over a time period during which there were important changes in ART regimens, particularly the addition of boosted PIs and NNRTIs. However, our analyses controlled for calendar year, and regimen type was not associated with changes in adherence over time in our models, so we do not believe that this is an important limitation. Finally, the studies we analyzed were potentially different on multiple dimensions, which we did not analyze, including the focus of the study (eg, adherence in patients receiving methadone therapy), entry criteria (eg, if only those with detectable viral loads were eligible to participate in an intervention trial), the nature of adherence support in study control arms, and the number of clinical care sites from which they recruited patients.
In conclusion, we found that, overall, adherence declined with time, but this was not consistent across studies. Studies that identify clinical and organizational factors associated with these different patterns are needed. Because this variability was not explained by the covariates we assessed, studies that look more carefully at how specific care processes may vary in the populations recruited by individual studies (eg, availability of mental health services) are warranted. In conducting longitudinal analyses of adherence, our analysis suggests that investigators should take account of site and/or study effects and consider the use of nonlinear models.
The authors would like to thank all the patients who participated in each of the individual studies.
1. Byakika-Tusiime J, Crane J, Oyugi JH, et al.. Longitudinal antiretroviral adherence in HIV+ Ugandan parents and their children initiating HAART in the MTCT-Plus family treatment model: role of depression in declining adherence over time. AIDS Behav. 2009;13(suppl 1):82–91.
2. Liu H, Miller LG, Hays RD, et al.. Repeated measures longitudinal analyses of HIV virologic response as a function of percent adherence, dose timing, genotypic sensitivity, and other factors. J Acquir Immune Defic Syndr Hum Retrovirol. 2006;41:315–322.
3. Mannheimer S, Friedland G, Matts J, et al.. The consistency of adherence to antiretroviral therapy predicts biologic outcomes for human immunodeficiency virus-infected persons in clinical trials. Clin Infect Dis. 2002;34:1115–1121.
4. Lima VD, Harrigan R, Bangsberg DR, et al.. The combined effect of modern highly active antiretroviral therapy regimens and adherence on mortality over time. J Acquir Immune Defic Syndr. 2009;50:529–536.
5. Tesoriero J, French T, Weiss L, et al.. Stability of adherence to highly active antiretroviral therapy over time among clients enrolled in the treatment adherence demonstration project. J Acquir Immune Defic Syndr. 2003;33:484–493.
6. Gross R, Bilker WB, Friedman HM, et al.. Effect of adherence to newly initiated antiretroviral therapy on plasma viral load. AIDS. 2001;15:2109–2117.
7. Lazo M, Gange SJ, Wilson TE, et al.. Patterns and predictors of changes in adherence to highly active antiretroviral therapy: longitudinal study of men and women. Clin Infect Dis. 2007;45:1377–1385.
8. Lucas GM, Gebo KA, Chaisson RE, et al.. Longitudinal assessment of the effects of drug and alcohol abuse on HIV-1 treatment outcomes in an urban clinic. AIDS. 2002;16:767–774.
9. Carrieri P, Cailleton V, Le Moing V, et al.. The dynamic of adherence to highly active antiretroviral therapy: results from the French National APROCO cohort. J Acquir Immune Defic Syndr. 2001;28:232–239.
10. Lima V, Fernandes K, Rachlis B, et al.. Migration adversely affects antiretroviral adherence in a population-based cohort of HIV/AIDS patients. Soc Sci Med. 2009;68:1044–1049.
11. Moss AR, Hahn JA, Perry S, et al.. Adherence to highly active antiretroviral therapy in the homeless population in San Francisco: a prospective study. Clin Infect Dis. 2004;39:1190–1198.
12. Arnsten JH, Demas PA, Farzadegan H, et al.. Antiretroviral therapy adherence and viral suppression in HIV-Infected drug users: comparison of self-report and electronic monitoring. Clin Infect Dis. 2001;33:1417–1423.
13. Maqutu D, Zewotir T, North D, et al.. Determinants of optimal adherence over time to antiretroviral therapy amongst HIV positive adults in South Africa: a longitudinal study. AIDS Behav. 2011;15:1465–1474.
14. Muyingo SK, Walker AS, Reid A, et al.. Patterns of individual and population-level adherence to antiretroviral therapy and risk factors for poor adherence in the first year of the DART trial in Uganda and Zimbabwe. J Acquir Immune Defic Syndr. 2008;48:468–475.
15. Cambiano V, Lampe FC, Rodger AJ, et al.. Long-term trends in adherence to antiretroviral therapy from start of HAART. AIDS. 2010;24:1153–1162.
16. Cooper V, Gellaitry G, Hankins M, et al.. The influence of symptom experiences and attributions on adherence to highly active anti-retroviral therapy (HAART): a six-month prospective, follow-up study. AIDS Care. 2009;21:520–528.
17. Glass TR, Battegay M, Cavassini M, et al.. Longitudinal analysis of patterns and predictors of changes in self-reported adherence to antiretroviral therapy: Swiss HIV Cohort Study. J Acquir Immune Defic Syndr. 2010;54:197–203.
18. Wagner GJ, Goggin K, Remien RH, et al.. A closer look at depression and its relationship to HIV antiretroviral adherence. Ann Behav Med. 2011;42:352–360.
19. Liu H, Wilson IB, Goggin K, et al.. MACH14: a multi-site collaboration on ART adherence among 14 institutions. AIDS Behav. 2013;17:127–141.
20. Genberg BL, Wilson IB, Bangsberg DR, et al.. Patterns of antiretroviral therapy adherence and impact on HIV RNA among patients in North America. AIDS. 2012;26:1415–1423.
21. Simoni JM, Huh D, Wilson IB, et al.. Racial/ethnic disparities in ART adherence in the United States: findings from the MACH14 study. J Acquir Immune Defic Syndr. 2012;60:466–472.
22. Wood SN. Generalized Additive Models. An Introduction With R. Boca Raton, FL: Chapman & Hall; 2006.
23. Daniels MJ, Hogan JW. Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. Boca Raton, FL: Chapman & Hall/CRC; 2008.
24. Little RJ, Rubin DB. Statistical Analysis With Missing Data. Vol 539. New York, NY: Wiley; 1987.
25. Elixhauser A, Eisen SA, Romeis JC, et al.. The effects of monitoring and feedback on compliance. Med Care. 1990;28:882–893.
26. Deschamps AE, Graeve VDE, van Wijngaerden E, et al.. Prevalence and correlates of nonadherence to antiretroviral therapy in a population of HIV patients using Medication Event Monitoring System. AIDS Patient Care STDS. 2004;18:644–657.
27. Cramer J, Rosenheck R, Kirk G, et al.. Medication compliance feedback and monitoring in a clinical trial: predictors and outcomes. Value Health. 2003;6:566–573.
28. Burnier M, Schneider MP, Chioléro A, et al.. Electronic compliance monitoring in resistant hypertension: the basis for rational therapeutic decisions. J Hypertens. 2001;19:335–341.
29. Wagner GJ, Ghosh-Dastidar B. Electronic monitoring: adherence assessment or intervention? HIV Clin Trials. 2002;3:45–51.
30. Hartzell JD, Spooner K, Howard R, et al.. Race and mental health diagnosis are risk factors for highly active antiretroviral therapy failure in a military cohort despite equal access to care. J Acquir Immune Defic Syndr. 2007;44:411–416.
31. Kleeberger CA, Buechner J, Palella F, et al.. Changes in adherence to highly active antiretroviral therapy medications in the Multicenter AIDS Cohort Study. AIDS. 2004;18:683–688.
32. Pence BW, Ostermann J, Kumar V, et al.. The influence of psychosocial characteristics and race/ethnicity on the use, duration, and success of antiretroviral therapy. J Acquir Immune Defic Syndr. 2008;47:194–201.
33. Sullivan PS, Campsmith ML, Nakamura GV, et al.. Patient and regimen characteristics associated with self-reported nonadherence to antiretroviral therapy. PLoS One. 2007;2:e552.
Oh DL, Sarafian F, Silvestre A, et al.. Evaluation of adherence and factors affecting adherence to combination antiretroviral therapy among White, Hispanic, and Black men in the MACS Cohort. J Acquir Immune Defic Syndr. 2009;52:290–293.
Silverberg MJ, Leyden W, Quesenberry CP Jr, Horberg MA. Race/ethnicity and risk of AIDS and death among HIV-infected patients with access to care. J Gen Intern Med. 2009;24:1065–1072.
HIV; antiretroviral therapy; medication adherence; longitudinal studies; pooled analysis
Supplemental Digital Content
© 2013 by Lippincott Williams & Wilkins
Highlight selected keywords in the article text.