Secondary Logo

Journal Logo

Epidemiology and Prevention

Retention Among North American HIV-Infected Persons in Clinical Care, 2000–2008

Rebeiro, Peter ScM*; Althoff, Keri N. PhD, MPH*; Buchacz, Kate PhD, MPH; Gill, John MB, ChB, MSc; Horberg, Michael MD§; Krentz, Hartmut PhD; Moore, Richard MD, MHS; Sterling, Timothy R. MD#; Brooks, John T. MD; Gebo, Kelly A. MD, MPH; Hogg, Robert PhD**; Klein, Marina MD, MSc††; Martin, Jeffrey MD, MPH‡‡; Mugavero, Michael MD, MHSc§§; Rourke, Sean PhD‖‖; Silverberg, Michael J. PhD, MPH¶¶; Thorne, Jennifer MD, PhD##; Gange, Stephen J. PhD*for the North American AIDS Cohort Collaboration on Research and Design

Author Information
JAIDS Journal of Acquired Immune Deficiency Syndromes: March 1st, 2013 - Volume 62 - Issue 3 - p 356-362
doi: 10.1097/QAI.0b013e31827f578a


There are marked disparities in the initial access to, and subsequent retention in, HIV clinical care according to patient demographic and clinical factors.1–8 Retention in care is inherently related to the epidemiological concept of "churn," introduced by Gill and Krentz9 as a framework for describing the continuous movement of individuals entering, reentering, and exiting clinical care at any given point in time. Whereas retention may be viewed as an individual's pattern of clinical care over time and summarized across a population, churn may be viewed as an aggregate measure reflecting differing patterns of retention among different segments of a population over time. Analyzing churn in a longitudinal clinical cohort can complement the analysis of clinical retention by revealing the structure of patient movement that underlies the status of patients described as “retained in care” or not in a given period.

Both the retention in care and the churn phenomenon have implications for the epidemiology and management of HIV in the United States and Canada and the design, implementation, and evaluation of prevention and treatment strategies. Recent national guidelines for Ryan White–funded clinics in the United States define longitudinal medical service utilization and provide a standard by which to quantify retention and churn.10,11 Application of these standards may facilitate the identification of factors associated with a lack of retention, presenting potential targets for intervention and improvement.

In this study, we describe patterns of churn and predictors of incomplete retention in care using data from the North American AIDS Cohort Collaboration on Research and Design (NA-ACCORD), a collaboration of clinical and interval cohort studies of HIV-infected individuals.12 Our goal was to provide both a descriptive overview and an analytically sound quantitation of factors related to retention and churn in the clinical HIV population of the United States and Canada between 2000 and 2008. Furthermore, we provide methods for tracking the future progress in improving retention in HIV care, consistent with the national HIV/AIDS strategies in both countries13,14 and recent consensus practice guidelines.15


Study Design and Population

The NA-ACCORD is a multisite collaboration of interval and clinic-based cohort studies of HIV-infected adults (≥18 years old) receiving care in the United States and Canada.16 NA-ACCORD is one of the multinational cohort studies sponsored by the International Epidemiological Databases to Evaluate AIDS consortium of the National Institutes of Health. Details on the NA-ACCORD collaboration have been published previously.12 Briefly, contributing cohorts have standardized cohort-specific methods of data collection. At scheduled intervals, cohorts submit data regarding enrolled participants' demographic characteristics, vital status, prescribed antiretrovirals, clinical diagnoses, and dates and results of laboratory tests, including HIV-1 RNA viral load and CD4+ lymphocyte count (HIV-lab), whether collected during routine outpatient care or from inpatient or trial settings if available. HIV-lab dates are submitted as dates of specimen collection, and among clinical cohorts, only patients with ≥2 clinical visits within 12 months are enrolled into the NA-ACCORD, therefore limiting the NA-ACCORD clinical population to those patients who have established themselves as "in care" upon cohort entry; this enrollment criterion is assessed by sites themselves and is based on actual clinic encounter data, not on HIV-lab data (on which the rest of our churn analyses are based). All data are transferred securely to the central Data Management Core of NA-ACCORD, where they undergo quality control for completeness and accuracy per a standardized protocol before they are combined into harmonized data files. Quality control includes measures to reduce the probability that an individual was concurrently participating in more than 1 clinical cohort. The activities of both the NA-ACCORD centrally and each of the participating cohort studies to participate in NA-ACCORD have been reviewed and approved by the respective local institutional review boards.

Inclusion Criteria

Only participants in clinical cohorts of the NA-ACCORD who had ≥1 HIV-lab any time between January 2000 and December 2008 were included in this analysis. This allowed us to focus on patterns of care retention using definitions compatible with guidelines issued by the Health Resources and Services Administration (HRSA) HIV/AIDS Bureau (HAB) of the US Department of Health and Human Service for HIV clinical care though not all participants were covered by HRSA HAB programs.10 The 13 included cohorts have clinical sites in 18 US states and territories and 3 Canadian provinces: Alabama, California, Colorado, Florida, Georgia, Illinois, Maryland, Michigan, Missouri, New York, North Carolina, Ohio, Oregon, Pennsylvania, Tennessee, Texas, Washington, Washington DC, Alberta, British Columbia, and Quebec. Contributing cohorts were categorized by care management structure to facilitate comparisons between US and Canadian cohorts; US cohorts were classified as centrally managed if all medical services were available through a single organization (Kaiser Permanente and Veterans Aging Cohort Study) or not (all other contributing clinical sites); the former are more similar to Canadian sites where all patients have access to centrally administered universal health care.


As in other studies,8,17 HIV-lab tests were used as a surrogate for access of care and adherence. For each individual, follow-up began the year of their first HIV-lab after enrollment in the participating cohort study; follow-up ended with their last HIV-lab up to December 31, 2008. As illustrated in Figure 1, individuals were categorized as in care in each calendar year if they had at least 1 HIV-lab >90 days and <12 months after the prior HIV-lab and "out of care" during that calendar year otherwise (regardless of when during the calendar year the prior HIV-lab occurred); this is concordant with the HRSA HAB performance measure for retention in HIV care, though status was determined by elapsed time between HIV-labs and then anchored to calendar time, not determined by HIV-lab collection within a specific quarter of the calendar year as HRSA HAB standards suggest (Fig. 1). These definitions also agree with the “core indicators of HIV clinical care” recommended by the Institute of Medicine of the National Academies of the United States in March 2012.18 Accordingly, incomplete retention in care during the study period was defined as having at least 1 out-of-care calendar year and complete retention as always having in-care calendar years, between the beginning and end of follow-up. Incomplete retention was used as the outcome in regression analyses, allowing identification of factors associated with a risk of falling out of care and thus possible intervention targets for improving retention.

Examples of patterns of retention and churn in HIV clinical care concordant with HRSA HAB guidelines and consequential categorization of contribution to the analysis. Patient A has 3 years “In Care.” Patient B has 2 years In Care and 1 year “Exiting” churn. Patient C has 1 year with no contribution, then 2 years In Care. Patient D has 1 year In Care, then 1 year Exiting churn, and 1 year with no contribution.

Churn is an aggregate measure of individual-level patterns of retention in care, including patients entering, moving out of, returning, remaining out of, or remaining stable in care (Fig. 1). Patients who died during the study period exited in the calendar year of their death and did not contribute beyond that year. Serial cross-sectional studies were nested by calendar year to describe the patterns of churn. The association of risk factors with incomplete retention and the consistency of within-patient behavior were described longitudinally using various regression approaches (described below).

Factors Investigated for Association With Incomplete Retention

We investigated year of birth, self-reported race/ethnicity, HIV transmission risk group, sex, and highly active antiretroviral therapy (ART) use as factors associated with incomplete retention in care. Race/ethnicity was categorized as non-Hispanic black, non-Hispanic white, Hispanic, and other/unknown. HIV transmission risk group was categorized as men who have sex with men, injection drug use (IDU), heterosexual contact, and other/unknown but was dichotomized as IDU vs. non-IDU in regression to avoid collinearity of men who have sex with men with male sex. Patients with both sexual and IDU transmission risks were categorized as IDU. ART was defined as a regimen of ≥3 antiretroviral agents from at least 2 classes or a triple nucleoside/nucleotide reverse transcriptase inhibitor regimen containing Abacavir or Tenofovir. Patients were classified as using ART if ART was prescribed for ≥2 months in a calendar period; measures of ART adherence were not available. CD4+ lymphocyte count and HIV-1 RNA were excluded from regression analyses because of their potential for time-varying mediation between the exposures and outcome of incomplete retention.

Statistical Analyses

Differences by clinical and demographic characteristics were determined by χ2 test for categorical variables and Kruskal–Wallis test for continuous variables. Trends in discontinuity of care during follow-up were evaluated using a generalized estimating equation logistic regression model for proportions remaining in care on calendar year; this trend test accommodated clustering within individuals.

To examine predictors of incomplete retention, multivariable models were constructed to address the likelihood of strong confounding between multiple factors examined and simultaneously obtain appropriately adjusted estimates of effect. Two types of models were fit. First, beta-binomial models were fit separately for those initiating care in 2000 and 2005. These models account for trends in proportions with continuous care in different time frames and minimize potential statistical artifacts of data truncation.19 Second, mixed effects logistic models were also fit with random intercepts for individuals and random slopes by participating cohort site to allow slightly more flexibility in individual trajectories of incomplete retention, both by individual and cohort.20,21 Both models adjusted for total time in care to account for differential times at risk for incomplete retention.

Beta-binomial models for 2000–2004 and 2005–2008 seemed to fit the data best; results for these models are reported here and were similar to those from other models available as supplementary data (see Text, Supplemental Digital Content 1,, which contains details about the statistical approach, fit diagnostics for the results, and applicable code). Analyses were conducted in Stata v. 11.2 (Stata Corporation, College Station, TX) and R v. 2.13.1 (


The subset of patients with ≥1 year out of care was statistically significantly younger (median 41 vs. 42 years) than those in care throughout the study. Those with incomplete retention were also overrepresented compared with patients consistently retained among non-Hispanic blacks (30% vs. ≤23% for all other racial groups), among those with IDU as HIV risk factor (35% vs. ≤25% for all other risk factors), among those who were ART naive at the time of first HIV-lab (29% vs. 23%), and among those who received care in the United States (vs. Canada) (26% vs. 18%), respectively. In addition, the subset of patients with ≥1 year out of care had higher CD4 count (median 347 vs. 319 cells per cubic millimeter) and was in clinical care for a longer period (maximum of 6.03 vs. 2.31 years) than those only contributing to the stably in-care portion of churn because of uninterrupted retention in care (Table 1).

Comparison of Persons With Complete Retention in HIV Care With Those With ≥1 "Out-of-Care" Year in HIV Care Among 61,438 Individuals With ≥1 HIV-Lab in the NA-ACCORD, 2000–2008

There was an improvement in retention in HIV clinical care in North America over calendar time from 2000 to 2008. Of the 61,438 patients meeting inclusion criteria, 25% had one or more out-of-care episodes, and thus incomplete retention, over the entire study period. We observed a significant decrease in proportions of those with incomplete retention (from 39% to 18%), comprised of patients entering and exiting the churn, per calendar year over follow-up (Fig. 2). These trends in the percentages of the population not stably retained, which the churn decomposes further into categories describing the nature of patient movements through care, were sustained even when stratifying contributing cohorts by care management structure: US Centralized, US Decentralized, and Canadian (see Table 1, Supplemental Digital Content 2,, which shows the percentages of patients in various categories of care over calendar time). However, participants in Canadian cohorts were in care in higher proportions from 2000 to 2007 compared with their American counterparts.

Churn” in the NA-ACCORD by calendar year, 2000–2008.

Because we sought to identify possible intervention targets to improve clinical retention in line with national HIV/AIDS strategies, our modeling strategy addressed factors associated with incomplete retention as the outcome, highlighting risk characteristics for falling out of care. In beta-binomial regression, among patients entering care in 2000 and 2005, age per 10-year increase and ART use at the time of first HIV-lab were associated with increased likelihood of complete retention. In contrast, black patients (vs. whites), those with IDU risk (vs. non-IDU), and those with increased time from enrollment in care (per year) had increased likelihood of incomplete retention (Table 2). Female sex was significantly associated with reduced likelihood of incomplete retention among those entering care in 2000 [odds ratios = 0.82, confidence intervals (CI): 0.70 to 0.95], but not among those entering care in 2005 (odds ratios = 1.11, CI: 0.91 to 1.36). There was a high within-individual correlation (ρ = 0.24, CI: 0.22 to 0.26; ρ = 0.20, CI: 0.17 to 0.23), denoting consistency of retention and churn patterns within individuals over time (Table 2).

Characteristics Associated With Incomplete Retention Among Individuals Contributing ≥1 HIV-Lab Between 2000 and 2008

Predicted patterns of retention and churn (ie, proportions of the cohort with 0 out-of-care years, 1 out-of-care year, etc.) derived from the beta-binomial models closely matched observed patterns, indicating the power of these models to account for within-individual correlation of observations over time and overdispersion from the binomial distribution (Fig. 3).

Characterizing individuals with different numbers of “out-of-care” years (or absences from care) over follow-up. Bars reflect observed percentages of individuals with different numbers of absent years; points reflect fitted values from multivariable beta-binomial models.


There were significant increases in both the total proportion and the demographic heterogeneity of the population stably retained in HIV clinical care in the United States and Canada between 2000 and 2008. A concerning level of patients over the study period (25%), however, experienced incomplete retention, highlighting a challenge in delivery of optimal HIV care and the necessity of accounting for patterns of retention and churn when describing the epidemiology of HIV/AIDS in North America.

Population-level trends in Figure 2 suggest that some, but incomplete, progress has been made over the past decade or so in increasing regional stability of retention in care across the United States and Canada. Further analysis is needed to discern possible reasons for this slow but steady improvement. The US National HIV/AIDS Strategy, released in 2010, proposes to both “establish a seamless system to immediately link people to continuous and coordinated quality care when they are diagnosed with HIV” and “increase the proportion of Ryan White HIV/AIDS Program clients who are in continuous care (at least 2 visits for routine HIV medical care in 12 months at least 3 months apart) from 73% to 80%” by 2015.13 The national goals of the Canadian Public Health Association outlined in 2005 also note the importance of sustained clinical care, though it does not specify a concrete threshold for improvement.14

As noted elsewhere, changes in the constitutive HIV population of a region over time can alter the prevalence and severity of disease among different at-risk groups, necessarily altering ideal targets for intervention, resource allocation, and assessment of prevention efficacy.9 These issues and the obvious health benefits of clinical care for patients remain important considerations even as national HIV/AIDS strategies set forth ambitious goals for patient engagement and retention in care, and the Centers for Disease Control and Prevention notes the impact of suboptimal retention on prevention of HIV transmission.7,8,13,14,18

Differences in incomplete retention identified by sex, race/ethnicity, ART use, and HIV risk present clear targets for interventional efforts to improve retention in HIV clinical care and advance these goals. The disappearance of sex as a significant factor in multivariable analyses restricted to the last 4 years of the study period suggests that disparities in retention by sex may be narrowing, though it is possible that unmeasured confounding factors affected this observation. Additionally, associations of increased “time in care” with increased likelihood of incomplete retention may imply a need to employ rigorous methods to encourage routine follow-up and reengagement even among care-experienced patients. Differences in proportions with incomplete retention between the United States and Canada throughout the study period may also deserve further scrutiny.22

Disconnecting from regular HIV care has been shown to have major clinical consequences with regard to immune health and virological failure.2–5 However, the observed CD4 count/HIV-1 RNA viral load patterns in this analysis seem to differ from observations in the literature, with higher CD4 counts among those with interruptions in care and no difference in HIV-1 RNA viral load between those who were consistently retained and those who were not. This is likely because of incomplete retention being an outcome here, not an exposure. Prior studies treated retention in care as the exposure and CD4 count/HIV-1 RNA viral load as outcomes and so observed that patients with out-of-care periods during follow-up often returned to care with lower CD4 count and higher HIV-1 RNA viral load, as one might expect.1,3 Such markers of disease progression were not included as time-varying covariates in this analysis because retention in care likely influences health status and health status likely influences retention in care. Quantification of potential differences in clinical outcomes by churn and retention status may be a logical next step to these analyses but will require more sophisticated methods that can account for time-updated mediation and causally interrelated factors such as these.

This study has structural and statistical limitations. Because visit data are not currently available throughout NA-ACCORD, HIV-labs were used as surrogates for accessing clinical care. Though this may have resulted in slight over- or underestimates of true health care access and absence of incomplete retention defined using laboratory access may not imply retention in clinical care, HIV-labs are themselves markers of care subject to clinical guidelines for annual frequency and have been used previously in similar analyses.8,15,18 The group with incomplete retention comprised, by definition, those that reengaged in care subsequent to being out of care but before the end of the study. This pattern may differ from the longitudinal view of visit contributions to follow-up in a time-to-event analysis; contributions to churn could be viewed more appropriately as successive cross-sectional slices of care patterns in a region over time, contrasted with longitudinal analyses where individuals can be "lost to follow-up" for the remainder of the study after exiting care. Considering these distinctions and the characteristics of entrants to the NA-ACCORD (enrollment with ≥2 visits in a 12-month period), observed patterns in these analyses may represent overestimates of proportions retained in care over the study period. Furthermore, NA-ACCORD cannot track the movement of individual patients between clinical cohorts, so it is possible that patients who “exited” the churn in a given period (or did not return to care and were subsequently not counted) actually accessed care within another clinic, private practice, public health department, or elsewhere in the United States or Canada. Because of the de-identified nature of the merged clinical data available for analysis, this issue may not be resolvable, though large geographic dispersion of sites within the NA-ACCORD may mitigate it. Finally, even though models capable of accounting for overdispersion from a binomial distribution and within-individual clustering of observations were used, they entailed fitting predictions to an observed distribution of out-of-care period counts that were highly right-skewed and flat; even so, deviations of predicted counts and probabilities from observed proportions with discontinuities in retention were not dramatic.

Despite these limitations, the analysis undertaken was an important first step toward quantifying churn and patterns of retention among large segments of the clinical HIV population in North America; in particular, the methods used were able to capture within-person tracking of care access over time and discern groups at higher risk of gaps in clinical care such as young persons, black individuals, and those with IDU HIV transmission risk or not accessing ART, groups which may benefit from enhanced retention efforts. Additional analyses indicate that patients in US clinical cohorts within NA-ACCORD have similar demographics to HIV cases captured in the National HIV surveillance system,23 supporting the notion that our findings may reflect trends in the larger adult population living with HIV.

As implied by the significant portion of this study population exiting or entering in the churn in each calendar year and the demographic and clinical differences between patients retained in care and those who are not across the study period, cross-sectional analyses attempting to address policy questions by quantifying disease burden or other health metrics may suffer from fundamental measurement issues and may therefore not offer an accurate picture of the clinical HIV-infected population at any one time. Therefore, the churn phenomenon will continue to play an important role in the epidemiology of HIV/AIDS in the United States and Canada and, in light of renewed focus on uninterrupted care for the improvement of patient health outcomes and the interruption of transmission, will continue to serve as a benchmark indicator of progress toward greater stability in clinical retention among the HIV population of North America. This topic therefore merits further analysis with this and other study populations and similarly sophisticated epidemiological methods capable of rendering valid estimates under the conditions of a dynamic population that is in churn.


The authors would like to thank all individuals involved with the NA-ACCORD collaboration, including staff, investigators, and patients, for their invaluable contributions to this work.


1. Stone VE. HIV/AIDS in women and racial/ethnic minorities in the U.S. Curr Infect Dis Rep. 2012;14:53–60.
2. Berg MB, Safren SA, Mimiaga MJ, et al.. Nonadherence to medical appointments is associated with increased plasma HIV RNA and decreased CD4 cell counts in a community-based HIV primary care clinic. AIDS Care. 2005;17:902–907.
3. Giordano TP, Gifford AL, White AC Jr, et al.. Retention in care: a challenge to survival with HIV infection. Clin Infect Dis. 2007;44:1493–1499.
4. Mugavero MJ, Lin HY, Allison JJ, et al.. Racial disparities in HIV virologic failure: do missed visits matter? J Acquir Immune Defic Syndr. 2009;50:100–108.
5. Mugavero MJ, Lin HY, Willig JH, et al.. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis. 2009;48:248–256.
6. Ulett KB, Willig JH, Lin HY, et al.. The therapeutic implications of timely linkage and early retention in HIV care. AIDS Patient Care STDS. 2009;23:41–49.
7. CDC. Vital signs: HIV prevention through care and treatment—United States. MMWR. 2011;60:1618–1623.
8. Hall HI, Gray KM, Tang T, et al.. Retention in care of adults and adolescents living with HIV in 13 U.S. areas. J Acquir Immune Defic Syndr. 2012;60:77–82.
9. Gill MJ, Krentz HB. Unappreciated epidemiology: the churn effect in a regional HIV care programme. Int J STD AIDS. 2009;20:540–544.
10. US Department of Health and Human Services, Health Resources and Services Administration HIV/AIDS Bureau. HAB HIV Performance Measures: Medical Case Management. Available at: Accessed November 23, 2009.
11. Dombrowski JC, Kent JB, Buskin SE, et al.. Population-based metrics for the timing of HIV diagnosis, engagement in HIV care, and virologic suppression. AIDS. 2012;26:77–86.
12. Gange SJ, Kitahata MM, Saag MS, et al.. Cohort profile: the North American AIDS Cohort Collaboration on Research and Design (NA-ACCORD). Int J Epidemiol. 2007;36:294–301.
13. The White House Office of National AIDS Policy. The National HIV/AIDS Strategy. Washington, DC: The White House; 2010.
14. The Canadian Public Health Association. Leading Together: Canada Takes Action on HIV/AIDS (2005–2010). Ottawa, Ontario, Canada: The Public Health Agency of Canada; 2005.
15. Thompson MA, Mugavero MJ, Amico KR, et al.. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Ann Intern Med. 2012;156:817–833.
16. Lau B, Gange SJ, Moore RD. Interval and clinical cohort studies: epidemiological issues. AIDS Res Hum Retroviruses. 2007;23:769–776.
17. Mugavero MJ, Davila JA, Nevin CR, et al.. From access to engagement: measuring retention in outpatient HIV clinical care. AIDS Patient Care STDS. 2010;24:607–613.
18. Institute of Medicine of the National Academies. Monitoring HIV Care in the United States, Indicators and Data Systems. [Table 1: ”Core Indicators of HIV Clinical Care”]. Washington, DC: National Academies Press; 2012.
19. Gange SJ, Muñoz A, Saez M, et al.. Use of the beta-binomial distribution to model the effect of policy changes on appropriateness of hospital stays. J R Stat Soc Ser C Appl Stat. 1996;45:371–382.
20. Xue X, Gange SJ, Zhong Y, et al.. Marginal and mixed-effects models in the analysis of human papillomavirus natural history data. Cancer Epidemiol Biomarkers Prev. 2010;19:159–169.
21. Skrondal A, Rabe-Hesketh S. Some applications of generalized linear latent and mixed models in epidemiology: repeated measures, measurement error and multilevel modeling. Nor Epidemiol. 2003;13:265–278.
22. Krentz HB, Siemieniuk RA, Gill MJ. Similar challenges with retention in care issues. Clin Infect Dis. 2007;45:1527.
23. Althoff KN, Buchacz K, Hall HI, et al.. U.S. trends in antiretroviral therapy use, HIV RNA plasma viral loads, and CD4 T-lymphocyte cell counts among HIV-infected persons, 2000 to 2008. Ann Intern Med. 2012;157:325–335.

retention; churn; HIV clinical care; North America; HRSA HAB; National HIV/AIDS Strategy

Supplemental Digital Content

© 2013 Lippincott Williams & Wilkins, Inc.