In July 2010, the National HIV/AIDS Strategy (NHAS) established specific goals for the response of the United States to the HIV epidemic, including reductions in new infections, improvements in access to high-quality care and improved health outcomes among people living with the disease, and reductions in HIV-related health disparities.1 These objectives align with scientific research highlighting the critical role of prompt HIV diagnosis, linkage to care, and initiation of antiretroviral therapy (ART). Since the introduction of the NHAS and its detailed implementation plan, significant progress has been made toward achieving the strategy's goals, including a more coordinated national response by HIV/AIDS programs across multiple federal agencies. On July 15, 2013, the White House Office of National HIV/AIDS Policy introduced the Accelerating Improvements in HIV Prevention and Care in the United States through the HIV Care Continuum initiative, which builds on the NHAS to improve outcomes along the continuum from HIV diagnosis to successful retention in HIV care (ie, the “HIV care continuum”).2
The Enhanced Comprehensive HIV Prevention Planning (ECHPP) initiative of the Centers for Disease Control and Prevention for the 12 US jurisdictions most affected by HIV is a central part of the response to the NHAS.3 This program involves the local planning and subsequent implementation of a combination of 14 required HIV-prevention interventions and several optional components. The ECHPP initiative holds the promise of significantly advancing our understanding of the barriers and facilitators to comprehensive HIV prevention and treatment and evaluating the initiative is crucial to elucidating best practices for realizing the goals of the NHAS. National Institutes of Health supported ECHPP evaluation efforts by supplementing the Centers for AIDS Research (CFAR) to enhance collaborations between the National Institutes of Health–funded clinical and behavioral investigators and local public health department officials implementing and evaluating the ECHPP initiative.
In this article, we describe our current collaboration's process to select optimal measures of linkage to care in response to the ECHPP program and to understand the implications of measure selection and the challenges of accessing and using multiple data sources to measure outcomes along the HIV care continuum.
THE SAN FRANCISCO CFAR ECHPP COLLABORATION
San Francisco has a long history of collaborative efforts across local community-based organizations, community advisory boards and planning councils, academic research institutions, clinical care providers, and branches of the San Francisco Department of Public Health (SFDPH).4–7 The ECHPP San Francisco effort built upon this existing network by including additional investigators with clinical and behavioral expertise in measuring linkage to care from the UCSF Center for AIDS Prevention Studies, a behaviorally focused, the National Institute of Mental Health–sponsored national AIDS research center; the UCSF–Gladstone Institute of Virology and Immunology CFAR, a National Institute of Allergy and Infectious Diseases-funded center for basic and clinical research; and HIV care providers at San Francisco General Hospital's Positive Health Program.
SAN FRANCISCO PREVENTION AND TREATMENT ENVIRONMENT
San Francisco, one of 12 ECHPP sites, has 15,705 living HIV/AIDS cases. This constitutes 13% of California's living HIV/AIDS cases and 2% of living HIV/AIDS cases nationwide.8 The current San Francisco HIV Prevention Strategy is a comprehensive, multilevel combination prevention, care, and treatment approach.9 Preceding ECHPP, there have been efforts to expand coverage and frequency of HIV testing for men who have sex with men (MSM) and other populations at risk for HIV and to institute earlier initiation of ART.10–12 San Francisco has implemented key evidence-based interventions using antiretroviral drugs, including universal offer of ART,11 nonoccupational postexposure prophylaxis,13 and preexposure prophylaxis.14–18 Access to HIV care and ART is high in San Francisco, with low-income or medically indigent individuals provided coverage through federal, state, and local programs, including Medi-Cal (the California Medicaid program); Ryan White; and Healthy San Francisco, a health coverage program for uninsured San Franciscans. In California, there is no waiting list for the AIDS Drug Assistance Program.19 The history of the HIV epidemic in San Francisco has also resulted in a large network of experienced HIV care providers and service organizations. Because of these factors, San Francisco shows robust outcomes along the HIV continuum of care: low rates of unrecognized HIV infection (7.5% among MSM), prompt linkage to care (87% within 6 months of diagnosis), moderate levels of engagement and retention in care, good ART uptake, and relatively high population rates of virologic suppression.12,20,21
Although San Francisco shows robust outcomes along the HIV continuum of care, there remains room for improvement, and significant disparities exist along the continuum of care for disadvantaged and marginalized populations. Selecting optimal measures for the assessment of improvements along the continuum and decreasing disparities was an identified priority of the SF CFAR ECHPP Collaboration, with a particular focus on linkage to care.
SELECTING MEASURES OF LINKAGE TO CARE
To arrive at a consensus for the optimal definitions and measurements relating to linkage to care, the SF CFAR ECHPP Collaboration conducted a review of relevant literature supplemented by key informant interviews and informal discussions with national and local HIV experts, HIV care providers, and patients. Two concurrent federal processes also guided the collaboration in choosing among numerous approaches to define optimal measures for outcomes along the HIV continuum of care, and in particular, the optimal definition of linkage to care. First, the Institute of Medicine report “Monitoring HIV Care in the United States: Indicators and Data Systems,” released March 15, 2012, identifies core indicators for use by the Health and Human Services (HHS) to gauge the effects of the NHAS and Affordable Care Act (ACA) on improvements in HIV care and access to supportive services for people with HIV; it also highlights 12 data-collection systems that could be used to monitor the impact of the NHAS and ACA.22 Second, the Department of HHS conducted an assessment of the numerous approaches to calculating outcomes along the HIV continuum of care and reached consensus on the set of 7 core population-level indicators measuring diagnosis (including late diagnosis), linkage, retention, ART use, viral suppression, and housing status.23 This information was then used to inform a small-group meeting of collaboration partners for conceptual synthesis and refinement.
Linkage-to-Care Conceptual Processes and Definitions
For quantitative analyses, linkage must be operationalized as an easily abstractable and discrete point or set of points based on objective measures. However, the reality is much more complex; linkage is a process involving physical locations, patient perspectives and decisions, laboratory testing, and care provider decisions (Fig. 1). Furthermore, measurement approaches may use clinical or surveillance data.
After an initial preliminary positive diagnosis with a rapid test in a community-based HIV testing program, the linkage process to HIV primary care is initiated. At the care location, staff may order or disclose results of confirmatory HIV testing (for those diagnosed through rapid testing), if not done already. To promote faster linkage to HIV primary care, SFDPH implemented a change in policy allowing preliminarily positive clients to be seen at primary care clinics and have their confirmatory testing done there rather than requiring the client to return to the community-based testing site for the confirmatory test. At the intake visit, all patients generally receive staging laboratory tests, including CD4 and HIV viral load (VL). At high-volume clinical sites, the intake usually happens with a social worker, nurse, or both. The first visit with a primary HIV care provider may or may not be on the same day as the intake. A second visit with the HIV provider is also important, as it suggests that the patient has made a connection both to the care location and to the provider. The HIV provider generally orders subsequent CD4 cell count and HIV VL tests as part of a treatment plan. The offer and prescription of ART often occur as early as the first HIV provider visit.
Based on this conceptual understanding, clinic-based definitions of linkage can include 1 HIV care location visit (during which the patient may or may not have been seen by a provider with prescribing privileges),25 1 visit with a prescribing provider,26 or 2 visits with a prescribing provider.27 Varying time parameters have been used to further define these measures—for example, 1 visit with a prescribing provider within 30, 60, or 90 days.28,29 The HHS recommendation for the linkage-to-care measure is the proportion of the persons who attended a routine HIV medical care visit within 3 months of HIV diagnosis over the number of persons with an HIV diagnosis in the 12-month measurement period.23
In contrast to definitions of linkage using data from clinical electronic medical records (EMRs) that give high-resolution detail (identifying intermediate steps in the linkage process), surveillance-based definitions of linkage-to-care use laboratory data (the CD4 and VL measurements) reported to the HIV surveillance system as a less-detailed proxy for an HIV primary care visit. Different measures used include 1 CD4 or VL measurement on a day other than the day of the test that diagnosed HIV infection,30 2 CD4 or VL measurements within 6 or 12 months of the test that diagnosed HIV infection,31 or 1 CD4 or VL measurement within 3 months of the test that diagnosed HIV infection and another within the following 9 months.20,32 The surveillance measure assumes that having a CD4 or VL drawn indicates that the individual presented to an HIV care setting and was seen by a primary care doctor. However, as the conceptual framework reflects, the laboratory-based measure rather than a clinic-based measure of clinic attendance may not be as high-resolution because laboratory values are a proxy measure only and do not necessarily indicate that a visit with an HIV primary care provider has occurred.33–35
In summary, options identified for measuring linkage to care ranged from high-resolution clinical visit information available from EMRs to proxy measures for clinical visits based on mandated laboratory reporting to the HIV/AIDS surveillance system.
Sources of Data for Measuring Linkage to Care
The quality and availability of data sources is an important structural constraint on choosing optimal measures for linkage to care. The SF CFAR ECHPP research collaboration mapped data sources along the HIV care continuum to understand the diversity and the quality of data available for measuring linkage to care and other continuum outcomes (Fig. 2).
The arrow representing the continuum of care, for the purposes of visualization, simplifies the details in the steps that take place after a client is diagnosed at a community-based testing site and linked to and engaged in care with a primary care HIV doctor. These steps include assessing the client's health insurance options (which may or may not be done by community-testing staff); determining availability of an appropriate provider; making an appointment; and ensuring that the client attends the visit, has a laboratory appointment, receives CD4 and VL screening along with HIV genotype and other key intake laboratory tests, possibly has a follow-up appointment, and is initiated on ART. As the boxes below highlight, for each step along the continuum of care, there are numerous distinct, noninteroperable, proprietary data systems that contain the information necessary to calculate outcomes.
In San Francisco, individuals can be diagnosed with HIV in numerous settings, including community-based HIV testing programs, the municipal sexually transmitted disease clinic, public health department primary care clinics, private medical providers, public or private emergency rooms and hospitals, or the jail. Each of these locations has a different data system to record the HIV testing date and results. In general, they are supported by distinct sources of funding and have different data-reporting requirements. Also, HIV testing programs vary greatly in their practices for ensuring and confirming that linkage to primary HIV care has occurred; currently, most programs rely mainly upon the public health department to do this. Further along the continuum of care, information for appointment dates may be in multiple data systems—for example, a clinic scheduling system may be distinct from the EMR, may connect to it, or may be part of the same software program. Similarly, prescription data could be contained in the EMR, a pharmacy data system, or in insurance or other administrative data systems. The IOM recommended that ART information be added to the HIV/AIDS surveillance system, and in San Francisco, current ART and history of ART have been collected by active surveillance activities for many years preceding these IOM recommendations.10,20 Finally, VL dates and values could be contained in the following distinct data systems: the EMR, a laboratory data system, and/or the HIV surveillance registry.
These examples illustrate a key issue that emerges from examination of the numerous data sources across the continuum of care: necessary information for the calculation of outcome measures along the continuum is housed in multiple systems. These include systems that may be paper-based or electronic, public or private, and clinical or administrative. These distinct data systems, in most cases, cannot share information with each other, thereby producing significant challenges to systematic measurement across the HIV care continuum. In conversations with other ECHPP implementation sites, this situation is a generalized one across many jurisdictions nationwide. Solutions that are being attempted in SF and other jurisdictions include the following: (1) allocating ad hoc or systematic manual extraction and collation of data across the disparate data sources using significant human resources and (2) implementation of a single integrated data management system. The results of these efforts will be revealed in the coming years.
IMPLICATIONS OF MEASURE SELECTION
Taken together, the numerous approaches for measuring linkage to care, along with the diversity of data sources with varying quality and availability, present competing options. At one extreme, there is high-resolution clinical visit data on a subset of HIV-infected patients linked or failing to link to care. On the other extreme, there is a low-resolution proxy measure (CD4 and VL) on a very high proportion of HIV-infected patients diagnosed and obtaining care in San Francisco. Choosing the high-resolution measure available only in a subset of individuals provides a more comprehensive understanding of the process of linkage to care at the expense of generalizability. This may be useful in specific subpopulations of interest to identify potential targets within the linkage process for linkage to care or other interventions. Choosing a lower-resolution, widely available measure has the advantages of broad generalizability, relative ease of use, and ability to measure longitudinal trends. Additionally, surveillance-based data are easy to compare across jurisdictions.
Engaging Community Stakeholders in Measure Selection
The SFDPH engaged key community stakeholders to provide input and feedback regarding the optimal measures of ECHPP efforts, including linkage to care. The HIV Prevention Planning Council (HPPC), the jurisdictional HIV prevention planning group, was asked to review San Francisco's options for measurement, taking into consideration the HHS proposed indicators. The HPPC addressed the competing priorities of lower-resolution, more generalizable data on linkage, with higher-resolution data on linkage on a smaller group of patients by endorsing the use of both approaches. The council noted that the best way to measure success of ECHPP and other prevention efforts at the population level was to use a “HIV continuum of care” approach, using the data that are most consistently and routinely available for the entire jurisdiction (such as CD4 and VL data from the HIV/AIDS surveillance system) to calculate the outcomes. For the HHS proposed linkage-to-care measure, the HPPC realized that it currently could only be calculated for public health department clinical settings because calculating these indicators on a jurisdiction level would require integration of data (information about clinical visit dates) from numerous proprietary clinical systems. Thus, they recommended that SFDPH continue its efforts to measure linkage to care using clinical visit data among the subset of clients in the public health department clinical settings for whom that data were routinely available, for the purposes of monitoring and improving clinical and public health testing program quality.
MEASURING LINKAGE-TO-CARE IN SAN FRANCISCO
Currently, the SFDPH is reporting the HHS recommended 3-month linkage-to-care measure utilizing routinely collected surveillance data.8 In 2010 and 2011, 85% of all newly diagnosed San Francisco HIV cases were linked to care within 3 months of diagnosis. Among individuals analyzed in 2009–2010, linkage-to-care rates were worse among the following groups: (1) MSM who also inject drugs, persons without an identified transmission risk, and (2) persons without health insurance or whose insurance status was unknown (compared with reference groups without those characteristics).20 In addition, the SFDPH continues to not only measure but also ensure linkage to care occurs for the subset of newly diagnosed HIV patients diagnosed by SFDPH-funded community sites and those seen in the SFDPH clinical system through a variety of human resource intensive methods, including review and collation of data from different systems, and communication by phone and in person with clients and medical providers to both verify and support linkage for these clients.
CFAR supported efforts to evaluate local NHAS implementation through ECHPP focused on improving outcomes along the HIV continuum of care. At a local level in San Francisco, implementation of procedures to measure continuum outcomes has revealed some challenges, particularly for linkage to care after HIV diagnosis. Chief among these challenges are the variety of possible definitions of linkage to care and the nonintegrative nature of the data systems necessary to measure the linkage to care. The choice of linkage-to-care measures, which at the extremes is a choice between higher-resolution measures based on clinical visit data in a subset of patients vs. a lower-resolution proxy measure based on surveillance data in a larger, more generalizable population, has key implications. Choosing between the options for the optimal measure needs to be informed by the primary purpose for which the measure will be used. For representing trends in overall performance and response to interventions such as ECHPP, more generalizable measures based on surveillance data covering the widest number of HIV-infected persons in the jurisdiction are optimal. For identifying barriers to linkage to care for specific populations and potential intervention targets within the linkage process, higher-resolution measures of linkage to care that include clinical, laboratory, social work, and benefits visit information are likely to be optimal.
Cataloging the different data systems along the continuum and observations around the challenges of data sharing between systems highlighted the promise of an integrated data management system that spans HIV surveillance and care systems. Such integrated data management systems would have the ability to support detailed investigation and measurement of the complex process of linkage to care and would provide simplified data to match newly developed, cross-agency HHS measures of HIV care continuum outcomes.23 Having such a system in place would facilitate moving beyond measuring these outcomes, to an improved capacity for real-time public health interventions to improve these outcomes.36 Potential approaches to address these challenges are already underway at the local level. For example, the SFDPH has been funded by the Centers for Disease Control and Prevention to implement an integrated data system in parallel with the implementation of new electronic health record systems in its clinics. As efforts are made to develop integrated data systems that promote high levels of data security and protect confidentiality, we will better be able to evaluate NHAS and the implementation of the ACA at the local level and to use these data to obtain optimal cross-agency measures of the HIV care continuum to enhance our national public health HIV prevention efforts.
4. Dilley JW. Implications for the San Francisco model of care. AIDS Care. 1990;2:349–352.
5. Silverman M. AIDS care: the San Francisco model. J Ambul Care Manage. 1988;11:14–18.
10. Das M, Chu PL, Santos GM, et al.. Decreases in community viral load are accompanied by reductions in new HIV
infections in San Francisco. PLoS One. 2010;5:e11068.
11. Geng EH, Hare CB, Kahn JO, et al.. The effect of a “universal antiretroviral therapy” recommendation on HIV
RNA levels among HIV
-infected patients entering care with a CD4 count greater than 500/μL in a public health setting. Clin Infect Dis. 2012;55:1690–1697.
12. Raymond HF, Chen YH, Ick T, et al.. A new trend in the HIV
epidemic among men who have sex with men, San Francisco, 2004-2011. J Acquir Immune Defic Syndr. 2013;62:584–589.
13. Kahn JO, Martin JN, Roland ME, et al.. Feasibility of postexposure prophylaxis (PEP) against human immunodeficiency virus infection after sexual or injection drug use exposure: the San Francisco PEP Study. J Infect Dis. 2001;183:707–714.
14. Grant RM, Lama JR, Anderson PL, et al.. Preexposure chemoprophylaxis for HIV
prevention in men who have sex with men. N Engl J Med. 2010;363:2587–2599.
15. HPTN 069: Phase II randomized, double-blind, study of safety and tolerability of maraviroc, maraviroc + emtricitabine, maraviroc + tenofovir or tenofovir + emtricitabine for preexposure prophylaxis to prevent HIV
transmission in at-risk men who have sex with men and in at-risk women. Available at: http://www.hptn.org/research_studies/hptn069.asp
18. Cohen SE, Liu AY, Bernstein KT, et al.. Preparing for HIV
pre-exposure prophylaxis: lessons learned from post-exposure prophylaxis. Am J Prev Med. 2013;44(1 suppl 2):S80–S85.
20. Muthulingam D, Chin J, Hsu L, et al.. Disparities in engagement in care and viral suppression among persons with HIV
. J Acquir Immune Defic Syndr. 2013;63:112–119.
21. Das M, Chu P, Santos GM. Success of test and treat in San Francisco? Reduced time to virologic suppression, decreased community viral load, and fewer new HIV
infections, 2004 to 2009. Paper presented at: 18th Conference on Retroviruses and Opportunistic Infections; February 27-March 2, 2011; Boston, MA.
22. Institute of Medicine. Monitoring HIV
Care in the United States: Indicators and Data Systems. Washington DC: National Academies Press; 2012.
23. Valdiserri RO, Forsyth AD, Yakovchenko V, et al.. Measuring what matters: development of standard HIV
core indicators across the U.S. Department of Health and Human Services. Public Health Rep. 2013;128:354–359.
24. Christopoulos KA, Charlebois ED. Enhancing HIV
Prevention in San Francisco. Presented at: the First National CFAR/APC ECHPP Conference. Washington, DC; November 20, 2012.
25. Christopoulos KA, Kaplan B, Dowdy D, et al.. Testing
and linkage to care
outcomes for a clinician-initiated rapid HIV testing
program in an urban emergency department. AIDS Patient Care STDS. 2011;25:439–444.
26. Christopoulos KA, Schackman BR, Lee G, et al.. Results from a New York City emergency department rapid HIV testing
program. J Acquir Immune Defic Syndr. 2010;53:420–422.
27. Bhatia R, Hartman C, Kallen MA, et al.. Persons newly diagnosed with HIV
infection are at high risk for depression and poor linkage to care
: results from the Steps Study. AIDS Behav. 2011;15:1161–1170.
28. Rothman RE, Kelen GD, Harvey L, et al.. Factors associated with no or delayed linkage to care
in newly diagnosed human immunodeficiency virus (HIV
)-1-infected patients identified by emergency department-based rapid HIV
screening programs in two urban EDs. Acad Emerg Med. 2012;19:497–503.
29. Reed JB, Hanson D, McNaghten AD, et al.. HIV testing
factors associated with delayed entry into HIV
medical care among HIV
-infected persons from eighteen states, United States, 2000-2004. AIDS Patient Care STDS. 2009;23:765–773.
30. Zetola NM, Bernstein K, Ahrens K, et al.. Using surveillance data to monitor entry into care of newly diagnosed HIV
-infected persons: San Francisco, 2006-2007. BMC Public Health. 2009;9:17.
31. Bertolli J, Shouse RL, Beer L, et al.. Using HIV
surveillance data to monitor missed opportunities for linkage and engagement in HIV
medical care. Open AIDS J. 2012;6:131–141.
32. Dombrowski JC, Kent JB, Buskin SE, et al.. Population-based metrics for the timing of HIV
diagnosis, engagement in HIV
care, and virologic suppression. AIDS. 2012;26:77–86.
33. Horberg MA, Hurley LB, Silverberg MJ, et al.. Missed office visits and risk of mortality among HIV
-infected subjects in a large healthcare system in the United States. AIDS Patient Care STDS. 2013;27:442–449.
34. Kozak M, Zinski A, Leeper C, et al.. Late diagnosis, delayed presentation and late presentation in HIV
: proposed definitions, methodological considerations and health implications. Antivir Ther. 2013;18:17–23.
35. Mugavero MJ, Amico KR, Westfall AO, et al.. Early retention in HIV
care and viral load suppression: implications for a test and treat approach to HIV
prevention. J Acquir Immune Defic Syndr. 2012;59:86–93.
36. Das M. Beyond measuring what matters to managing what matters. Public Health Rep. 2013;128:360–363.