Sabharwal, Charulata J. MD, MPH; Braunstein, Sarah L. PhD, MPH; Robbins, Rebekkah S. MPH; Shepard, Colin W. MD
The National HIV/AIDS Strategy (NHAS), released by the White House in 2010, formalizes national goals for increasing access to and receipt of medical care for persons living with HIV (PLWH). In response to the NHAS, the Centers for Disease Control and Prevention (CDC) published the United States Continuum of HIV care in late 2011,1 as a framework for evaluating the clinical status of the nation's PLWH population. CDC recommended that public health officials and HIV care providers work to improve engagement by PLWH in each stage of the continuum to meet NHAS goals. Under current guidelines that recommend initiation of antiretroviral therapy (ART) immediately after HIV diagnosis, patients are expected to move quickly through each stage of the continuum and to achieve viral suppression soon after diagnosis.2
Increasingly, widespread electronic reporting of HIV-related laboratory test data to HIV surveillance programs has enabled longitudinal population-based monitoring of the clinical status of PLWH by public health officials. CD4 counts and viral load (VL) tests are typically used by surveillance programs as proxy measures for the receipt of HIV medical care,1,3–6 defined by the Health Resources and Services Administration as an outpatient HIV medical care visit with a provider authorized to prescribe ART.7 However, despite growing interest in the use of HIV surveillance data for measuring care-related outcomes among PLWH at both the national and local levels, there has been only limited comprehensive evaluation of the validity of surveillance data for such purposes.4,8–10 To date, most nationally representative estimates for HIV-related care measures have validated data from cohort studies against clinical data on HIV care visits.11,12 In addition, several studies have raised concerns about the accuracy of laboratory data in reflecting linkage to care.6,10 To the best of our knowledge, no study has used population-level surveillance data to comprehensively validate care-related outcomes against gold-standard medical record (MR) data on HIV care visits.
New York City (NYC) has approximately 3500 new diagnoses of HIV infection annually.13 Comprehensive electronic reporting of all CD4 and VLs began in 2005, which has enabled the HIV Epidemiology and Field Services Program (HEFSP) of the NYC Department of Health and Mental Hygiene (DOHMH) to monitor care-related outcomes of PLWH. We conducted the NYC Care Validation Study to validate CD4 and VL tests reported to HEFSP as proxy measures for medical care during the first year after HIV diagnosis. The primary objective of the study was to evaluate whether patients' first laboratory test after HIV diagnosis represents linkage to outpatient HIV primary care. We hypothesized that laboratory tests drawn in the early postdiagnostic period are part of the diagnostic workup and therefore do not represent actual linkage. The secondary objective was to assess how well outpatient HIV primary care received by PLWH in the first year after HIV diagnosis is captured by laboratory tests reported to HEFSP. Additionally, we evaluated 4 commonly used approaches for measuring the establishment of ongoing HIV primary care after initial linkage.
Study Population and Data Sources
New York State (NYS) Public Health Law requires named reporting to the NYC DOHMH of all HIV/AIDS diagnoses, all HIV-related illness, and all CD4, VL, and genotype tests conducted for PLWH.14 HEFSP manages the population-based Registry, which is continuously updated with demographic, clinical and other information on persons meeting the CDC's HIV surveillance case definitions, and with results of laboratory tests conducted in NYC for PLWH. Information on deaths among PLWH is appended to the Registry through regular data matches with death certificate data managed by the DOHMH's Bureau of Vital Statistics, as well as with national vital statistics databases.
We selected a population of newly HIV-diagnosed patients from the Registry who had laboratory-based evidence of linkage to HIV care within the 12 months after HIV diagnosis and were therefore eligible to establish in care. Eligible patients were aged ≥13 years at confirmed HIV diagnosis in NYC between January 1, 2009 and December 31, 2009, and reported to the Registry by September 30, 2010. Because of the large number of HIV-diagnosing providers (over 3500), including small private medical providers who diagnose only a few persons per year, we restricted patient selection to high-volume, HIV-diagnosing facilities who reported at least 20 new HIV diagnoses in 2009 and offered co-located HIV medical care (N = 24). Information on demographic characteristics, name of diagnosing provider, and patient vital status within 12 months of HIV diagnosis was obtained from the Registry. An independent database was developed for this project and prepopulated with CD4/VL tests ordered by the diagnosing facility during the 12 months after each patient's HIV diagnosis date and reported to the Registry.
Medical Record Abstraction Data and Population
To enable validation of both linkage and establishment measures, we abstracted dates for all outpatient HIV care visits reported in the MR that occurred within 12 months of HIV diagnosis at the co-located care site. Generally, DOHMH considers patients to be linked to or receiving care based on the laboratory tests ordered by any clinical facility in NYC. However, for study feasibility, we limited MR abstraction to the diagnosing facility with co-located care. Cases without an available MR for review were excluded from the analytic population.
To determine how well laboratory reporting reflects medical care visits in the first year after diagnosis, abstracted care visit dates were aligned with the dates of each CD4/VL test reported from the facility to the Registry. Each care visit was classified as associated or unassociated with a Registry laboratory test if the laboratory test date was within 14 days of the care visit date. Care visits were considered to be associated with a laboratory test(s), if the HIV provider ordered or reviewed a test(s) at the visit. Although a care visit could be associated with multiple laboratory tests, each laboratory test was only associated with 1 care visit. Finally, we abstracted whether the HIV diagnosis occurred in the inpatient setting. All MR reviews were performed by a NYC DOHMH HIV physician specialist.
Linkage to Care
Linkage to care was defined as evidence of an HIV-related laboratory test (CD4 or VL, per the Registry) or HIV care visit (per MR) within 12 months of HIV diagnosis. Timely linkage to care was defined as an HIV-related laboratory test (per Registry) or care visit (per MR) within 91 days of HIV diagnosis.
Establishment in Ongoing HIV Primary Care (Establishment in Care)
We selected 4 approaches for measuring establishment in care and applied them to the first year after HIV diagnosis. The 4 measures differ primarily in the frequency of laboratory tests or care visits required of patients in a 12-month period. The first 2 measures, “continuous care7” and “sustained care9” both require 2 visits in a 12-month period. Continuous care requires ≥2 tests or visits at least 90 days apart, and sustained care, specific to the first year, requires ≥1 test or visit 3–9 months after timely linkage. The continuous care measure is used by Health Resources and Services Administration as a clinic performance measure and is an NHAS indicator. The second 2 measures are aligned with current guidance on clinical visit schedules. “Trimester visits” require ≥1 test or visit in each 4-month period over 12 months,15 and “visit constancy” requires ≥1 test or visit in each quarter over 12 months.16 The trimester visits performance measure was developed by the NYS AIDS institute to monitor patients' compliance with the recommended clinic visit schedule at Designated AIDS Centers in New York. The visit constancy measure mirrors the clinical schedule outlined in the Department of Health and Human Services (DHHS) treatment guidelines.2
We performed a descriptive analysis of the demographic and risk characteristics of the analytic population. To assess potential selection bias, characteristics of the analytic population were compared with those of all persons with a confirmed diagnosis of HIV in NYC in 2009.
Linkage to Care
Timely linkage-to-care rates were calculated for the analytic population using data from the Registry and MR. By definition, all patients in the analytic population were considered linked to care within 12 months based on the Registry data. We determined the proportion of patients that linked to care based on the MR, and categorized patients into 2 groups: “medical-visit” group, if MR confirmed ≥1 HIV care visit occurred at the site of diagnosis, or “no-medical-visit group” for patients without a care visit at the diagnosing site. We compared the demographic characteristics of the 2 subgroups and the proportion: (1) concurrently HIV-diagnosed (AIDS diagnosis within 31 days of HIV diagnosis), (2) diagnosed in the inpatient setting, and (3) who died during the 12 months after HIV diagnosis.
To test the hypothesis that early postdiagnosis laboratory tests are part of the diagnostic workup, we calculated median time in days from HIV diagnosis to patients' first CD4/VL and compared the distribution of the timing of patients' first CD4/VL by subgroup within 0–7 and 0–14 days from diagnosis. We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of Registry data in correctly classifying patients' true timely linkage-to-care status (per MR) based on the first CD4/VL test within: (1) 0–91 days, (2) 8–91 days, or (3) 15–91 days after HIV diagnosis. Sensitivity was defined as the probability that Registry data indicated timely linkage among persons confirmed to be timely linked by MR. Specificity was defined as the probability that Registry data did not indicate timely linkage among persons confirmed not to be timely linked by MR. PPV was defined as the proportion of persons with MR-confirmed timely linkage among all persons with Registry evidence of timely linkage. NPV was defined as the proportion of persons with MR-confirmed no timely linkage among all persons for whom Registry data did not indicate timely linkage.
Establishment in Care
Only patients in the medical-visit group were eligible for these analyses because MR care visit data were necessary for evaluating the validity of Registry data as a reflection of clinical encounters made to establish HIV care. For these patients, the total number, median and range per patient of Registry tests, and MR care visits were calculated. The number and proportion of care visits that were not associated with a Registry laboratory test were quantified.
To evaluate the performance of the 4 approaches for measuring establishment, we calculated the proportion of patients that fulfilled the criterion of each measure using Registry and MR data. The proportion of patients who were considered to be established or not established by both Registry and MR data (percent-agreement) was calculated and compared across measures.
Given the current recommendation to initiate ART immediately after diagnosis, patients should achieve viral suppression (defined as VL ≤200 copies/mL17) by or soon after they become established in care.2,18 Consequently, a good measure of establishment in care should capture patients who are virally suppressed. To corroborate the measure that would reflect the full extent of care, the proportion of patients that met each establishment measure per both Registry and MR data who achieved viral suppression within 6 months after establishment (based on the first suppressed VL in the Registry for that period) was calculated. VL data reported to the NYC DOHMH as of September 30, 2012, were used.
All statistical tests were 2-sided, and P values of <0.05 were considered statistically significant. Analyses were conducted using SAS version 9.2 (SAS Institute, Inc., Cary, NC).
Of the 3536 new, confirmed HIV diagnoses in NYC in 2009, 35.6% (N = 1263) occurred at one of 24 clinical sites, with 75.0% (N = 947) of patients having Registry evidence of linkage within 12 months of diagnosis (Fig. 1). The remaining 25.0% (N = 316) were excluded because they had no Registry evidence of linkage within 12 months of diagnosis. Based on the MR availability, the analytic population was 782 patients from 22 facilities. Compared with all persons newly diagnosed in 2009, the analytic population included more Hispanics and fewer whites, and more persons with heterosexual HIV transmission risk and fewer men who have sex with men (Table 1).
Linkage to Care
By design, all 782 patients (analytic population) had Registry evidence of linkage within 12 months of HIV diagnosis. The proportion that linked timely to care was significantly lower based on the MR data compared with Registry data (74.6% vs. 97.2%, respectively; P < 0.01). Per MR, only 79.9% (N = 625) of patients had an HIV care visit/linkage event during the 12-month period and were classified in the medical-visit group; the remaining 157 (20.1%) were classified in the no-medical-visit group.
The medical-visit and no-medical-visit groups differed on several demographic and key clinical outcomes (Table 1). Compared with the medical-visit group, patients in the no-medical-visit group were older (median age 42 vs. 37 years, P = 0.001), more likely to have an unknown HIV transmission risk (38.9% vs. 23.7%, P < 0.0001), to be diagnosed concurrently with HIV/AIDS (61.8% vs. 37.0%, respectively; P < 0.0001), to be diagnosed in the inpatient setting (73.2% vs. 31.5%, P < 0.0001), and to die within 12 months of diagnosis (17.8% vs. 1.4%, P < 0.0001).
Patients in the no-medical-visit group had a significantly shorter time between diagnosis and linkage based on the Registry data than the medical-visit group (1 day interquartile range, 0–5 vs. 8 days interquartile range, 0–20; P < 0.01). In both subgroups, laboratory test dates clustered close to the diagnosis date, with 88% of all patients having their first CD4/VL within the first month after diagnosis. However, more patients in the no-medical-visit group had their first CD4/VL in both of the early postdiagnosis periods (80.3% within 0–7 days; 85.4% within 0–14 days) compared with the medical-visit group (48.8% and 65.8%, respectively; P < 0.001 for all comparisons). Moreover, among patients whose only laboratory tests were drawn in the first 7 days after diagnosis, 85% had no care visits at the diagnosing facility within the 12 months of diagnosis.
Registry data were highly sensitive in classifying patients' timely linkage-to-care status (99.8%), but specificity was poor (10.6%) (Table 2). After excluding laboratory tests drawn within 0–7 and 0–14 days after diagnosis, sensitivity decreased to 89.0% and 82.0%, respectively. Specificity, however, increased with early postdiagnostic tests removed, especially after removing tests from 0 to 7 days after diagnosis (10.6% to 44.2%). PPV remained stable throughout and NPV decreased substantially after early laboratory tests were excluded.
Establishment in Care
The 625 patients included in the establishment analyses had a total of 3542 CD4 and VL tests with unique Registry dates during the 12 months after diagnosis (median, 5 per patient; range, 1–20). During the same time period, patients made a total of 4413 care visits at the diagnosing sites (median, 7 per patient; range, 1–28). Nearly half (48.5%, N = 2141) of the care visits were not associated with a Registry test.
The proportion of patients considered to be established in care by the 4 measures evaluated ranged from 35.5% to 81.7% using Registry data and from 43.8% to 81.0% using MR data (Table 3). In general, as the test/visit schedule became more stringent with a higher number of tests/visits required in the 12-month period, fewer patients met the measure. The most stringent measure—visit constancy (1 test/visit in each quarter) had the fewest patients considered established (35.5% Registry data; 43.8% MR data) of the 4 measures. This pattern was reflected in the percent agreement between the Registry and MR for the 4 measures, with values ranging from 72.8% to 88.8% and being highest for the 2 least stringent measures, continuous care and sustained care.
Finally, the proportion of patients who achieved viral suppression within 6 months of establishment ranged from 62.2% to 77.8% across the measures. As expected, the highest viral suppression rates were among patients who were considered established per the visit constancy measure, the most stringent of the four. However, only 173 patients were captured as virally suppressed on this measure, 55.6% of the 311 total patients who were actually suppressed within 6 months of establishment. By contrast, the less-stringent sustained care measure captured 95.5% (N = 297) of suppressed patients.
In a comparison of NYC surveillance data against gold-standard MR data, we found that surveillance data overestimated linkage rates but underestimated the extent of HIV-related medical care among persons in the first year after diagnosis. Linkage rates were overestimated as many initial CD4/VL reported to surveillance occurred as part of the diagnostic workup and did not represent an outpatient medical visit. However, care throughout the first year of diagnosis was underestimated as many medical visits were focused on the delivery of non-HIV primary care, i.e., diabetes and hypertension. Despite this, surveillance data remain an important data source for population-level monitoring of PLWH. Our study findings suggest refinements in measurement approaches to more accurately reflect linkage and establishment in care.
Surveillance programs typically rely on the first HIV-related laboratory test after diagnosis as an indication of linkage to medical care. As expected, we found that surveillance data were highly sensitive in capturing interactions with the health care system. However, we found substantial misclassification of timely linkage status in the early postdiagnosis period in which many reported laboratories were part of the diagnostic workup and not indicative of an actual linkage event. Based on this study finding, the NYC DOHMH HEFSP implemented a refined definition of timely linkage to care in 2012. The modified definition considers only patients with CD4/VL between 8 days and 91 days to be linked timely to HIV care; laboratory tests in the first 7 days after diagnosis are considered part of the diagnostic workup. The application of this refinement initially shifted timely linkage rates downward but has resulted in more accurate NYC estimates. Even with the exclusion of laboratories within the first 7 days after diagnosis, specificity improved but remained low given that CD4 or VL tests can be drawn during the diagnostic workup or an inpatient admission as well as during an outpatient medical visit.
Surveillance data were most likely to overestimate linkage for older persons, those with non-men who have sex with men/injection drug user transmission risk, concurrently diagnosed, and those with advanced HIV disease. Many who were misclassified as linked by surveillance died soon after diagnosis. With the refined timely linkage to care definition, most of these persons will no longer have misclassified linkage-to-care status and will be included in public health outreach to promote linkage. Recently passed state laws that remove barriers to HIV testing, such as the 2010 revised NYS HIV testing law that requires clinicians to offer an HIV test to all person ages 13–64 and to link all newly HIV-diagnosed persons to HIV care,14 should support this goal, as will large-scale community-wide NYC HIV testing and linkage initiatives such as The Bronx Knows19 and Brooklyn Knows.20 In addition, NYC DOHMH is providing feedback on facility-specific linkage rates to assist high-volume HIV providers in tracking their progress toward timely linkage goals. Accuracy of local and national surveillance data as a measure of linkage will improve as these initiatives in NYC and similar ones in other jurisdictions21,22 result in improvements in true linkage to HIV care.
Establishment in care after linkage is an equally critical step in reducing morbidity, mortality, and secondary transmissions among PLWH and should be included among key care indicators. In the year after HIV diagnosis, study patients sought frequent medical care with a cumulative total of >4000 visits. Notably, we found that nearly half (48.5%) of patient's care visits were not associated with a CD4/VL. The marked underestimation of care among these patients by surveillance data underscores the range of primary care now being provided to PLWH. The DHHS-recommended patient visit schedule aligns most closely with the 2 most stringent measures of establishment in care we evaluated (trimester visits and visit constancy). However, DHHS guidelines do incorporate some flexibility (eg, monitoring every 3–6 months) and leave the frequency of in-person encounters to clinicians' discretion. We recommend that surveillance programs use sustained care (≥1 laboratory test occurring 3–9 months after initial timely linkage event) for measuring establishment because it requires timely linkage but allows for some variability in the timing of subsequent CD4/VL testing. HEFSP plans to incorporate this measure into routine monitoring of establishment in care among recently diagnosed PLWH after timely linkage. Figure 2 provides an example of a continuum of care for persons newly HIV diagnosed in NYC in 2009 using HEFSP's modified linkage to care definition and the sustained care measure for establishment in care.
This is one of the first studies to validate population-level surveillance laboratory data against information on outpatient care visits recorded in the MR. We evaluated 2 key indicators, and study findings informed an important change to the way NYC HEFSP measures timely linkage to care. Limitations are also noted. A small proportion (188 or 5% of total test dates for study patients) of CD4/VL test dates noted in patients' medical charts could not be identified in the Registry for the specific patient and ordering provider. Study conclusions are limited to the first year after diagnosis. Additional validation work is needed to determine the best measurement of retention among long-standing PLWH. For logistical reasons, we only performed chart reviews at patients' diagnosing sites; linkage to and establishment in care could have occurred at other clinical facilities in NYC or even outside our jurisdiction, particularly for patients who were found to have no medical care at the diagnosing facility. We oversampled patients from acute care facilities, which may explain slight demographic differences between the analytic population and all newly diagnosed NYC PLWH during the time period. However, study facilities represent the highest-volume HIV diagnosing facilities in NYC and collectively care for a large proportion of NYC PLWH. DHHS treatment guidelines in effect at the time our analytic population was diagnosed recommended ART initiation at CD4 <500; though patients were established in care they may not have been ART-eligible and therefore expected to be virally suppressed.
The US national HIV surveillance system is a critical data source for understanding the epidemiology of HIV at the population level, for monitoring the clinical and care status of PLWH and evaluating NHAS goals, and for informing high-impact HIV prevention efforts.23 However, for the system to reach its full potential, all jurisdictions will need to implement comprehensive laboratory reporting. At the local level, surveillance systems require constant upkeep to ensure that reporting of laboratory data are complete, and even with comprehensive and well-established laboratory reporting in place, changes in clinical practices24 will directly impact our understanding of the HIV care continuum. Surveillance will need to stay current and flexible to achieve its goals while overcoming these important challenges.
The authors would like to thank Rachel Manners for assistance in chart abstraction data entry and the HIV Epidemiology and Field Services Program staff for their dedication to HIV Surveillance and Partner Services in NYC.
1. Centers for Disease Control and Prevention. Vital Signs: HIV prevention through care and treatment—United States. MMWR Morb Mortal Wkly Rep. 2011;60:1618–1623.
2. Panel on Antiretroviral Guidelines for Adults and Adolescents. Department of health and human services. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. 2012. Available at: http://www.aidsinfo.nih.gov/contentfiles/lvguidelines/
. Accessed December 5, 2012.
3. Torian LV, Wiewel EW, Liu K, et al.. Risk factors for delayed initiation of medical care after diagnosis of human immunodeficiency virus. Arch Intern Med. 2008;168:1181–1187.
4. Zetola NM, Bernstein K, Ahrens K, et al.. Using surveillance data to monitor entry into care of newly diagnosed HIV-infected persons: San Francisco, 2006-2007. BMC Public Health. 2009;9:17.
5. Hall HI, Gray KM, Tang T, et al.. Retention in care of adults and adolescents living with HIV in 13 U.S. areas. J Acquir Immune Defic Syndr. 2012;60:77–82.
6. Bertolli J, Shouse RL, Beer L, et al.. Using HIV surveillance data to monitor missed opportunities for linkage and engagement in HIV medical care. Open AIDS J. 2012;6:131–141.
8. Hu YW, Kinsler JJ, Sheng Z, et al.. Using laboratory surveillance data to estimate engagement in care among persons living with HIV in Los Angeles County, 2009. AIDS Patient Care STDS. 2012;26:471–478.
9. Dombrowski JC, Kent JB, Buskin SE, et al.. Population-based metrics for the timing of HIV diagnosis, engagement in HIV care, and virologic suppression. AIDS. 2012;26:77–86.
10. Keller SC, Yehia BR, Eberhart MG, et al.. Accuracy of definitions for linkage to care in persons living with HIV. J Acquir Immune Defic Syndr. 2013;63:622–630.
11. Yehia BR, Fleishman JA, Metlay JP, et al.. Comparing different measures of retention in outpatient HIV care. AIDS. 2012;26:1131–1139.
12. Mugavero MJ, Westfall AO, Zinski A, et al.. Measuring retention in HIV care: the Elusive gold standard. J Acquir Immune Defic Syndr. 2012;61:574–580.
13. New York City Department of Health and Mental Hygiene. HIV Epidemiology and Field Services Semiannual Surveillance Report. New York, NY: 2012.
14. New York State. NY health laws. 63.4. Vol New York state.
15. AIDS Institute. In+care campaign measures.
16. Mugavero MJ, Davila JA, Nevin CR, et al.. From access to engagement: measuring retention in outpatient HIV clinical care. AIDS Patient Care STDS. 2010;24:607–613.
17. Centers for Disease Control and Prevention. Using Viral Load Data to Monitor HIV Burden and Treatment Outcomes in the United States. Atlanta, Georgia: Centers for Disease Control and Prevention; 2012:2012.
18. Cohen MS. HIV treatment as prevention and “the Swiss statement”: in for a dime, in for a dollar? Clin Infect Dis. 2010;51:1323–1324.
19. New York City Department of Health and Mental Hygiene. The Bronx knows HIV testing initiative: final report. New York, NY: New York City Department of Health and Mental Hygiene; 2012.
20. New York City Department of Health and Mental Hygiene. Mayor Bloomberg Commemorates World AIDS Day and Launches “Brooklyn Knows” Voluntary HIV Testing Initiative (Press Release). New York, NY: 2010.
21. HIV Prevention Trials Network (HPTN). HPTN 065 TLC-Plus: a study to evaluate the feasibility of an enhanced test, link to care, plus treat approach for HIV prevention in the United States. Available at: http://www.hptn.org/research_studies/hptn065.asp
. Accessed June 1, 2013.
22. Centers for Disease Control and Prevention. Expanded HIV testing and trends in diagnoses of HIV infection - District of Columbia, 2004-2008. MMWR Morb Mortal Wkly Rep. 2010;59:737–741.
23. Centers for Disease Control and Prevention. High-impact HIV Prevention: CDC's Approach to Reducing HIV Infections in the United States. Atlanta, GA: Centers for Disease Control and Prevention; 2011.
24. Gale HB, Gitterman SR, Hoffman HJ, et al.. Is frequent CD4+ T-lymphocyte count monitoring necessary for persons with counts >=300 cells/muL and HIV-1 suppression? Clin Infect Dis. 2013;56:1340–1343.