Between 366 and 730 days after diagnosis, 780 patients (43.8%) successfully attended ≥1 clinic visit, and 503 patients (28.2%) met the HRSA retention measure (2 clinic visits ≥90 days apart in 1 year). Similarly, 366–730 days after diagnosis, 23.5% (N = 419) had virological suppression. In total, 1108 patients (62.2%) were neither retained in care nor had virological suppression 366–730 days after diagnosis.
Using retention in care as the outcome of appropriate linkage, the measure with the highest PPV for retention was attending 2 clinic visits ≥90 days apart within 365 days of diagnosis (74.2%; 95% CI: 70.8% to 78.0%). The linkage measure with the lowest PPV for retention was laboratory monitoring within 30 days (32.0%; 95% CI: 29.1% to 34.9%). PLWH who did not have a clinic visit between 21 and 365 days after diagnosis had the highest NPV for retention in care (94.5%; 95% CI: 0.928 to 0.962), whereas the absence of laboratory monitoring within 30 days of diagnosis had the lowest NPV for retention (77.0%; 95% CI: 74.0% to 80.0%). The measure with the highest AUC for retention in care was attending a clinic visit between 21 and 365 days after diagnosis (Table 3).
Using virological suppression as the outcome of appropriate linkage, the measure with the highest NPV for virological suppression was lacking laboratory testing within 365 days of diagnosis (98.2%; 95% CI: 96.1% to 99.3%) and the measure with the lowest NPV was the absence of a clinic visit within 30 days of diagnosis (87.0%; 95% CI: 85.0% to 89.0%). The measure with the lowest PPV for virological suppression was having 2 tests in 180 days separated by 90 days (41.0%; 95% CI: 37.0% to 45.0%), whereas having 2 visits in 180 days separated by 90 days had the highest PPV for virological suppression (49.0%; 95% CI: 43.0% to 55.0%). Patients completing 2 laboratory tests separated by 90 days in 365 days had the highest AUC for virological suppression (Table 4).
Sensitivity analyses were performed to account for care received outside of RWP-funded clinics. Linkage measures associated with the highest and lowest odds of retention in care, sensitivities, specificities, NPVs, and PPVs did not change. Sensitivity analyses were also performed to account for the possibility of unreported undetectable viral loads. NPVs and PPVs did not differ greatly. When sensitivity analyses were performed assuming that both laboratory monitoring–based measures and the outcome measure of virological suppression were underestimated, PPV changed a little but NPV decreased to as little as 11% for having any laboratory test sent within 365 days (see Tables S3–S6, Supplemental Digital Content, http://links.lww.com/QAI/A425).
Additional analyses were performed to determine the effect of mortality on estimates. Point analyses of those alive at 730 days after diagnosis did not differ by 10% of their baseline. We also performed subanalyses examining only those patients who did not move out of Philadelphia during the study. Point analyses did not differ by 10% of their baseline.
This is one of the first studies comparing multiple laboratory- and clinic-based measures of linkage to care. Clinic-based linkage measures, in particular completing ≥1 clinic visits between 21 and 365 days after diagnosis, best predict retention in care. However, completing 2 laboratory tests separated by 90 days within 365 days of diagnosis best predicts virological suppression. These data suggest that both clinic- and laboratory-based linkage measures have value. Selection of a linkage measure should be tailored to the outcome of interest.
Government agencies and professional organizations vary in criteria used to define linkage to care. The Emergency Department National HIV Testing Consortium metric is a clinic visit within 30 days of diagnosis.27 In our study, only 62.3% of patients meeting this measure were retained in care, and only 40.5% achieved virological suppression. The US National HIV/AIDS Strategy recommends linking 85% of persons to care within 90 days of diagnosis, using a laboratory-based measure.47 Although 72.0% of patients met this measure, only 32.5% were retained, and 27.7% achieved virological suppression. Using the linkage metric most predictive of retention in care, 38.2% of patients completed a clinic visit between 21 and 365 days after diagnosis, with 65.0% of patients meeting this measure retained in care. Alternatively, using the linkage metric most predictive of virological suppression, 56.3% of patients had 2 laboratory tests 90 days apart within 365 days of diagnosis, and 38.0% of them achieved virological suppression. Our data suggest that agencies and organizations should consider which linkage metrics best meet their outcomes of interest when recommending linkage criteria.
Laboratory-based linkage measures had lower predictive abilities for retention in care than clinic-based measures. Within 365 days of diagnosis, 81.6% of the population had a laboratory test, whereas only 39.5% had a clinic visit within the Ryan White system within 365 days of diagnosis. Clinic visits were underreported to a greater extent than laboratory testing because we used the HRSA definition for HIV clinic visits10 (which excludes pharmacy, nursing, social work, and other visits to providers without prescribing privileges) and because we were unable to detect visits to non–RWP-funded clinics. Furthermore, laboratory testing may occur outside of the primary HIV care setting, including in non-HIV clinics, inpatient hospitals, and emergency departments. Our data differ from recent studies suggesting the use of laboratory testing as a proxy for clinic visits.51 Although using laboratory tests as a proxy for clinic visits may be helpful in clinic cohort–based studies,51 our data suggest that laboratory tests may not be as predictive of clinic visits in all settings, such as surveillance.
Our study had limitations. First, generalizability was limited as we only studied patients in 1 US city. Also, linkage, retention, and virological suppression are lower than reported elsewhere.1,2,45,46,55–58 For example, a meta-analysis of 28 studies estimated that 77% of patients are linked to care, 51% are retained in care, and 35% achieve virological suppression.1,2 Linkage rates in our study ranged from 17.5% to 81.6%, 28.2% of patients were retained in care and 23.5% achieved virological suppression. Our inability to access clinic visit data on patients seen outside RWP-funded clinics would have underestimated linkage to and retention in care for the metrics that used these data. However, sensitivity analyses accounting for PLWH not attending RWP-funded clinics demonstrated that the linkage metrics most predictive for retention in care and virological suppression did not change. If laboratories were not reporting undetectable HIV-1 RNA levels, we also may have underestimated virological suppression. Sensitivity analyses accounting for this did not alter the results. We also performed sensitivity analyses to account for those who died or migrated out of the city. Although it is possible that PLWH in care may lack laboratory monitoring, this is rare (unpublished data, Philadelphia DPH). Similarly, it is unlikely that PLWH not in care would achieve virological suppression.
We focused on virological suppression and retention in care between 366 and 730 days after diagnosis to differentiate linkage to care from retention in care, as some researchers defined linkage to care within the first 365 days after diagnosis.38 Some patients may have dropped out of care before this time, potentially contributing to the lower retention rates seen in the study. In addition, we did not collect data on the timing of ART initiation, which has been associated with both improved retention in care and virological suppression.59,60 Low rates of ART initiation among PLWH in our cohort could further lower the proportion of patients not meeting retention in care and virological suppression targets.
Despite these adjustments, linkage and retention metrics were still lower than reported elsewhere.1,2,45,46,55–58 Our data came from a large city with many racial minorities and persons below the poverty line, which may be associated with lower rates of retention in care. Care provider characteristics may also differ from other cities. For example, providers, including non-HIV providers working in urgent care, emergency, inpatient, and primary care clinic settings, may be less likely to send laboratory tests than in other regions, lowering laboratory-based linkage rates. Further studies are needed to evaluate how linkage measures perform in other locals and settings.
Understanding the predictive ability of measures of linkage to care is necessary for improving the quality of HIV care and reducing HIV transmissions. Our data suggest that selection of the ideal linkage measure depends on the outcome of interest being evaluated. The clinic-based measure of completing a visit between 21 and 365 days after diagnosis best predicted retention in care and may be useful for testing centers focused on referring PLWH to care. Meanwhile, the laboratory-based measure of completing 2 laboratory tests separated by 90 days within 365 days of diagnosis best predicted virological suppression and may be a helpful definition for test-and-treat strategies aimed at reducing community viral load. Researchers studying retention and virological suppression in PLWH, testing agencies seeking to improve the quality of their work, and funding agencies deciding how to allocate resources should tailor linkage measures based on the outcome of interest.
The authors would like to acknowledge the staff of the City of Philadelphia Department of Public Health AIDS Activities Coordinating Office for their assistance in creating the Ryan White dataset, especially Jane Baker; Coleman Terrell; Mark Shpaner, MD; Marlene Matosky, MPH, RN; and Ethan Schofer.
1. Marks G, Gardner LI, Craw J, et al.. Entry and retention in medical care among HIV-diagnosed persons: a meta-analysis. AIDS. 2010;24:2665–2678.
2. Centers for Disease Control and Prevention. Vital signs: HIV prevention through care and treatment—United States. MMWR Morb Mortal Wkly Rep. 2011;60:1618–1623.
4. Fleishman JA, Yehia BR, Moore RD, et al.. Establishment, retention, and loss to follow-up in outpatient HIV care. J Acquir Immune Defic Syndr. 2012;60:249–259.
5. Centers for Disease Control and Prevention. HIV surveillance—United States, 1981-2008. MMWR Morb Mortal Wkly Rep. 2011;60:689–693.
6. Craw JA, Gardner LI, Marks G, et al.. Brief strengths-based case management promotes entry into HIV medical care: results of the antiretroviral treatment access study II. J Acquir Immune Defic Syndr. 2008;47:597–606.
7. Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the Use of Antiretroviral Agents in HIV-1-infected Adults and Adolescents. Washington, DC: US Department of Health and Human Services; 2011. Available at: http://www.aidsinfo.nih.gov/contentfiles/adultandadolescentgl.pdf
. Accessed September 27, 2012.
8. Tripathi A, Youmans E, Gibson JJ, et al.. The impact of retention in early HIV medical care on viro-immunologic parameters and survival: a statewide study. AIDS Res Hum Retroviruses. 2011;27:751–758.
9. Yehia BR, Fleishman JA, Metlay JP, et al.. Comparing different measures of retention in outpatient HIV care. AIDS. 2012;26:1131–1139.
11. Roberts KJ, Grusky O, Swanson AN. Outcomes of blood and oral fluid rapid HIV testing: a literature review, 2000-2006. AIDS Patient Care STDs. 2007;21:621–637.
13. Garland PM, Valverde EE, Fagan F, et al.. HIV counseling, testing, and referral experiences of persons diagnosed with HIV who have never entered HIV medical care. AIDS Educ Prev. 2011;23(suppl 3):117–127.
14. Branson BM, Handsfield HH, Lampe MA, et al.. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Morb Mortal Wkly Rep. 2006;55:1–17.
15. Centers for Disease Control and Prevention. Revised guidelines for HIV counseling, testing, and referral. MMWR Recomm Rep. 2001;50:1–57.
16. Hidalgo J, Coombs E, Cobbs WO, et al.. Roles and challenges of outreach workers in HIV clinical and support programs serving young racial/ethnic minority men who have sex with men. AIDS Patient Care STDS. 2011;25(suppl 1):S15–S22.
17. Hoxhaj S, Davila JA, Modi P, et al.. Using nonrapid HIV technology for routine, opt-out HIV screening in a high-volume urban emergency department. Ann Emerg Med. 2011;58(suppl 1):S79–S84.
18. Sattin RW, Wilde JA, Freeman AW, et al.. Rapid HIV testing in a southeastern emergency department serving a semiurban-semirural adolescent and adult population. Ann Emerg Med. 2011;58(suppl 1):S60–S64.
19. Torres GW, Heffelfinger JD, Pollack HA, et al.. HIV screening programs in US emergency departments: a cross-site comparison of structure, process, and outcomes. Ann Emerg Med. 2011;58(suppl 1):S104–S113.
20. Christopoulos KA, Kaplan B, Dowdy D, et al.. Testing and linkage to care outcomes for a clinician-initiated rapid HIV testing program in an urban emergency department. AIDS Patient Care STDS. 2011;25:439–444.
21. Blackstock OJ, King JR, Mason RD, et al.. Evaluation of a rapid HIV testing initiative in an urban, hospital-based dental clinic. AIDS Patient Care STDS. 2010;24:781–785.
22. Ulett KB, Willig JH, Lin HY, et al.. The therapeutic implications of timely linkage and early retention in HIV care. AIDS Patient Care STDs. 2009;23:41–49.
23. Molitor F, Waltermeyer J, Mendoza M, et al.. Locating and linking to medical care HIV-positive persons without a history of care: findings from the California Bridge Project. AIDS Care. 2006;18:456–459.
24. Glick NR, Silva A, Zun L, et al.. HIV testing in a resource-poor urban emergency department. AIDS Educ Prev. 2004;16:126–136.
25. Martinez J, Bell D, Dodds S, et al.. Transitioning youths into care: linking identified HIV-infected youth at outreach sites in the community to hospital-based clinics and or community-based health centers. J Adolesc Health. 2003;33(suppl 2):23–30.
26. Hightow-Weidman LB, Jones K, Wohl AR, et al.. Early linkage and retention in care: findings from the Outreach, Linkage, and Retention in Care Initiative Among Young Men of Color who have Sex with Men. AIDS Patient Care STDs. 2011;25(suppl 1):S31–S38.
27. Rothman RE, Kelen GD, Harvey L, et al.. Factors associated with no or delayed linkage to care in newly diagnosed Human Immunodeficiency Virus (HIV)-1-infected patients identified by emergency department-based rapid HIV screening programs in two urban EDs. Acad Emerg Med. 2012;19:497–503.
28. Reed JB, Hanson D, McNaghten AD, et al.. HIV testing factors associated with delayed entry into HIV medical care among HIV-infected persons from eighteen states, United States, 2000-2004. AIDS Patient Care and STDs. 2009;23:765–773.
29. Ickovics JR, Forsyth B, Ethier KA, et al.. Delayed entry into health care for women with HIV disease. AIDS Patient Care STDs. 1996;10:21–24.
30. Konkle-Parker D. How can we facilitate entry into and retention in HIV care? HIV Clin. 2010;22:10–11.
31. Konkle-Parker DJ, Amico KR, Henderson HM. Barriers and facilitators to engagement in HIV clinical care in the Deep South: results from semi-structured patient interviews. J Assoc Nurses AIDS Care. 2011;22:90–99.
32. Milberg J, Sharma R, Scott F, et al.. Factors associated with delays in accessing HIV primary care in rural Arkansas. AIDS Patient Care STDs. 2001;15:527–532.
33. Craw J, Gardner L, Rossman A, et al.. Structural factors and best practices in implementing a linkage to HIV care program using the ARTAS model. BMC Health Serv Res. 2010;10:246.
34. Aziz M, Smith KY. Challenges and successes in linking HIV-infected women to care in the United States. Clin Infect Dis. 2011;52(suppl 2):S231–S237.
35. Mugavero MJ, Lin HY, Willig JH, et al.. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis. 2009;48:248–256.
36. Bhatia R, Hartman C, Kallen MA, et al.. Persons newly diagnosed with HIV infection are at high risk for depression and poor linkage to care: results from the Steps Study. AIDS Behav. 2011;15:1161–1170.
37. Keller S, Jones J, Erbelding E. Choice of rapid HIV testing and entrance into care in Baltimore City sexually transmitted infections clinics. AIDS Patient Care STDS. 2011;25:237–243.
38. Gardner LI, Metsch LR, Anderson-Mahoney P, et al.. Efficacy of a brief case management intervention to link recently diagnosed HIV-infected persons to care. AIDS. 2005;19:423–431.
39. Hsu LC, Chen M, Kali J, et al.. Assessing receipt of medical care and disparity among persons with HIV/AIDS in San Francisco, 2006-2007. AIDS Care. 2011;23:383–392.
40. Torian LV, Wiewel EW, Liu KL, et al.. Risk factors for delayed initiation of medical care after diagnosis of human immunodeficiency virus. Arch Intern Med. 2008;168:1181–1187.
41. Torian LV, Wiewel EW. Continuity of HIV-related medical care, New York City, 2005-2009: do patients who initiate care stay in care? AIDS Patient Care STDS. 2011;25:79–88.
42. Bamford LP, Ehrenkranz PD, Eberhart MG, et al.. Factors associated with delayed entry into primary care after HIV diagnosis. AIDS. 2010;24:928–930.
43. Bertolli J, Shouse RL, Beer L, et al.. Using HIV surveillance data to monitor missed opportunities for linkage and engagement in HIV medical care. Open AIDS J. 2012;6(suppl 1):131–141.
44. Holtgrave DR. On the epidemiologic and economic importance of the National AIDS Strategy for the United States. J Acquir Immune Defic Syndr. 2010;55:139–142.
45. Silva A, Glick NR, Lyss SB, et al.. Implementing an HIV and sexually transmitted disease screening program in an emergency department. Ann Emerg Med. 2007;49:564–572.
46. White DA, Scribner AN, Schulden JD, et al.. Results of a rapid HIV screening and diagnostic testing program in an urban emergency department. Ann Emerg Med. 2009;54:56–64.
47. Dombrowski JC, Kent JB, Buskin SE, et al.. Population-based metrics for the timing of HIV diagnosis, engagement in HIV care, and virologic suppression. AIDS. 2012;26:77–86.
49. Ikard K, Janney J, Hsu LC, et al.. Estimation of unmet need for HIV primary medical care: a framework and three case studies. AIDS Educ Prevent. 2005;17(suppl B):26–38.
51. Dean B, Debes R, Bozzette S, et al.; the HIV Outpatient Study Investigators. HIV laboratory tests used as a proxy for medical visits for defining engagement in care. Abstr No. 1036. Presented at: 20th Conference on Retroviruses and Opportunistic Infections; March 3–6, 2013; Atlanta, GA.
52. Mugavero MJ, Davila JA, Nevin CR, et al.. From access to engagement: measuring retention in outpatient HIV clinical care. AIDS Patient Care STDs. 2010;24:607–613.
53. Mugavero MJ, Westfall AO, Zinski A, et al.. Measuring retention in HIV care: the inclusive gold standard. JAIDS. 2012;61:574–580.
54. Terzian AS, Bodach SD, Wiewel EW, et al.. Novel use of surveillance data to detect HIV-infected persons with sustained high viral load and durable virologic suppression in New York City. PLoS One. 2012;7:e29679.
55. Althoff K, Rebeiro P, Horberg M, et al.. Application of indicators to monitor US Department of Health and Human Services-funded HIV services in the North American AIDS Cohort Collaboration on Research and Design. Abstr No. 1026. Presented at: 20th Conference on Retroviruses and Opportunistic Infections; March 3–6, 2013; Atlanta, GA.
56. Dombrowski J, Kent J, Buskin S, et al.. An encouraging HIV care cascade: anomaly, progress, or just more accurate data? Abstr No. 1027. Presented at: 20th Conference on Retroviruses and Opportunistic Infections; March 3–6, 2013; Atlanta, GA.
57. Doshi R, Matthews T, Isenberg D, et al.. Continuum of HIV care among Ryan White HIV/AIDS Program clients: US, 2010. Abstr No. 1031a. Presented at: 20th Conference on Retroviruses and Opportunistic Infections; March 3–6, 2013; Atlanta, GA.
58. Horberg M, Hurley L, Towner W, et al.. HIV spectrum of engagement cascade in a large integrated care system by gender, age, and methodologies. Abstr No. 1033. Presented at: 20th Conference on Retroviruses and Opportunistic Infections; March 3–6, 2013; Atlanta, GA.
59. Rebeiro P, Althoff KN, Buchacz K, et al.. Retention among North American HIV-infected persons in clinical care, 2000-2008. JAIDS. 2013;62:356–362.
60. Thompson MA, Aberg JA, Hoy JF, et al.. Antiretroviral treatment of adult HIV infection: 2012 recommendations of the International Antiviral Society-USA panel. JAMA. 2012;308:387–402.