Abstract: The accuracy of a novel community health worker antiretroviral therapy eligibility assessment tool was examined in community members in Blantyre, Malawi. Nurses independently performed World Health Organization (WHO) staging and CD4 counts. One hundred ten (55.6%) of 198 HIV-positive participants had a CD4 count of <350 cells per cubic millimeter. The community health worker tool significantly outperformed WHO clinical staging in identifying CD4 count of <350 cells per cubic millimeter in terms of sensitivity (41% vs. 19%), positive predictive value (75% vs. 68%), negative predictive values (53% vs. 47%), and area under the receiver–operator curve (0.62 vs. 0.54; P = 0.017). Reliance on WHO staging is likely to result in missed and delayed antiretroviral therapy initiation.
*Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, United Kingdom;
†TB and HIV Group, Malawi-Liverpool-Wellcome Trust Clinical Research Programme, Blantyre, Malawi;
‡Department of Clinical Research, London School of Hygiene and Tropical Medicine, London, United Kingdom;
§Department of Pathology and Laboratory Medical Sciences, College of Medicine, University of Malawi, Blantyre, Malawi;
‖Department of International Public Health, Liverpool School of Tropical Medicine, Liverpool, United Kingdom; and
¶HIV Department, Ministry of Health, Lilongwe, Malawi.
Correspondence to: Peter MacPherson, MBChB, MPH, Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, United Kingdom (e-mail: email@example.com).
P.M. was funded by the Wellcome Trust (grant no. WT089673). E.L.C. was funded by a Wellcome Trust Senior Research Fellowship in Clinical Science (grant no. WT091769). For the remaining authors, none were declared.
The authors have no conflicts of interest to disclose.
P.M., E.L.C., D.G.L., S.B.S., and S.D.M. designed the study. P.M., D.T., A.T.C., E.L.W., and E.L.C collected and analyzed data. G.A.C. performed the laboratory analysis. P.M., D.G.L., and E.L.C. wrote the first draft of the manuscript. All authors contributed to the final manuscript.
This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License, where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially. http://creativecommons.org/licenses/by-nc-nd/3.0.
Received February 19, 2013
Accepted June 12, 2013
The 2010 World Health Organization (WHO) guidelines recommend that HIV-infected adults with a CD4 count of <350 cells per cubic millimeter be initiated onto antiretroviral therapy (ART).1 Availability of CD4 count measurement remains low in much of sub-Saharan Africa. For example, in Malawi, only 12% of HIV care facilities have functional on-site capacity for CD4 count measurement.2 Where facilities for CD4 count are not available, individuals assessed, as being in WHO clinical stage 3 or 4, should receive ART.1
The WHO clinical staging system for HIV was introduced in 19903 and was based on previous highly specific WHO case definitions for AIDS that were used in the era before the availability of reliable and rapid HIV diagnostic tests.4,5 Although the WHO clinical staging system has good prognostic value in predicting death6 and response to ART,7 it was not originally designed to be used for as screening tool for ART eligibility. Previous studies8–11 have shown the sensitivity of nurse-performed WHO stage 3 or 4 assessment in identifying CD4 count of <350 cells per cubic millimeter to be low (between 18% and 65%), potentially resulting in missed and delayed ART initiation.12
We previously found that eligibility assessments were a frequent cause of dropout before ART initiation,13 with health workers finding WHO staging overly complex and time consuming, and individuals reporting that completion of CD4 count measurement required multiple facility visits.14,15 With task shifting of HIV testing to community health workers (CHWs)16 and moves toward “test-and-treat,”17 eligibility assessments should be completed immediately following HIV diagnosis to maximize opportunities for linkage to ART. We therefore developed a brief CHW tool for assessing ART eligibility and compared its accuracy with that of with the current WHO clinical staging system.
An accuracy study nested within a cluster-randomized trial that was investigating community-based approaches to HIV diagnosis and treatment.18
ART Eligibility Guidelines and Development of Novel ART Eligibility Screening Tool
National ART eligibility guidelines in Malawi matched WHO recommendations (ie, CD4 count of <350 cells/mm3 or in WHO stage 3 or 4).1,19
The CHW tool was designed based on the WHO “Ask, Look, Classify, Act” approach incorporated within guidelines for the Integrated Management of Adult and Adolescent Illness.20 Questions were constructed for inclusion in the CHW tool from existing components of the WHO clinical staging system21; with additional validated questions that have previously assessed to be important predictors of ART eligibility (self-rated general health14,22) and advanced immunosuppression (self-rated skin or hair change).23–25 All questions included had to be easily understood and performed by CHWs and not require further laboratory investigations. The CHW tool classifies HIV-positive patients into 2 groups: those who are ART eligible (analogous to WHO stage 3 or 4) and those who are not ART eligible (analogous to WHO stage 1 or 2). Individuals who responded positively to any of the following items were classified as ART eligible:
* reports fever or weight loss or diarrhea for all of last month,21
* enough weight loss to affect fit of clothing,21
* sore mouth or swallowing,21
* treated for TB or any other infection (excluding malaria and “flu”) in the last year,21
* self-rated change in skin or hair,23–25
* unable to carry out daily duties (work, go to school, do housework),21 and
* self-rated general health poor or fair,14
Study Site and Population
Adult (≥16 years) residents of 3 high-density urban townships in the north west of Blantyre, Malawi, were provided with access to home HIV self-testing through CHWs. CHWs were adult residents selected by a participatory approach and who received training in HIV testing and counseling (HTC). CHWs performed the novel ART eligibility assessment tool for individuals who disclosed a positive self-test result. HIV-positive individuals received a home visit from a nurse who independently performed confirmatory HTC, TB screening (symptom screen and sputum collection for smear and culture if positive), WHO clinical staging (without reference to the result of the CHW tool assessment), and drew blood for CD4 count. ART eligible participants (on national criteria) received offer of home or facility ART initiation and noneligible participants were referred to pre-ART clinics at local health centers.
Blood samples were analyzed using the Partec Cyflow SL-3 platform (Partec, Görlitz, Germany).
Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for both the CHW tool and WHO clinical staging using CD4 count of <350 cells per cubic millimeter as the referent. As WHO has recently proposed expanding ART eligibility to include individuals with CD4 count of <500 cells per cubic millimeter,3,26 we repeated analysis with this criteria as the referent. A sample size of 191 patients was required to estimate sensitivity and specificity with a precision of ±10% at the P = 0.05 level.27 Statistical analysis was done using Stata 12.1 (Statacorp, College Station, TX).
The research ethics committees of the College of Medicine of Malawi, Liverpool School of Tropical Medicine, and London School of Hygiene and Tropical Medicine granted ethical approval for the parent cluster-randomized trial. All participants gave written informed consent to participate or witnessed thumbprint consent if illiterate.
Between February and November 2012, 198 of 206 HIV-positive adult community members were recruited to the study (96.1% participation). Men comprised 42% (84/198) of participants and were younger, had a higher level of schooling, and were more likely to have salaried employment than women (Table 1).
Outcomes of ART Eligibility Assessments
A total of 31 of 198 (15.6%) individuals were assessed to be in WHO stage 3 or 4 by nurses, with men significantly more likely to be in the more advanced WHO stage 3 or 4 than women [19/84 (22.6%) vs. 12/114 (10.5%) respectively; P = 0.021]. Men also had a significantly lower median CD4 count at assessment compared with women [262 cells/mm3 (interquartile range, IQR: 148–373) vs. 374 cells/mm3 (IQR: 195–512); P = 0.002]. A total of 110 of 198 (55.6%) participants had a CD4 count below the Malawian national ART eligibility threshold of <350 cells per cubic millimeter.
Performance of CHW Tool in Identifying CD4 count of <350 Cells per Cubic Millimeter
One participant did not have the CHW tool completed by the CHW, leaving 197 participants who were analyzed with complete data for outcome of CHW tool, CD4 count, and WHO clinical staging assessment.
The sensitivity of the CHW tool (41.3%; 95% CI: 31.9% to 51.1%) was substantially higher than that of nurse-performed WHO clinical stage 3 or 4 assessment (19.1%, 95% CI: 12.2% to 27.7%) in identifying CD4 count of <350 cells per cubic millimeter (Table 2). The specificity of the CHW tool was 83.0% (95% CI: 73.4% to 90.1%) compared with 88.6% (95% CI: 80.1% to 94.4%) for WHO clinical stage 3 or 4 assessment. The area under the curve (AUC) was significantly higher for the CHW tool (0.62; 95% CI: 0.56 to 0.68) compared with WHO clinical stage 3 or 4 assessment (0.54; 95% CI: 0.49 to 0.59; P = 0.017).
Unexpectedly, there were differences in test performance when results were stratified by gender. The sensitivity and specificity of the CHW tool were higher among men (45.6% and 92.3%, respectively) than among women (36.5% and 79.0%, respectively). For both men and women, however, the PPV (92.9% vs. 78.9% and 59.4% vs. 50.0%, respectively) and NPV (43.6% vs. 33.8% and 59.8% vs. 54.9%, respectively) of the CHW tool was higher than that of WHO clinical staging. The largest AUC was observed for men identified as ART eligible by the CHW tool (AUC = 0.69), which was significantly higher than for WHO stage 3 or 4 assessed men (AUC = 0.55; P = 0.016). For women, although the AUC for the CHW tool (AUC = 0.58) was greater than for WHO stage 3 or 4 assessment (AUC = 0.51), the difference was relatively small and did not reach statistical significance (P = 0.104).
Using CD4 count of <500 cells per cubic millimeter as the referent, the CHW tool similarly outperformed WHO clinical staging assessment in terms of sensitivity, PPV, NPV, and AUC (Table 2), although performance of both tools was worse than at the CD4 count of <350 cell per cubic millimeter cutoff.
The main finding from this study was that our brief CHW tool significantly outperformed the current WHO clinical staging system for identifying ART eligible individuals with CD4 count of <350 cells per cubic millimeter. The ease with which we were able to improve on the current WHO staging system emphasizes the need for improved tools that can be used by community and facility-based health workers to ensure “same-day, same-site” eligibility assessments and prompt linkage to ART. Moreover, the low sensitivity and high risk of misclassification when using the standard WHO clinical staging system in this and other studies8–11 questions its appropriateness as an ART eligibility assessment tool, especially with increasing CD4 count thresholds.28
The CHW tool was found to be significantly more accurate than WHO clinical staging assessment in terms of sensitivity, PPV, and NPV. In contrast to the WHO clinical staging system, which, strictly followed, requires nurses or doctors to undertake a complex and time-consuming assessment,14 our CHW tool was easily performed in the community and could be used in settings without sophisticated laboratories. The poor performance of nurse WHO clinical staging system in this and other studies8–11 is therefore not a factor of the capability of nurses. Instead it is a reflection of the inherent limitations of the WHO screening tool for identifying ART eligibility, a task for which it was never originally intended.
The CD4 count is the gold standard for determining ART eligibility, but access to laboratory measurement is problematic in most resource-limited settings.29 Point of care CD4 count measurement has the potential to negate some of these issues and improve retention in pre-ART care,30 but it is not yet widely available. Where CD4 count is not available at the point of HIV diagnosis, improved eligibility assessment tools such as ours could allow an immediate decision on ART initiation to be made, facilitating “test-and-treat” delivery of ART.
The accuracy of the CHW tool was noted to be higher among men than women. This may reflect the relatively more advanced immunosuppression of men compared with women in the study, consistent with findings from HIV care programmes in sub-Saharan Africa showing that men initiate ART later than women.28 Women may have also been less likely to report symptoms included in the tool. Future, larger studies should examine the performance of CHWs tool among men and women across CD4 count strata.
Our CHW tool was designed pragmatically to combine ease of use by CHWs with questions that were hoped would have a high discriminatory power. Component parts of the tool were not captured and so we were unable to assess the contribution of individual items to the tool performance. As such, it is possible that the number of items could be reduced, or the sensitivity and specificity improved beyond that reported here. Future research should attempt to refine the tool by examining the contribution of the included items to the test performance and evaluating any modifications.
In conclusion, this study has shown that a novel screening tool performed by CHWs can distinguish ART eligible from ART ineligible HIV-positive individuals. This could be added to HTC counselors' posttest evaluations, allowing same-day, same-site HIV diagnosis and ART eligibility assessment. The accuracy of the CHW tool significantly outperformed WHO staging by a nurse and the ease with which we were able to improve on the current WHO clinical staging system shows the urgent need for improved ART eligibility assessment tools.
The authors would like to acknowledge Dr A. Jahn, HIV Department, Ministry of Health of Malawi, for review and insightful comments. We also acknowledge the participants and providers in the study.
1. World Health Organization. Antiretroviral Therapy for HIV Infection in Adults and Adolescents: Recommendations for a Public Health Approach. Geneva, Switzerland: World Health Organization; 2010.
2. HIV Department, Ministry of Health of Malawi. Integrated HIV Program Report January—March 2012. Lilongwe, Malawi: Ministry of Health Malawi; 2012.
3. World Health Organization. Interim proposal for a WHO staging system for HIV infection and disease. Wkly Epidemiol Rec. 1990;65:221–224.
4. Colebunders R, Mann JM, Francis H, et al.. Evaluation of a clinical case-definition of acquired immunodeficiency syndrome in Africa. Lancet. 1987;1:492–494.
5. De Cock KM, Colebunders R, Francis H, et al.. Evaluation of the WHO clinical case definition for AIDS in rural Zaire. AIDS. 1988;2:219–221.
6. Malamba SS, Morgan D, Clayton T, et al.. The prognostic value of the World Health Organisation staging system for HIV infection and disease in rural Uganda. AIDS. 1999;13:2555.
7. Lawn SD, Harries AD, Anglaret X, et al.. Early mortality among adults accessing antiretroviral treatment programmes in sub-Saharan Africa. AIDS. 2008;22:1897–1908.
8. Carter RJ, Dugan K, El-Sadr WM, et al.. CD4+ cell count testing more effective than HIV disease clinical staging in identifying pregnant and postpartum women eligible for antiretroviral therapy in resource-limited settings. J Acquir Immune Defic Syndr. 2010;55:404–410.
9. Torpey K, Lartey M, Amenyah R, et al.. Initiating antiretroviral treatment in a resource-constrained setting: does clinical staging effectively identify patients in need? Int J STD AIDS. 2009;20:395–398.
10. Jaffar S, Birungi J, Grosskurth H, et al.. Use of WHO clinical stage for assessing patient eligibility to antiretroviral therapy in a routine health service setting in Jinja, Uganda. AIDS Res Ther. 2008;5:4.
11. Baveewo S, Ssali F, Karamagi C, et al.. Validation of World Health Organisation HIV/AIDS clinical staging in predicting initiation of antiretroviral therapy and clinical predictors of low CD4 cell count in Uganda. PLoS One. 2011;6:e19089.
12. Athan E, O'Brien DP, Legood R. Cost-effectiveness of routine and low-cost CD4 T-cell count compared with WHO clinical staging of HIV to guide initiation of antiretroviral therapy in resource-limited settings. AIDS. 2010;24:1887–1895.
13. Macpherson P, Corbett EL, Makombe SD, et al.. Determinants and consequences of failure of linkage to antiretroviral therapy at primary care level in Blantyre, Malawi: a prospective cohort study. PLoS One. 2012;7:e44794.
14. Macpherson P, Lalloo DG, Choko AT, et al.. Suboptimal patterns of provider initiated HIV testing and counselling, antiretroviral therapy eligibility assessment and referral in primary health clinic attendees in Blantyre, Malawi. Trop Med Int Health. 2012;17:507–517.
15. MacPherson P, MacPherson EE, Mwale D, et al.. Barriers and facilitators to linkage to ART in primary care: a qualitative study of patients and providers in Blantyre, Malawi. J Int AIDS Soc. 2012;15:18020.
16. Zachariah R, Ford N, Philips M, et al.. Task shifting in HIV/AIDS: opportunities, challenges and proposed actions for sub-Saharan Africa. Trans R Soc Trop Med Hyg. 2009;103:549–558.
17. Granich RM, Gilks CF, Dye C, et al.. Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model. Lancet. 2009;373:48–57.
19. Ministry of Health. Clinical Management of HIV in Children and Adults. Lilongwe, Malawi: Ministry of Health Malawi; 2011.
20. World Health Organisation. Integrated Management of Adolescent and Adult Illness. Chronic HIV Care With ARV Therapy and Prevention. Interim Guidelines for Health Workers at Health Centres or District Hospital Outpatients Clinics. Geneva, Switzerland: World Health Organization; 2007.
21. World Health Organization. WHO Case Definitions for Surveillance and Revised Clinical Staging and Immunological Classification of HIV-Related Disease in Adults and Children. Geneva, Switzerland: World Health Organization; 2007.
22. Singh-Manoux A, Dugravot A, Shipley MJ, et al.. The association between self-rated health and mortality in different socioeconomic groups in the GAZEL cohort study. Int J Epidemiol. 2007;36:1222–1228.
23. Ferrand RA, Weiss HA, Nathoo K, et al.. A primary care level algorithm for identifying HIV-infected adolescents in populations at high risk through mother-to-child transmission. Trop Med Int Health. 2011;16:349–355.
24. Lewis DK, Callaghan M, Phiri K, et al.. Prevalence and indicators of HIV and AIDS among adults admitted to medical and surgical wards in Blantyre, Malawi. Trans R Soc Trop Med Hyg. 2003;97:91–96.
25. Cedeno-Laurent F, Gomez-Flores M, Mendez N, et al.. New insights into HIV-1-primary skin disorders. J Int AIDS Soc. 2011;14:5.
26. Hirnshall G, Harries AD, Easterbrook PJ, et al.. The next generation of the World Health Organization's global antiretroviral guidance. J Int AIDS Soc. 2013;16:18757.
27. Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res. 1998;7:371–392.
28. When To Start Consortium, Sterne JA, May M, Costagliola D, et al.. Timing of initiation of antiretroviral therapy in AIDS-free HIV-1-infected patients: a collaborative analysis of 18 HIV cohort studies. Lancet. 2009;373:1352–1363.
29. Losina E, Bassett IV, Giddy J, et al.. The “ART” of linkage: pre-treatment loss to care after HIV diagnosis at two PEPFAR sites in Durban, South Africa. PLoS One. 2010;5:e9538.
30. Jani IV, Sitoe NE, Alfai ER, et al.. Effect of point-of-care CD4 cell count tests on retention of patients and rates of antiretroviral therapy initiation in primary health clinics: an observational cohort study. Lancet. 2011;378:1572–1579.