The frequency of virological failure by risk factor and the crude LHRs are shown in Table 2. Table 3 shows the adjusted LHRs and the resulting score. Predictors of a viral load >1000 copies per milliliter were prior ART exposure, CD4 count below baseline, a 25% or 50% drop from the peak CD4 count, a hemoglobin drop of ≥1 g/dL, an absolute CD4 count of <100 cells per microliter, a new onset of papular pruritic eruption, and VAS <95%. SMAQ was not predictive of treatment failure.
The total predictor score per patient ranged from 0 to 5. The percentage of virological failure in the different score groups is shown in Figure 2. Using as cutoff for virological failure a viral load >10,000 copies per milliliter (as proposed by WHO),12 the same predictors and predictor scores were obtained (data not shown).
The sensitivity, specificity, and PPV of the different score cutoffs to predict viral loads >1000 copies per milliliter in comparison with the stringent and lenient WHO criteria are shown in Table 4. A score ≥2 seems to provide the optimal combination of sensitivity and specificity if we assume that the medical consequences of a false-positive or false-negative classification have a similar weight. This cutoff has a sensitivity (41.4%) similar to the WHO lenient criteria but has a better specificity (92.6% versus 75.9%). Based on the PPV, we distinguish 3 risk categories for virological failure: score 0-1, low probability (n = 1640, 91%); score 2-4, intermediate probability (n = 160, 8.7%); and score ≥5, high probability (n = 3, 0.2%).
In a consensus meeting with SHCH physicians, we developed a 2-step algorithm based on a predictor score (Fig. 3A) to target the viral load testing and applied this algorithm to the original dataset (Fig. 3B). Two patient categories were considered not to require viral load measurements: those with a low probability (score 0 or 1) and those with a high probability (score ≥5) of virological failure. Together, they accounted for 91% of all observations. For the remaining 9% with an intermediate probability of failure, it was suggested to perform viral load testing to confirm who is on a truly failing regimen. The sensitivity of this algorithm was similar to the lenient WHO criteria (41.4%) but required less viral load measurements (9% versus 24.9% following WHO lenient criteria, Fig. 3C).
Wherever viral load assays are easily accessible and affordable, a more sensitive approach could be adopted. If viral load testing is performed in all patients with a score of ≥1, this would result in a better sensitivity (63.2%), but many more assays would be required (636 or 35.3% of visits).
Our research showed that a scoring system based only on clinical, immunological, and adherence data but without viral load testing was inadequate to predict first-line treatment failure. A threshold score of ≥2 had a sensitivity of 41.4% and a specificity of 92.6%, with a PPV of 22.1%. With a prevalence of failure fixed at the observed level of 4.8%, this means that for every 100 visits, 3 treatment failures will not be detected and 7 premature switches will occur. Therefore, we developed a 2-step algorithm based on the score followed by viral load testing in those with an intermediate risk. This algorithm reduced the false-positive rate to 0% and the overall misclassification to 3% (false-negatives), whereas a viral load was needed in only 9% of patient visits. These features compare favorably with the WHO criteria for the assessment of treatment failure. We believe that our approach is a feasible and effective strategy in LMICs. Nonetheless, the performance of our algorithm is likely to be site and time dependent. The median time of follow-up on ART in this Cambodian cohort was 18 months, and the number of patients failing first-line ART at 18 months was less than 10%. Although this low failure rate confirmed the high efficacy of first-line ART as described in other studies in LMIC,17,27-30 it also contributed to the high negative predictive value of our scoring system. Failure rates and follow-up duration may vary though, and validation studies of the algorithm are therefore needed in other populations, including in cohorts with longer periods of follow-up.
Another limitation of our study is that the definition of virological failure was based on 1 viral load measurement only. A study in South Africa has shown that 53% of patients with a viral breakthrough returned to undetectable viral loads after a targeted adherence intervention.31 Studies have shown that in LMICs, between 20% and 50% of patients with a detectable viral load have no major resistance mutations.17,32 WHO recommends that patients should only be switched to second-line ART if a CD4 count decrease is confirmed by a repeat CD4 count and if the CD4 count is below 200 cells per microliter.12 In our study, we did not repeat CD4 counts. When we restricted switching only to patients with CD4 count <200 cells per microliter, the WHO criteria (stringent) would still have a false-positive rate of 82.9%.
We used patient visits, not patients, as units of observation in our analysis. In certain patients, up to 3 viral load time points were available for analysis. While we could have restricted the analysis to 1 viral load time point per patient, we preferred to maximize the use of available information. Predictors for treatment failure were selected based on point estimates of effect size (LHR), which are unaffected by correlations between observations. Therefore, the dependence between observations for a same patient did not influence the scoring system.
Changes in total lymphocyte count were not predictive for failure, which confirms the findings of other studies.44-45 We found no association between weight loss and treatment failure. Moreover, clinical stage 3 and 4 conditions were not predictive of treatment failure. This is probably explained by the early detection of treatment failure, the use of cotrimoxazole prophylaxis, or because some stage 3 and 4 conditions were late-onset immune reconstitution inflammatory syndrome events. The most frequent stage 3 and 4 conditions were >10% weight loss, bacterial infections, and chronic genital herpes.
What are the possible implications of our study for clinicians and policy makers? In LMICs, where treatment options are limited, patients should be able to benefit for as long as possible from first-line regimens. Although the sensitivity of our algorithm to detect treatment failure is low, the predictor score can easily be repeated every 6 months. Low sensitivity leads to delayed diagnosis of treatment failure. Late switch will lead to accumulation of resistance mutations, which is happening at all levels of viral load in patients who are kept on a failing regimen.46-50 Data from Malawi suggest that when first-line ART failure diagnosis is based exclusively on clinical and immunological monitoring, extensive resistance to nucleoside and non-nucleoside reverse transcriptase inhibitors is present, impacting future treatment options.51
On the other hand, there are hardly any data from LMICs on the risk of disease progression or death in patients who had a late compared with an early treatment switch.52 A study in Uganda showed that after a median follow-up of 3 years, there was no difference in the rate of AIDS-defining events or death in the group with viral load measurements compared with those without.53 A recent computer-simulation model by Phillips et al54 showed that the effect on survival of having access to viral load testing was negligibly small. Both studies suggest that late treatment switches do not have a large effect on survival rates, which leads to a conservative approach in terms of viral load testing. The benefits of targeted viral load testing however were not discussed in the article by Philips et al.55 Moreover, using a new WHO stage 4 event as a criterion for switching, as proposed by Phillips et al, would result in our cohort in a 93.6% premature switching to second-line treatment. Taking into account the current high cost of second-line treatment, it would be cost-effective to try to avoid this with a system of targeted viral load testing.56 In settings where resources are limited, we propose that clinicians and policy makers adopt such an approach based on a treatment failure risk assessment. Further operational research is needed to determine the performance of the algorithm in other settings to study the evolution of scores over time, the rapidity of changes in score, and the acceptability of the use of such an algorithm by clinicians in LMICs.
Clinical and immunological criteria have low diagnostic accuracy in identifying patients in need of second-line ART. Systematic assessment of viral load at regular intervals is a reliable method for detecting treatment failure early, but it is expensive and technically demanding. Cheaper and simple viral load assays are needed. Meanwhile, targeted viral load testing in a subgroup of patients with an intermediate risk of treatment failure in which the diagnostic benefit is greatest may be a feasible and effective strategy in LMICs. Our 2-step algorithm based on a predictor score coupled with targeted viral load testing is now being implemented at our study site in Cambodia. This predictor score and algorithm require further evaluation and possibly adaptation for use in other settings.
We would like to thank the patients and the doctors of the SHCH who participated in the study. We thank Marianne Mangelschots and Teav Syna for their excellent management of viral load samples. We thank Anne Buvé for the opportunity to participate in the Europe AID project.
3. Ministry of Health Cambodia, NCHADS, Surveillance Unit. Report of a Consensus Workshop: HIV Estimates and Projections for Cambodia, 2006-2012. 2008. Available at: http://www.nchads.org/Events/28062007.php
. Accessed December 2007.
5. Gazzard B, Bernard AJ, Boffito M, et al. British HIV Association (BHIVA) guidelines for the treatment of HIV-infected adults with antiretroviral therapy (2006). HIV Med
6. Kent DM, McGrath D, Ioannidis JP, et al. Suitable monitoring approaches to antiretroviral therapy in resource-poor settings: setting the research agenda. Clin Infect Dis
8. Calmy A, Ford N, Hirschel B, et al. HIV viral load monitoring in resource-limited regions: optional or necessary? Clin Infect Dis
9. Schooley RT. Viral load testing in resource-limited settings. Clin Infect Dis
10. Bagchi S, Kempf MC, Westfall AO, et al. Can routine clinical markers be used longitudinally to monitor antiretroviral therapy success in resource-limited settings? Clin Infect Dis
11. Gilks CF, Crowley S, Ekpini R, et al. The WHO public-health approach to antiretroviral treatment against HIV in resource-limited settings. Lancet
13. An S, Koole O, Haverkamp M, et al. Predictors of virological failure in a Cambodian setting: findings from a cross-sectional study at Sihanouk Hospital Centre of HOPE, Phnom Penh [WEAB103]. Presented at: 4th IAS Conference on HIV Pathogenesis and Treatment; July 22-25, 2007; Sydney, Australia.
14. Van Griensven J, Corthouts A, Atté EF, et al. Validity of WHO-recommended immunological criteria for treatment failure . Presented at: 2007 HIV/AIDS Implementers' Meeting; June 16-19, 2007; Kigali, Rwanda.
15. Hosseinipour M, van Oosterhout J, Weigel R, et al. Validating clinical and immunological definitions of antiretroviral treatment failure in Malawi [WEAB101]. Presented at: 4th IAS Conference on HIV Pathogenesis and Treatment; July 22-25, 2007; Sydney, Australia.
16. Colebunders R, Moses KR, Laurence J, et al. A new model to monitor the virological efficacy of antiretroviral treatment in resource-poor countries. Lancet Infect Dis
17. Ferradini L, Laureillard D, Prak N, et al. Positive outcomes of HAART at 24 months in HIV-infected patients in Cambodia. AIDS
18. Lawn SD, Orrell C, Wood R. Evaluating a model for monitoring the virological efficacy of antiretroviral treatment in resource-limited settings. Lancet Infect Dis
20. Knobel H, Alonso J, Casado JL, et al. Validation of a simplified medication adherence questionnaire in a large cohort of HIV-infected patients: The GEEMA Study. AIDS
21. Giordano TP, Guzman D, Clark R, et al. Measuring adherence to antiretroviral therapy in a diverse population using a visual analog scale. HIV Clin Trials
22. Seymour DG, Green M, Vaz FG. Making better decisions: construction of clinical scoring systems by the Spiegelhalter-Knill-Jones approach. BMJ
23. Spiegelhalter DJ. Statistical methodology for evaluating gastrointestinal symptoms. Clin Gastroenterol
24. Berkley JA, Ross A, Mwangi I, et al. Prognostic indicators of early and late death in children admitted to district hospital in Kenya: Cohort study. BMJ
25. Zhou XH, Obuchowski NA. Statistical Methods in Diagnostic Medicine
(Wiley Series in Probability and Statistics). New York, NY: Wiley-Interscience; 2002.
26. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess
27. Madec Y, Laureillard D, Pinoges L, et al. Response to highly active antiretroviral therapy among severely immuno-compromised HIV-infected patients in Cambodia. AIDS
28. Calmy A, Pinoges L, Szumilin E, et al. Generic fixed-dose combination antiretroviral treatment in resource-poor settings: multicentric observational cohort. AIDS
29. Ferradini L, Jeannin A, Pinoges L, et al. Scaling up of highly active antiretroviral therapy in a rural district of Malawi: an effectiveness assessment. Lancet
30. Braitstein P, Brinkhof MW, Dabis F, et al. Mortality of HIV-1-infected patients in the first year of antiretroviral therapy: comparison between low-income and high-income countries. Lancet
31. Orrell C, Harling G, Lawn SD, et al. Conservation of first-line antiretroviral treatment regimen where therapeutic options are limited. Antivir Ther
32. Marconi VC, Sunpath H, Lu Z, et al. Prevalence of HIV-1 drug resistance after failure of a first highly active antiretroviral therapy regimen in KwaZulu Natal, South Africa. Clin Infect Dis
33. Bisson GP, Gross R, Strom JB, et al. Diagnostic accuracy of CD4 cell count increase for virologic response after initiating highly active antiretroviral therapy. AIDS
34. Meya D, Tibenderana H, John L, et al. Evaluation of clinical and laboratory parameters to predict viral response to ART in Uganda . Presented at: 14th Conference on Retroviruses and Opportunistic Infections; February 25-28, 2007; Los Angeles, CA.
35. Moore DM, Mermin J, Awor A, et al. Performance of immunologic responses in predicting viral load suppression: implications for monitoring patients in resource-limited settings. J Acquir Immune Defic Syndr
36. Berhane K, Karim R, Cohen MH, et al. Impact of highly active antiretroviral therapy on anemia and relationship between anemia and survival in a large cohort of HIV-infected women: women's interagency HIV study. J Acquir Immune Defic Syndr
37. Moore RD, Forney D. Anemia in HIV-infected patients receiving highly active antiretroviral therapy. J Acquir Immune Defic Syndr
38. Weiser S, Wolfe W, Bangsberg D, et al. Barriers to antiretroviral adherence for patients living with HIV infection and AIDS in Botswana. J Acquir Immune Defic Syndr
39. Weidle PJ, Malamba S, Mwebaze R, et al. Assessment of a pilot antiretroviral drug therapy programme in Uganda: patients' response, survival, and drug resistance. Lancet
40. Nachega JB, Hislop M, Dowdy DW, et al. Adherence to highly active antiretroviral therapy assessed by pharmacy claims predicts survival in HIV-infected South African adults. J Acquir Immune Defic Syndr
41. Bisson GP, Gross R, Bellamy S, et al. Pharmacy refill adherence compared with CD4 count changes for monitoring HIV-infected adults on antiretroviral therapy. PLoS Med
42. Perno CF, Ceccherini-Silberstein F, De Luca A, et al. Virologic correlates of adherence to antiretroviral medications and therapeutic failure. J Acquir Immune Defic Syndr
. 2002;31(Suppl 3):S118-S122.
43. Castelnuovo B, Byakwaga H, Menten J, et al. Can response of a pruritic papular eruption to antiretroviral therapy be used as a clinical parameter to monitor virological outcome? AIDS
44. Florence E, Schrooten W, Van Esbroek M, et al. The role of non-viral load surrogate markers in HIV-positive patient monitoring during antiviral treatment. HAART monitoring without viral load. Int J STD AIDS
45. Mahajan AP, Hogan JW, Snyder B, et al. Changes in total lymphocyte count as a surrogate for changes in cd4 count following initiation of HAART: implications for monitoring in resource-limited settings. J Acquir Immune Defic Syndr
46. Kantor R, Shafer RW, Follansbee S, et al. Evolution of resistance to drugs in HIV-1-infected patients failing antiretroviral therapy. AIDS
47. Kristiansen TB, Pedersen AG, Eugen-Olsen J, et al. Genetic evolution of HIV in patients remaining on a stable HAART regimen despite insufficient viral suppression. Scand J Infect Dis
48. Cozzi-Lepri A, Phillips AN, Ruiz L, et al. Evolution of drug resistance in HIV-infected patients remaining on a virologically failing combination antiretroviral therapy regimen. AIDS
49. Napravnik S, Edwards D, Stewart P, et al. HIV-1 drug resistance evolution among patients on potent combination antiretroviral therapy with detectable viremia. J Acquir Immune Defic Syndr
50. Riddler SA, Jiang H, Tenorio A, et al. A randomized study of antiviral medication switch at lower- versus higher-switch thresholds: AIDS Clinical Trials Group Study A5115. Antivir Ther
51. Hosseinipour M, van Oosterhout JJ, Weigel R, et al. Resistance profile of patients failing first line ART in Malawi when using clinical and immunological monitoring [TUAB0105]. Presented at: XVII International AIDS conference, August 3-8, 2008, Mexico City, Mexico.
52. Humphreys EH, Hernandez LB, Rutherford GW. Antiretroviral regimens for patients with HIV who fail first-line antiretroviral therapy. Cochrane Database Syst Rev
53. Coutinho A, Mermin J, Ekwaru J, et al. Utility of routine viral load, CD4 cell count, and clinical monitoring among HIV-infected adults in Uganda: a randomized trial . Presented at: 15th Conference on Retroviruses and Opportunistic Infections; February 3-6, 2008; Boston, MA.
54. Phillips AN, Pillay D, Miners AH, et al. Outcomes from monitoring of patients on antiretroviral therapy in resource-limited settings with viral load, CD4 cell count, or clinical observation alone: a computer simulation model. Lancet
55. Moore DM, Mermin J. Monitoring antiretroviral failure in resource-poor settings. Lancet