Secondary Logo

Journal Logo

Clinical Science

A Classifier to Predict Viral Control After Antiretroviral Treatment Interruption in Chronic HIV-1–Infected Patients

Fehér, Csaba MDa,b,c; Plana, Montserratc; Crespo Guardo, Albertoc; Climent, Nuriac; Leal, Lornab,c; Ugarte, Ainoab; Fernández, Ireneb; Etcheverry, María F.b; Gatell, Josep M.d; Sánchez-Palomino, Sonsolesc; García, Felipeb; Aloy, Patricka,e

Author Information
JAIDS Journal of Acquired Immune Deficiency Syndromes: April 15, 2020 - Volume 83 - Issue 5 - p 479-485
doi: 10.1097/QAI.0000000000002281



Despite the immense efforts invested in the development of a therapeutic vaccine against HIV-1, long-term viral control has not yet been achieved by any of the therapeutic vaccine candidates.1 Analytical treatment interruptions (ATI) are currently the only available method to reliably evaluate treatment efficacy in immune-based therapies aiming to achieve a functional cure in HIV-1 infection.2,3 The potential safety risks notwithstanding, this strategy is considered essential in HIV cure research by researchers and patients alike.4,5

An early surrogate marker of viral control after withdrawing antiretroviral treatment (ART) would be an extremely useful tool in therapeutic vaccine strategies to avoid unnecessary delays in ART reinitiation, thus improving study safety. Various viral and host factors, such as baseline viral reservoir,6 certain HLA profiles,7 or different T-cell–associated cytokines,8 have previously been associated with different parameters of post-ATI viral rebound. The practical utility of these proposed biomarkers, however, has not yet been demonstrated.

In this study, we aimed to identify significant, early assessable markers of viral control after ATI. To improve the predictive power of the identified significant parameters, we constructed a naive Bayes classifier based on a combination of these variables. We used data from a dendritic cell–based therapeutic vaccine trial (DCV2)9 to select the significant predictors and to build the classifier and a historical cohort of ATI episodes collected from 6 previously published studies as validation cohort.10–15


The DCV2 trial was a partially successful therapeutic vaccine trial conducted by our group.9 In this study, 36 patients on successful ART and with CD4+ >450 cells/mL were randomized to a blinded protocol to receive 3 immunizations (separated by 2-week intervals) with peripheral blood monocyte-derived dendritic cells (MD-DC) pulsed with autologous heat-inactivated HIV-1 virions (n = 24) or with nonpulsed MD-DC (n = 12) according to the same schedule. ART was stopped on the day of the last immunization, and patients were followed for 48 weeks afterward. Viral load (VL) rebounded in all patients during this period. One of the primary endpoints of the DCV2 study was the drop of VL set point after 12 weeks of ATI with respect to pre-ART VL (delta VL12), which was significantly greater in vaccines than in control patients [−0.91 (SD 0.11) log10 copies/mL vs. −0.39 (SD 0.18) log10 copies/mL; P = 0.01].9 For the purposes of our present analysis, participants were classified as “controllers” (delta VL12 >1 log10 copies/mL) and “noncontrollers” (delta VL12 < 0.5 log10 copies/mL). To avoid the risk of misclassification in cases with near-cutoff values due to possible laboratory technique inaccuracies,16 we did not include patients with a delta VL from 0.5 to 1 log10 copies/mL in the analysis.

The following data were collected from the original study and from the patients' clinical files: (1) demographics and clinical history, (2) general biochemistry, (3) complete blood count, (4) lymphocyte phenotype subsets, (5) inflammatory markers, (6) reservoir data, (7) ELISPOT data, and (8) lymphoproliferative responses. All data were collected from a prevaccination time point (1–8 weeks before the first vaccine dose) and from a postvaccination time point (1–2 weeks after the second vaccine dose); the differences of these 2 values (“delta” variables) were also calculated. The methods for the determination of the analyzed laboratory readouts were reported elsewhere.17 For the complete list of the variables included in the analysis, see Table, Supplemental Digital Content 1,, which resumes the comparisons of all analyzed variables between controllers and noncontrollers. This study was evaluated and approved by the institutional ethical board of the Hospital Clinic of Barcelona (HCB/2015/0763).

Optimal cutoff values for the variables significantly associated with viral control were determined by Youden J statistics—a commonly used index to determine the cutoff value that maximizes the discriminatory accuracy of a diagnostic test, and these cutoffs were confirmed by leave-p-out cross-validation (P = 5). Next, naive Bayes classifiers were constructed from all the possible combinations of these significant variables. The optimal model was selected taking into account the following criteria: (1) good discriminative power, (2) no/low correlations between components, (3) components preferably belonging to the same time point (prevaccination, postvaccination, or delta variable), and (4) minimum number of component variables.

External validation of the model was performed on a historical cohort comprising of ATI episodes documented in 6 previously published studies,10–15 using the same virological endpoint as described above (delta VL12). In addition, a sensitivity analysis was performed (substituting the missing VL data at week 12 by the last available VL—“last observation carried forward” method), and the performance of the model was also tested for an alternative virological endpoint: delta set point. Delta set point was defined as the difference between pre-ART VL and the mean value of all available VL values after reaching a steady state with a margin of 0.5 log10 copies/mL.

The analysis was performed in R (version 3.4.1, R Foundation for Statistical Computing, Vienna, Austria) using RStudio (version 1.0.143, RStudio Inc., Boston, MA). Continuous and discrete variables were given in median and interquartile range (IQR) and in absolute numbers and percentage, respectively. To compare variables between controllers and noncontrollers, the Mann–Whitney U test and Fisher exact test were used for appropriate data types. Pairwise correlations between significant variables were evaluated by Spearman correlation coefficients. The naive Bayes classifiers were built using the R package e1071.


Twenty-two participants of the DCV2 trial were classified as controllers (n = 12) or noncontrollers (n = 10), while 13 patients with a delta VL between 0.5 and 1 log10 copies/mL were excluded from the analysis. Five (22.7%) of them were women, and the median age was 40.5 years (IQR 39.25–45.00 years). The demographic and clinical characteristics of the patients are shown in Table 1.

Basic Characteristics of the Derivation Data Set and the Validation Cohort

Identification of Significant Variables

From the parameters analyzed (see Table, Supplemental Digital Content 1,, we found that pre-ART VL and some prevaccination and postvaccination lymphocyte subsets were significantly associated with a control of VL after ART discontinuation. We observed a significantly higher pre-ART VL in controllers than in noncontrollers [110,250 (IQR 71,968–275,750) vs. 28,600 (IQR 18,737–39,365) copies/mL, respectively; P = 0.003]. The following T-lymphocyte subsets were significantly less abundant in controllers than in noncontrollers at the prevaccination timepoint: CD4+CD45RA+RO+ [1.72% (IQR 0.61%–3.87%) vs. 7.47% (IQR 5.12%–13.26%), P = 0.036]; CD8+CD45RA+RO+ [7.92% (IQR 3.97%–12.77%) vs. 15.69% (IQR 14.19%–18.78%), P = 0.017]; CD4+CCR5+ [4.25% (IRQ 1.80%–5.76%) vs. 7.40% (IQR 5.94%–10.15%), P = 0.011]; and CD8+CCR5+ [14.53% (IQR 11.65%–21.60%) vs. 27.30% (IQR 17.45%–29.93%), P = 0.043]. In addition, the proportion of a postvaccination T-lymphocyte subset was also significantly lower in controllers than in noncontrollers: CD4+CXCR4+ [12.44% (IQR 8.59%–23.07%) vs. 22.80% (IQR 20.69%–39.30%), P = 0.021]. There were no statistically significant differences between controllers and noncontrollers in any other parameters. The threshold that optimally differentiates between controllers and noncontrollers (ie, that minimizes misclassification rate) was determined for each significant parameter by means of the Youden index. The distributions of the 6 significant parameters with the corresponding optimal cutoff values are shown in Figure 1 (see Figure, Supplemental Digital Content 2,, which shows the results of the cross-validation of the optimal cutoff values). At the selected cutoff values, the overall accuracies of these 6 variables to differentiate between responders and nonresponders were 0.82 for pre-ART VL, prevaccine CD4+CD45RA+RO+, prevaccine CD8+CD45RA+RO+ T cells, and postvaccine CD4+CXCR4+ T cells and 0.77 for prevaccine CD4+CCR5+ and CD8+CCR5+ T cells.

The distribution of significantly different parameters between the controller and noncontroller groups. The corresponding optimal cutoffs are indicated with a red line. A, Pre-ART VL, (B) prevaccine CD4+CD45RA+RO+ lymphocytes, (C) prevaccine CD4+CCR5+ lymphocytes, (D) prevaccine CD8+CD45RA+RO+ lymphocytes, (E) prevaccine CD4+CCR5+ lymphocytes, and (F) postvaccine CD4+CXCR4+ lymphocytes.

Building and Selecting the Optimal Classifier

To further improve discriminative capacity, we decided to combine the significant variables in a naive Bayes classifier. A naive Bayes classifier is a simple supervised machine learning algorithm that shows a good predictive performance even with a relatively small training data set. To keep our model simple, we decided to focus exclusively on the 5 prevaccination parameters. Since the components of a naive Bayes classifier should ideally be independent from each other, we established pairwise Spearman correlations between these 5 parameters. Significant correlations were observed between some CD4+ and CD8+ T-lymphocyte subsets [CD4+CD45RA+RO+ vs. CD8+CD45RA+RO+ (rho = 0.822, P < 0.001); CD4+CCR5+ vs. CD8+CCR5+ (rho = 0.670, P = 0.001)] and between pre-ART VL and prevaccination CD4+ T-lymphocyte subsets [pre-ART VL vs. CD4+CD45RA+RO+ (rho = −0.436, P = 0.049); pre-ART VL vs. CD4+CCR5+ (rho = −0.577, P = 0.007)] (see Figure, Supplemental Digital Content 3,, which illustrates the pairwise correlations between the 5 significant prevaccination predictors of viral control).

We built naive Bayes classifiers to predict viral control from all possible combinations of the above 5 parameters. Nine of these models had the highest observed overall accuracy of 0.91 (see Table, Supplemental Digital Content 4,, which summarizes the performance measures of the 31 candidate classifiers); we discarded 3 of these for having the greatest number of components, including highly correlated ones (CD4+ and CD8+ CD45RA+RO+ T lymphocytes). After cross-validating the remaining 6 models (see Figure, Supplemental Digital Content 5,, which illustrates the cross-validation of the 6 candidate classifiers), we selected the one based on pre-ART VL and the relative abundance of CD8+CD45RA+RO+ T lymphocytes as the most robust one with the lowest number of components. This classifier identified controllers with 92% sensitivity and 90% specificity in the DCV2 cohort. Its positive predictive value for viral control was 92%, and its negative predictive value was 90%, yielding an overall accuracy (ie, the proportion of correctly classified cases) of 91%.

Validation of the Classifier

For the external validation of the classifier, we selected 6 previously published studies with similar ATI episodes. Among the participants of 6 previous ATI studies, we identified 148 ATI episodes where the predictive model could be applied (ie, both preinterruption CD8+CD45RA+RO+ T-lymphocyte data and pre-ART VL were available). Among these, data on VL at week 12 of ATI were available in 134 cases, and 107 of them could be categorized as controllers or noncontrollers according to our study definitions. These 107 cases constituted the validation cohort. In 16 (15.0%) of these cases, an immunological intervention accompanied the ATI episode: 12 patients received a similar dendritic cell–based therapeutic vaccine as the patients in the DCV2 trial,11 and 4 patients were treated with mycophenolate mofetil.14Table 1 summarizes the basic characteristics of these patients grouped by source study and their comparison with the derivation data set.

Applying the predictive model that we selected earlier, based on pre-ART VL and the relative abundance of CD8+CD45RA+RO+ T lymphocytes, on the validation cohort, we observed a sensitivity of 88% and a negative predictive value of 94%. However, only 1 of every 5 predicted controllers proved to be real controllers (positive predictive value: 20%), which resulted in a low overall accuracy of the model in the validation data set (42%). The performance of the model was similar when we applied it to different subsets of the validation cohort (grouped by source study or by the use of immunological interventions). (Figs. 2A, B).

Sensitivity, specificity, positive and negative predictive values, and overall accuracy of the predictive model in different subgroups of the validation data set and using alternative virological outcome definitions. Shape size is indicative of group size; shape color indicates the proportion of responders within each group according to the gradient legend. A, Model performance in individual studies of the validation cohort, (B) model performance at week 12 and at the set point, with or without the use of immunological intervention, and (C) model performance in the sensitivity analysis at week 12 and at the set point, with or without the use of immunological intervention.

For the sensitivity analysis, we substituted the missing week 12 VL data for the last available VL value (last observation carried forward method), thus having an endpoint value for all 148 cases included. From these, 120 patients qualified as controllers or noncontrollers. The set point could be estimated for 57 cases, and this number increased to 113 applying the LOFC method. As can be observed in Figure 2B, C, the performance of the model remained fundamentally unchanged in the sensitivity and set point analyses as well.

Unlike the DCV2 study, the validation data set suffers from a great degree of data imbalance, since the proportion of controllers is rather low (17/107, 15.9%). To examine the expected performance of our model in case of higher controller prevalence, we performed an additional test. We calculated the distributions of the performance measures of 1000 subsamples of 47 cases of the validation cohort, comprising the 17 controllers and randomly selected samples of 30 noncontrollers. We observed that apart from the expectable increase of positive predictive value (Md = 43%) and overall accuracy (Md = 53%), the median negative predictive value remained good (83%) (see Figure, Supplemental Digital Content 6,, which shows the distribution of the expected performance measures in subcohorts of the validation data set with a 36% controller prevalence).


In this study, we constructed a naive Bayes classifier based on easily available baseline characteristics (pre-ART VL and the relative abundance of CD8+CD45RA+RO+ T lymphocytes) to predict viral control after treatment interruption. Although the model poorly classified patients with high probability of response (it had a low positive predictive value in the validation data set), it could reliably identify individuals with a low probability of viral control, irrespective of the immunological intervention we may apply (19 of every 20 patients identified as noncontrollers were correctly classified). The utility of this information could be great in therapeutic vaccine trials, since the classifier could be used as exclusion criteria, thus avoiding the recruitment (and ATI) of patients with low probability of response.

Pre-ART VL has previously been related to different virological and immunological outcome measures in some studies. It has been reported to be directly correlated to post-ATI VL set point,18,19 an observation that seems to contradict our results. However, it has to be taken into account that our definition of response is not an absolute number as the one used in these studies18,19 but a delta value. Although the final objective of any immunotherapy strategy is to achieve an undetectable level of VL after ATI, no clinical trial has been able to achieve this objective so far. Therefore, we need to find the best surrogate markers of response based on studies with partial response, as it is the DCV2 clinical trial. Our model is based on the results of this trial and, therefore, selects the patients with low probability of presenting a greater than 1 log10 copies/mL drop of VL with respect to the pre-ART value, not the ones with a certain probability of controlling the VL below a determined threshold. Other authors have found in a large cohort of treatment-naive patients that a higher pre-ART VL was associated with higher probability of CD4 recovery after ART initiation.20 Considering these data together with our results, one can speculate that the margin of improvement may be easier to achieve in patients with poorer baseline situation.

The other component of our classifier was the prevaccine proportion of CD8+CD45RA+RO+ T lymphocytes. The role of CD45RA+RO+ T lymphocytes has not been completely elucidated as yet. They have originally been described as recently recruited lymphocytes that are in the process of changing from CD45RA+RO- (naive cells) to CD45RA-RO+ (memory cells).21 However, it seems that memory and effector T cells cannot reliably be classified on the basis of a particular differentiation phenotype but would be better defined on the basis of their activation status and functional characterization.22

Effector memory T cells, which re-express the CD45RA antigen, are usually referred to as TEMRA, but their relevance is not yet fully understood. In CD4+ T cells, it was described that resting memory cells may start re-expressing the CD45RA antigen in the absence of antigenic stimuli, without losing CD45RO positivity.23 In another study, the re-expression of CD45RA in CD4+CD45RA-RO+ cells was observed as a consequence of gp120 stimulation, and it was shown to induce apoptosis in these activated cells.24 In addition, a report on malnourished children suggests that an elevated percentage of CD45RA+RO+ T lymphocytes may be indicative of an impaired T-cell function.25

Although probably the subset of CD8+CD45RA+RO+ T cells mentioned in our study should most probably be considered CD8+ TEMRA, the limited surface markers used to define this cell population in our study—as well as the lack of analysis of its functional properties—does not allow us to determine their real relevance in the control of HIV replication. Moreover, even if the presence of HIV-specific CD8 + T cells with an TEMRA phenotype has been previously described to be associated with HIV control in early infection,26 the scenario seems to be more complex in chronic HIV-infected patients.27

In summary, a higher proportion of CD45RA+RO+ T lymphocytes in noncontrollers in our study may suggest a greater percentage of activated and/or impaired lymphocytes in these patients. This is further supported by the fact that CCR5+ T lymphocytes were also significantly more abundant in noncontrollers than in controllers. Further research shall clear the precise nature of this T-cell population.

Our study has certain limitations. First, the derivation cohort is retrospective and has a relatively low size. Secondly, although the constructed classifier could be validated externally, the validation cohort was also retrospective. Thirdly, there is no widely used standardized method for the determination of the abundance of CD45RA+RO+ T lymphocytes, which may in theory limit the application of our model in other centers. However, we believe that their association with viral control could be relevant and should be further explored in prospective trials specifically designed to evaluate this relation. Finally, as it has been mentioned above, our model only predicts the likelihood of no response as defined as a drop of VL >1 log10 copies/mL; it should be further explored if the model could also be useful for making predictions about the probability of attaining other common efficacy endpoints.

In conclusion, the naive Bayes classifier we constructed based on easily obtainable baseline parameters could be a useful tool to improve patient recruitment criteria in future HIV cure studies. At the same time, our results call the attention on the possible role of certain lymphocyte subsets as markers of the quality of host anti-HIV immune response, although these data should be experimentally verified in the future.


1. Leal L, Lucero C, Gatell JM, et al. New challenges in therapeutic vaccines against HIV infection. Expert Rev Vaccin. 2017;16:587–600.
2. Anderson JL, Fromentin R, Corbelli GM, et al. Progress towards an HIV cure: update from the 2014 international AIDS society symposium. AIDS Res Hum Retroviruses. 2015;31:36–44.
3. Julg B, Dee L, Ananworanich J, et al. Recommendations for analytical antiretroviral treatment interruptions in HIV research trials—report of a consensus meeting. Lancet HIV. 2019;3018:1–10.
4. Dube K, Evans D, Dee L, et al. “We need to deploy them very thoughtfully and carefully”: perceptions of analytical treatment interruptions in HIV cure research in the United States—a qualitative inquiry. AIDS Res Hum Retroviruses. 2018;34:67–79.
5. Jeffreys R, Treatment Action Group. Community Recommendations for Clinical Research Involving Antiretroviral Treatment Interruptions in Adults. 2018. Available at: Accessed January 17, 2019.
6. Yerli S, Günthard HF, Fagard C, et al. Proviral HIV-DNA predicts viral rebound and viral setpoint after structured treatment interruptions. Age Aging Res Lett. 2004;18:1951–1964.
7. Park YJ, Etemad B, Ahmed H, et al. Impact of HLA class I alleles on timing of HIV rebound after antiretroviral treatment interruption. Pathog Immun. 2017;2:431–445.
8. Huang Y, Pantaleo G, Tapia G, et al. Cell-mediated immune predictors of vaccine effect on viral load and CD4 count in a phase 2 therapeutic HIV-1 vaccine clinical trial. EBioMedicine. 2017;24:195–204.
9. García F, Climent N, Guardo AC, et al. A dendritic cell-based vaccine elicits T cell responses associated with control of HIV-1 replication. Sci Transl Med. 2013;5:166ra2.
10. León A, Martinez E, Milinkovic A, et al. Influence of repeated cycles of structured therapy interruption on the rate of recovery of CD4+ T cells after highly active antiretroviral therapy resumption. J Antimicrob Chemother. 2009;63:184–188.
11. García F, Lejeune M, Climent NN, et al. Therapeutic immunization with dendritic cells loaded with heat-inactivated autologous HIV-1 in patients with chronic HIV-1 infection. J Infect Dis. 2005;191:1680–1685.
12. Castro P, Plana M, González R, et al. Influence of a vaccination schedule on viral load rebound and immune responses in successfully treated HIV-infected patients. AIDS Res Hum Retroviruses. 2009;25:1249–1259.
13. García F, Plana M, Ortiz GM, et al. The virological and immunological consequences of structured treatment interruptions in chronic HIV-1 infection. AIDS. 2001;15:F29–F40.
14. García F, Plana M, Arnedo M, et al. Effect of mycophenolate mofetil on immune response and plasma and lymphatic tissue viral load during and after interruption of highly active antiretroviral therapy for patients with chronic HIV infection: a randomized pilot study. J Acquir Immune Defic Syndr. 2004;36:823–830.
15. Fagard C, Oxenius A, Günthard H, et al. A prospective trial of structured treatment interruptions in human immunodeficiency virus infection. Arch Intern Med. 2003;163:1220–1226.
16. Fiscus S, Denny T, Habiyambere V, et al. Systematic review of the performance of HIV viral load technologies on plasma samples. PLoS One. 2014;9:e85869.
17. Andrés C, Plana M, Guardo AC, et al. HIV-1 reservoir dynamics after vaccination and antiretroviral therapy interruption are associated with dendritic cell-vaccine induced T-cell responses. J Virol. 2015;89:9189–9199.
18. Treasure GC, Aga E, Bosch RJ, et al. Relationship among viral load outcomes in HIV treatment interruption trials. J Acquir Immune Defic Syndr. 2016;72:1.
19. Plana M, Garcia F, Oxenius A, et al. Relevance of HIV-1-specific CD4+helper T-cell responses during structured treatment interruptions in patients with CD4+T-cell nadir above 400/mm3. J Acquir Immune Defic Syndr. 2004;36:791–799.
20. Roul H, Mary-Krause M, Ghosn J, et al. CD4 cell count recovery after combined antiretroviral therapy in the modern cART era. AIDS. 2018;32:2605–2614.
21. Co MDT, Kilpatrick ED, Rothman AL. Dynamics of the CD8 T-cell response following yellow fever virus 17D immunization. Immunology. 2009;128:e718–e727.
22. Appay V, Dunbar PR, Callan M, et al. Memory CD8+ T cells vary in differentiation phenotype in different persistent virus infections. Nat Med. 2002;8:379–385.
23. Hamann D, Baars PA, Hooibrink B, et al. Heterogeneity of the human CD4+ T-cell population: two distinct CD4+ T-cell subsets characterized by coexpression of CD45RA and CD45RO isoforms. Blood. 1996;88:3513–3521.
24. Trushin SA, Bren GD, Badley AD. CD4 T cells treated with gp120 acquire a CD45R0+/CD45RA+ phenotype. Open Virol J. 2009;3:21–25.
25. Nájera O, González C, Toledo G, et al. CD45RA and CD45RO isoforms in infected malnourished and infected well-nourished children. Clin Exp Immunol. 2001;126:461–465.
26. Northfield JW, Loo CP, Barbour JD, et al. Human immunodeficiency virus type 1 (HIV-1)-Specific CD8+ TEMRA cells in early infection are linked to control of HIV-1 viremia and predict the subsequent viral load set point. J Virol. 2007;81:5759–5765.
27. Champagne P, Ogg GS, King AS, et al. Skewed maturation of memory HIV-specific CD8 T lymphocytes. Nature. 2001;410:106–111.

viral load; HIV-1; supervised machine learning; T-lymphocyte subsets

Supplemental Digital Content

Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.