With the success of highly active antiretroviral therapy (HAART), concerns regarding HIV infection have shifted toward comorbid illness, longer term outcomes, and aging, although premature death remains the most serious concern. Prognostic tools to help identify those at risk of these adverse outcomes have been developed and several have been validated, usually in study cohorts comprised of subsets of the general population, for example, military service veterans, inner city gay men, intravenous drug users, and others. As HIV infection in the United States has spread broadly, evaluation of such tools among a population of young, otherwise healthy, ethnically diverse, physically active individuals is important to assess the generalizability of these models. The US Military HIV Natural History Study (NHS) cohort is primarily composed of active duty service members found to be HIV infected on routine screening and thus have early diagnosis and entry to care. Health care is a military benefit provided along with medications at no cost to the individual. Active duty service is also associated with relatively stable income/socioeconomic status, education at a high school level or above, and near absence of intravenous drug use.
The Veterans Aging Cohort Study (VACS) index has been validated among many cohorts as an excellent predictor of all-cause mortality when applied to any single time point from 1 to 5 years after initiation of HAART in HIV-infected individuals1,2 and strongly associated with biomarkers of inflammation.3–6 However, the majority of subjects in these cohorts have been older or with a high prevalence of comorbidities. The index is comprised of measures capturing standard HIV-associated mortality risk (age, CD4, HIV viral load), those identifying comorbidity and organ system dysfunction (hemoglobin, aspartate aminotransferase, alanine aminotransferase, platelet count, and creatinine levels), and hepatitis C virus serostatus. Evaluation of the index has yet to take into account previous VACS index scores.
Our purpose was 2-fold. We first sought to evaluate the predictive utility of the VACS index for death in a young, otherwise healthy, military HIV population. Second, we investigated whether considering VACS index values at previous time points provides a better prediction of the 5-year mortality risk given the VACS index value at 1 year.
The US Military HIV NHS is a prospective continuous enrolment cohort study of consenting military beneficiaries with HIV infection, including active duty personnel, retirees, and dependents.7 Active duty personnel must be HIV negative before entry into US military service and subsequently undergo routine HIV screening every 1–5 years. Those identified with HIV infection are referred to military medical centers for evaluation and treatment and are invited to enroll in the NHS. Study visits occur approximately every 6 months when data, including demographics, medical history, medications, and laboratory measurements, are collected. The NHS has been institutional review board approved at all participating sites, and all subjects provide written informed consent.
Definitions and Inclusion Criteria
Subjects in the NHS who initiated HAART between 1996 and 2011 and had available age and all components of the VACS index at HAART initiation (HI) were included in these analyses. The VACS index combines traditional factors associated with HIV outcome, including age, CD4, HIV viral load, and additional clinical factors including hemoglobin, FIB-4, estimated glomerular filtration rate, and hepatitis C virus serostatus, with a total possible score ranging from 0 to 164. VACS index was calculated according to the method of Justice et al.2 Briefly, clinically relevant variables associated with mortality among those with HIV infection are assigned points by value category, for example, hemoglobin (values in grams per deciliter) ≥14, 0 points; 12–13.9, 10 points; 10–11.9, 22 points; and <10, 38 points; and FIB-4, a measure of liver fibrosis, <1.45, 0 points; 1.45–3.25, 6 points; >3.25, 25 points.
Deaths among NHS subjects are actively ascertained through annual search of a number of US national databases, including the National Death Index, Social Security Death Index, Department of Defense, Veterans Administration, and others. Those not known to be deceased were classified as living at the end of the 2011.
The VACS index was evaluated for each subject at HI, 6 months after HI (6M), and 1 year after HI (1Y). The predictive value of the VACS index was evaluated with respect to 2 outcomes: time to death (continuous variable) and 5-year mortality (binary variable) in those with at least 5 years of potential follow-up.
The external validation was conducted as follows8,9: Correlation between the VACS index and the time to death was assessed using Harrell c statistics. Discrimination was tested by evaluating differences in time to death or 5-year mortality among the VACS index tertiles using the log-rank test and Greenwood formula after log–log transformation of the Kaplan–Meier survival curve, respectively. Finally, calibration of the model was evaluated by comparing the observed and predicted 5-year mortality at HI, 6M, and 1Y.
We then assessed whether there was an additional predictive value by combining VACS index information from the earlier 2 time points (HI and 6M) with that from the 1Y time point for both time to death and 5-year mortality. Only subjects with VACS index values at all 3 time points were included in the analysis, and multiple imputations were used to evaluate the effect of missing data. Cox models were employed, and model performance was evaluated in terms of goodness of fit (R2), statistical significance (P values), Akaike information criteria, and ability to reclassify subjects in given risk groups (net reclassification improvement, NRI). Inverse probability weighting of the cases and controls was used to account for censoring when evaluating the NRI.10,11
Additional analyses (details included in the Supplemental Digital Content, http://links.lww.com/QAI/A480) were performed to assess the sensitivity of our findings to the model considered. One approach evaluated a Cox model using principal components of the VACS index values at the 3 time points. Another approach categorized subjects into low- and high-risk groups according to their scores at HI and 6M. Results of the analyses were not sensitive to the approach used, and the more robust Cox model is presented herein.
Baseline characteristics for all subjects by VACS index tertiles are shown in Table 1. Out of the 1659 subjects, 92% were male, 40% white, 45% African American, and 14% Hispanic or other. Median CD4 count at HI was 332 (interquartile range 223–451), whereas median age was 34 years (interquartile range 28–40). Those in the lowest tertile (<14) were more likely to be male and white. Those in the highest tertile (>23) were more likely to be female and/or African American.
Among the 1659 included subjects, the VACS index was also calculated for 1594 subjects at 6M and 1724 at 1Y, whereas 1089 subjects had VACS index values at all 3 time points. Subjects without a calculated VACS index were most often missing only one of the necessary components.
There were 176 deaths among the subjects with a VACS index at HI, 145 deaths among the subjects with a VACS index at 6M, and 155 among those with a VACS index at 1Y, whereas 86 deaths were observed among the 1089 subjects with VACS index values at all 3 time points. For the 5-year mortality (binary) endpoint, there were 83, 67, and 77 deaths among the subjects with VACS index values at HI, 6M, and 1Y, respectively, and 40 deaths among the 1089 subjects with VACS index values at all 3 time points. The VACS index discriminated risk of mortality at all 3 time points with Harrell c statistics of 0.73, 0.77, and 0.78 at HI, 6M, and 1Y. Similar results were obtained after stratifying by age (see Table S1, Supplemental Digital Content, http://links.lww.com/QAI/A480). Time to death differed by VACS index tertile at all 3 time points (log-rank P < 0.05; see Figure S2, Supplemental Digital Content, http://links.lww.com/QAI/A480). Five-year mortality was not significantly different between the first 2 VACS index tertiles but was different between the second and third tertiles (P < 0.05).
Calibration of the VACS index in this cohort was assessed by comparing predicted and observed mortality. These were 10.8%, 6.8%, and 6.5% predicted for HI, 6M, and 1Y versus 2.5% (95% CI: 1.7% to 3.3%), 5.0% (3.8% to 6.3%), and 3.3% (1.6% to 5.0%) observed for the same time points, respectively. Similar overestimates were also present when the analyses were stratified by age and VACS index.
Combining VACS Index Values Over Time
Among subjects with a 5-year follow-up, 86.0% and 91.8% of the subjects in the lowest VACS index tertile at HI and 6 months, respectively, remained in the lowest tertile 1Y, whereas 51.8% and 63.1% of subjects in the highest tertile at HI and 6 months remained in the highest tertile 1Y (see Tables S2 and S3, Supplemental Digital Content, http://links.lww.com/QAI/A480). Mortality was highest among subjects in the third VACS index tertile at 1Y (29%), reinforcing the predictive power of the VACS index at this time point. Mortality was lowest among the subjects in the 2 lowest 1Y tertiles (3.3%), particularly those who started in tertile 1 or 2 at HI and were in the lowest tertile at 6M (1.7%).
Combining these observations, Cox models were used to assess the importance of the VACS index values at HI and 6M, in addition to the value at 1Y. The best model in terms of Akaike information criteria included the VACS index at HI, 6M, and 1Y and an interaction between scores at HI and 6M (Table 2). This demonstrates additional independent information contained in earlier values and in change between HI and 6M. Interaction terms between the other time points were not significant.
The independent contribution of VACS index values at HI and 6M and their interaction (to the prediction based on VACS index at 1Y) were further evaluated using NRI for 5-year mortality for subjects with a VACS index at all 3 time points (Table 3). Among those (n = 40) who died during follow-up, the Cox model, including all 3 time points, resulted in 38.7% correctly reclassified to a higher risk group and 0% to a lower risk group, for a 38.7% net positive reclassification. Among those who did not die, 2.1% were correctly reclassified to a lower risk group and 15.6% were incorrectly reclassified to a higher risk group, for a −13.5% net negative reclassification. Combining these, the overall net reclassification index was positive and significant, NRI = 25.2% (95% CI: 10.8% to 48.9%), again confirming the additional information contained in the earlier index values and the change between them. Similar results were obtained using multiple imputations to assess the effect of the missing VACS index values (see Supplemental Digital Content, http://links.lww.com/QAI/A480).
The VACS index has been well validated among HIV-infected veterans and a broad aggregate of HIV cohorts in the North American AIDS Cohort Collaboration on Research and Design and the Antiretroviral Therapy Cohort Collaboration4,6; however, younger, healthier subjects represent only a small minority of the studied subjects while accounting for a significant portion of those infected with HIV across the United States and in the world. We validated the VACS index in a relatively young, healthy HIV-infected population with low prevalence of comorbidity, early diagnosis, and entry into care in the military system with open access and free medications, low prevalence of injection drug use, relatively stable socioeconomic status, and at least a high school level of education. Using both 5-year and time to all-cause mortality, our findings showed good correlation and discrimination for the VACS index, with adequate calibration although predicted mortality was modestly overestimated compared with observed. It is likely that this results from underlying population age and health status differences. Future work in a larger sample of subjects will evaluate whether a generalization allowing model tuning or recalibration, for example, an additional age cutoff below the age of 50 years (the current VACS index model uses <50, 50–64, and ≥65), can be developed and validated. Such an addition can be useful because only 13.1% of new HIV infections and 13.7% of new AIDS diagnoses in the United States are in those over 50 years of age, whereas 50.3% of new HIV infections and 36.4% of new AIDS diagnoses occur in those 34 years or younger (Centers for Disease Control and Prevention 2010 data12).
The VACS index has been shown to be strongly associated with biomarkers of inflammation and highly predictive of all-cause mortality when applied at any point from 1 to 5 years after starting HAART.3–6 We add to this knowledge, by considering the VACS index history, along with the VACS index value at 1Y. The benefit of following the VACS index longitudinally is 2-fold. First, it allows early identification of patients both at low and high risk of mortality beginning from HI. This is consistent with early reports of a similar index to predict mortality in HIV patients initiating HAART that was studied and validated at this time point13 although subsequent work with the VACS index has shown somewhat stronger predictions using later time points after the initial response to treatment.4 Second, our results suggest that additional information can be gained by taking into account previous VACS index values along with VACS index value at 1Y. Addition of the VACS index values at HI and 6M to that at 1Y provided good NRI for those who died, improving prediction both in terms of risk magnitude and earlier identification of risk. Subjects with a lower risk at HI and 6M had a significantly lower risk of death, both unadjusted and adjusted for the VACS index at 1Y. This indicates that the VACS index at these early time points can be used as an additional independent predictor of outcome in conjunction with the VACS index at 1Y. Although we cannot yet determine the reason for this finding, we suspect that earlier changes in VACS index score reflect differences in adherence—those with excellent adherence likely respond to ART initiation more rapidly.
The additional information from these early time points is also useful to allow identification of patients at higher mortality risk who were not identified as such by the VACS index value at 1 year. An example is the 15% mortality among those subjects who moved to or remained in a higher VACS index tertile (2 or 3). Identification of patients at high mortality risk is of particular interest, and addition of the earlier VACS index information seems to increase sensitivity for detection. Such information provides a powerful tool that may become useful in the management of HIV infection.
The VACS index, a marker calculated from routinely obtained information, was shown to be well correlated with mortality and to provide good discrimination among HIV-infected subjects in the US Military HIV NHS cohort, a relatively young and healthy population representative of many people living with HIV in the United States. VACS index at the time of HI and 6 months later provides important additional information to the VACS index at 1 year and help identify high-risk patients at HI.
The investigators would like to thank their patients for their enormous contributions over the years. In addition to authors listed above, the Infectious Disease Clinical Research Program HIV Working Group includes Susan Banks RN, William Bradley MS, Helen Chun MD, Nancy Crum-Cianflone, MD, MPH, Cathy Decker, MD, Conner Eggleston, LTC Tomas Ferguson, COL Susan Fraser, MD, MAJ Joshua Hartzell, MD, MAJ Joshua Hawley, LTC Gunther Hsue, Arthur Johnson, MD, COL Mark Kortepeter, MD, MPH, Tahaniyat Lalani, MD, Robbin Lockhart, MS, Scott Merritt, LTC Robert O'Connell, MD, Sheila Peel, PhD, Michael Polis, MD, John Powers, MD, MAJ Roseanne Ressner, MD, COL(ret) Edmund Tramont, LT Tyler Warkentien, Timothy Whitman, MD, and COL Michael Zapor, MD.
1. Justice AC. HIV
and aging: time for a new paradigm. Curr HIV
/AIDS Rep. 2010;7:69–76.
2. Justice AC, Freiberg MS, Tracy R, et al.. Does an index composed of clinical data reflect effects of inflammation, coagulation, and monocyte activation on mortality among those aging with HIV
? Clin Infect Dis. 2012;54:984–994.
3. Justice AC, McGinnis KA, Skanderson M, et al.. Towards a combined prognostic index for survival in HIV
infection: the role of ‘non-HIV
' biomarkers. HIV
4. Tate JP, Justice AC, Hughes MD, et al.. An internationally generalizable risk index for mortality after one year of antiretroviral therapy. AIDS. 2013;27:563–572.
5. Akgun KM, Gordon K, Pisani M, et al.. Risk factors for hospitalization and medical intensive care unit (MICU) admission among HIV
-infected veterans. J Acquir Immune Defic Syndr. 2013;62:52–59.
6. Justice AC, Modur S, Tate JP, et al.. Predictive Accuracy of the Veterans Aging Cohort Study index for mortality with HIV
infection: a North American cross cohort analysis. J Acquir Immune Defic Syndr. 2013;62:149–163.
7. Marconi VC, Grandits GA, Weintrob AC, et al.. Outcomes of highly active antiretroviral therapy in the context of universal access to healthcare: the U.S. Military HIV
Natural History Study. AIDS Res Ther. 2010;7:14.
8. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–387.
9. Harrell F. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001.
10. Graf E, Schmoor C, Sauerbrei W, et al.. Assessment and comparison of prognostic classification schemes for survival data. Stat Med. 1999;18:2529–2545.
11. Uno H, Tian L, Cai T, et al.. A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data. Stat Med. 2012;32:2430–2442.
13. May M, Royston P, Egger M, et al.. Development and validation
of a prognostic model for survival time data: application to prognosis of HIV
positive patients treated with antiretroviral therapy. Stat Med. 2004;23:2375–2398.