For almost 30 years after the introduction of antiretroviral therapy (ART), when to start treatment has been a key decision in the care of HIV-positive people. To address this question, the recent START (Strategic Timing of AntiRetroviral Treatment) randomized trial compared the effect of immediate ART initiation with deferred initiation until a confirmed CD4+ cell count became less than 350 cells/μl or an AIDS diagnosis in individuals with CD4+ cell count became more than 500 cells/μl at randomization. The primary outcome was a composite of any serious AIDS event, serious non-AIDS event, or death from any cause. An intention-to-treat (ITT) analysis estimated a 57% lower incidence of the primary outcome in the immediate treatment group than in the deferred treatment group [hazard ratio: 0.43, 95% confidence interval (CI) 0.30, 0.62] over an average of 3 years of follow-up .
Although participants generally adhered well to the protocol, some deviations occurred. In particular, 30% of the participants randomized to the deferred arm started ART with a latest CD4+ cell count at least 350 cells/μl . Hence, the ITT effect estimate may have underestimated the benefits of immediate initiation . In addition, a small proportion (4.3%) of participants were lost to follow-up .
To complement the published ITT effect estimates in the START trial and to understand the potential underestimation of the benefit of immediate initiation resulting from premature treatment in the deferred group, we estimated the per-protocol effect. This is the effect had all participants in the trial adhered to the treatment initiation strategy they were assigned to and, unless they had experienced the primary endpoint, remained under follow-up for the duration of the study. These estimates will assist healthcare planners quantify the potential impact of immediate ART initiation for all HIV-positive individuals, which is now recommended by national and international guidelines [3–5].
The START trial has been described elsewhere . The study randomized adult HIV-positive participants with two CD4+ cell counts more than 500 cells/μl to start ART immediately after randomization (immediate initiation group), or to wait until CD4+ cell count dropped below 350 cells/μl, or an AIDS diagnosis (deferred initiation group). Overall median (IQR) time since HIV diagnosis was 1 year (0.4, 3.1). We estimated the 3- and 5-year risks after randomization of the primary outcome that would have been observed in each group if all participants had fully adhered to the protocol. That is, if they had initiated ART at the assigned time and had remained under follow-up until diagnosis of the primary outcome, death, or the administrative end of follow-up (26 May 2015). As measures of per-protocol effect, we estimated the risk difference.
Estimating the per-protocol effect requires a precise definition of what constitutes a protocol deviation. The maximum time window (grace period) between eligibility for ART initiation and ART initiation was not explicitly specified in the protocol. Therefore, we defined immediate initiation as starting ART within 1 month of randomization and deferred initiation as starting ART within 1 month of a confirmed CD4+ cell count less than 350 cells/μl. We conducted a sensitivity analysis using a grace period of 2 months. Although the protocol did not mandate ART initiation at pregnancy, it was allowed and 49 women in the deferred group started ART because of pregnancy. We therefore conducted a sensitivity analysis in which failure to initiate ART because of pregnancy was considered a protocol deviation.
Follow-up began at randomization and ended at the first of a primary endpoint, administrative end of follow-up (at 60 months or 26 May 2015), or loss to follow-up (12 months after the latest CD4+ cell count or HIV-RNA measurement). The latter form of censoring was unnecessary in the published ITT analysis, which did not rely on postrandomization information on ART and prognostic factors.
Our estimates were adjusted for the following baseline and postrandomization variables defined a priori: age (<35 and ≥35 years), CD4+ cell count (≤650 and >650 cells/μl), and HIV-RNA at randomization (<5000 and ≥5000 copies/ml), sex, geographical area (high-income regions versus low-mid income regions), the square root of the latest CD4+ cell count value, the natural logarithm of HIV-RNA value, the number of months since the last CD4+ cell count and HIV-RNA measurements, ART initiation status (never started ART versus initiated ART), and months since ART initiation.
To adjust for the above variables, we used the parametric g-formula [6,7] that, in contrast to traditional methods, appropriately adjusts for baseline and postrandomization factors associated with ART initiation and the outcome. The estimation procedure has been described elsewhere . The procedure has two steps. First, we fitted parametric regression models to estimate the joint distribution of the outcome, treatment, and time-varying covariates conditional on previous treatment and covariate history. Second, using the parameter estimates from these models, we simulated a dataset of 100 000 individuals under each of the two per-protocol initiation strategies. Finally, we computed and compared the outcome risk at 3 and 5 years in the simulated data. The immediate and deferred initiation groups were analyzed separately because the predictors of ART initiation (hazard ratios of initiation estimated using pooled logistic regression models), and hence the confounding structure, varied between groups. We used a nonparametric bootstrap procedure based on 1000 samples to obtain percentile-based 95% CIs. More specifically, for each bootstrap repetition, we estimated the regression models, repeated the simulation procedure, and estimated the risk of the outcome.
To explore the goodness-of-fit of our parametric models, we simulated a dataset under the same degree of adherence to the assigned strategies that was observed in START and compared the estimates with those from the observed data. For comparison with the published ITT analysis, in which the median follow-up was approximately 3 years, we estimated the 3-year average per-protocol hazard ratio by fitting Cox models to our simulated data. We compared this estimate with that from a naïve per-protocol analysis that censored participants when they deviated from the protocol without adjusting for any covariates and adjusting for baseline covariates only.
All analyses were conducted with the publicly available SAS macro GFORMULA (http://www.hsph.harvard.edu/causal/software/).
Of the 4685 START participants, nine were excluded from this analysis due to missing baseline HIV-RNA, resulting in a cohort of 4676. Median (IQR) follow-up was 34 (25, 46) and 33 (25, 45) months in the immediate and deferred initiation groups, respectively; 365 (8%) participants were defined as lost to follow-up due to there being at least a 12-month period in which no CD4+ or HIV-RNA was measured. The distribution of participants’ characteristics at randomization is shown in Appendix Table 1, (https://links.lww.com/QAD/A977). During 13 831 person-years of follow-up, 38 participants experienced the primary outcome in the immediate initiation group and 90 in the deferred initiation group. Ten outcome events in the original article were excluded because they occurred after the date the participants were lost to follow-up according to our definition (the ‘Methods’ section).
Of 2321 participants assigned to immediate initiation, 2061 and 2213 started ART by months 1 and 2 after randomization, respectively, and 45 never started ART. Of 2355 participants assigned to deferred initiation, 1045 initiated ART during follow-up, and 650 started ART with a latest CD4+ cell count at least 350 cells/μl. The predictors for ART initiation differed in the two initiation groups (Appendix Table 2, https://links.lww.com/QAD/A977). Predictors of initiation in the deferred initiation group were younger age, being in a high-income country, higher baseline and latest HIV-RNA, lower latest CD4+ cell count, and shorter time since the last CD4+ cell count or HIV-RNA measurement.
Table 1 shows the ITT and per-protocol risk estimates. The estimated per-protocol 5-year risk (95% CI) was 3.2% (1.9, 4.7) for immediate initiation and 7.0% (5.2, 9.6) for deferred initiation. The 5-year risk difference was −3.8% (–6.7, –1.5) compared with an ITT difference of −3.1% (−5.2, −0.8), corresponding to a difference between the per-protocol and ITT effect estimates of 0.72% (−0.35, 2.35). The per-protocol hazard ratio (95% CI) for immediate versus deferred initiation was 0.34 (0.21, 0.52). This was stronger than the originally reported ITT hazard ratio of 0.43 (0.30, 0.62). The naïve per-protocol analysis included 109 events (37 and 72 in the immediate and deferred group). The total follow-up while participants adhered to the protocol was 6516 (94%) and 5375 (78%) person-years in the immediate and deferred initiation group, respectively. The estimated hazard ratio was 0.41 (0.28, 0.61) with no adjustment for baseline covariates and 0.41 (0.27, 0.61) with adjustment for baseline covariates, similar to the ITT hazard ratio.
The results did not materially change in the sensitivity analyses (Appendix Table 3, https://links.lww.com/QAD/A977). The time-varying means predicted by our models under the observed adherence were similar to the observed means in both initiation groups (Appendix Fig. 1, https://links.lww.com/QAD/A977).
Our per-protocol analysis suggests that the potential benefits of immediate initiation compared with the deferral strategy are about 20% larger than previously suggested by the ITT analysis, although the CIs for our estimates were wide. Compared with deferred initiation until CD4+ cell count dropped below 350 cells/μl, immediate initiation reduced the risk of the composite outcome encompassing serious AIDS, serious non-AIDS, or death events by 2.7% at 3 years and by 3.8% at 5 years. This stronger protective effect of immediate initiation was anticipated because approximately 30% of participants in deferred ART group initiated ART with a CD4+ cell count above 350 cells/μl, whereas protocol deviations were uncommon in the immediate group.
The hazard ratio estimates from the naïve per-protocol analyses (with and without adjustment for baseline variables) were similar to that from the ITT analysis. These estimates underestimated the per-protocol effect of immediate versus deferred initiation compared with our per-protocol analysis, which further adjusted for postrandomization variables via the parametric g-formula [2,7,9]. Our findings show that approaches based on the parametric g-formula can be used for the per-protocol analysis of clinical trials with protocol deviations.
Our analysis has limitations. First, our adjusted per-protocol analysis assumes that all prognostic factors that predict protocol deviations are identified and accurately measured. Although this condition cannot be guaranteed, we adjusted for the main factors used in clinical treatment decisions. Second, all models need to be correctly specified. Again, this condition cannot be guaranteed, but it seems plausible because our models resulted in simulated datasets with distributions of outcome and time-varying covariates similar to those in the original data. Finally, we could not obtain per-protocol effect estimates for each component of the primary endpoint (i.e., serious AIDS events, serious non-AIDS events, and mortality) because the small number of events led to unstable outcome models. Semiparametric methods, such as inverse-probability weighting, may be an alternative to estimate the per-protocol effect for those outcomes.
In conclusion, our estimates of the per-protocol effect of immediate versus deferred ART initiation provide additional support to recent changes in clinical recommendations. Per-protocol effect estimates are especially relevant to patients and clinicians, and can be used by modelers and healthcare planners to estimate an upper bound of the impact of changes in recommendations. Although there are good reasons for ITT analyses to remain the primary analyses of many randomized trials, appropriately adjusted per-protocol analyses help extract additional important information from clinical trial data.
Principal contributions made by the authors: data collection – S.S., J.D.L., A.N.P., B.K.A., A.B., H.K., H.C., M.L., and J.D.N.; statistical analyses – S.L., M.A.H., and R.L.; interpretation of results – all authors; read and approved the manuscript – all authors; revised the work for important intellectual content – all authors; drafted the manuscript – S.L. and M.A.H. S.L., the corresponding author, had full access to all the data in the study and had final responsibility for the decision to submit for publication.
NIH R01 AI102634, PCORI ME-1503-28119, NIH grants UM1-AI068641 and UM1-AI120197, Y1-AI-5072, and NIH R01 AI100654. The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The views expressed are those of the author(s) and do not necessarily reflect the official views of the Uniformed Services University of the Health Sciences, the NIH, the Department of Defense, or the Departments of the Army, Navy, or Air Force.
Conflicts of interest
AP, Consultancy with GSK Biologicals, Speaker fees for Gilead Sciences, Advisory Board for Abbvie. No other conflict of interest to report.
1. Insight Start Study Group. Initiation of antiretroviral therapy in early asymptomatic HIV infection
. N Engl J Med
2. Hernan MA, Hernandez-Diaz S. Beyond the intention-to-treat in comparative effectiveness research
. Clin Trials
3. Department of Health and Human Services (DHHS), Department of Health and Human Services (DHHS). Panel on antiretroviral guidelines for adults and adolescents. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents
4. European AIDS Clinical Society (EACS). European guidelines for treatment of HIV infected adults in Europe
5. World Health Organization (WHO). Guideline on when to start antiretroviral therapy and on preexposure prophylaxis for HIV
. Switzerland: World Health Organization (WHO); 2015.
6. Robins J, Hernan M. Fitzmaurice GD, Davidian M, Verbeke M, Molenberghs G. Estimation of the causal effects of time-varying exposures
. Advances in longitudinal data analysis
. Boca Raton, Florida: Chapman and Hall/CRC Press; 2009. 553–599.
7. Robins JM. A new approach to causal inference in mortality studies with a sustained exposure period: application to the healthy worker survivor effect
. Math Model
8. Young JG, Cain LE, Robins JM, O’Reilly EJ, Hernan MA. Comparative effectiveness of dynamic treatment regimes: an application of the parametric g-formula
. Stat Biosci
9. Murray EJ, Hernan MA. Adherence adjustment in the Coronary Drug Project: a call for better per-protocol effect estimates in randomized trials
. Clin Trials