Secondary Logo

Journal Logo


Design challenges facing clinical trials of the effectiveness of new HIV-prevention technologies

Grobler, Anneke C.a; Abdool Karim, Salim S.a,b

Author Information
doi: 10.1097/QAD.0b013e3283509a29


Several randomized placebo-controlled trials of microbicides or antiretroviral preexposure prophylaxis (PrEP) have been conducted to assess their effectiveness in preventing sexual transmission of HIV infection. Prior to 2010, none of these trials had demonstrated effectiveness in preventing HIV [1]. In July 2010, a microbicide gel containing the antiretroviral drug tenofovir (CAPRISA 004) demonstrated 39% reduction in HIV acquisition in women [2] and in November 2010, oral emtricitabine and tenofovir disoproxil fumarate (FTC–TDF) showed 44% reduction in HIV acquisition (iPrEx) in MSM [3]. Whereas oral FTC–TDF (FEM–PrEP) [4] and oral TDF (VOICE) [5] were not found to be effective in heterosexual women, two trials showed in July 2011 that oral TDF and FTC–TDF were found to reduce HIV transmission by 62 and 73% in serodiscordant couples (Partners PrEP trial) [6] and oral FTC–TDF reduced HIV transmission by 63% in heterosexual men and women (TDF2) [7]. Furthermore, early antiretroviral treatment initiation in the HIV-positive partner (HPTN052) was shown to reduce HIV acquisition from the positive partner in the HIV-negative partner in serodiscordant couples by 96% [8]. These recent successes have raised questions about the design of future HIV-prevention trials and whether further placebo-controlled trials of some new HIV-prevention technologies are ethically justifiable [9].

As long as evidence about the efficacy of PrEP in preventing HIV infection is ambiguous, future placebo-controlled trials are required. Currently estimates of the efficacy of daily oral FTC–TDF in women ranges from no efficacy [4] to 73% [6]. However, there is hope that accumulating evidence may demonstrate a more consistent estimate of PrEP effectiveness in the near future. In this scenario, future placebo-controlled trials may no longer be possible.

One option for future trials which are designed to assess the efficacy of new HIV-prevention strategies is to include these efficacious antiretroviral intervention(s) in both intervention and control arms. In this case, the net effect is to lower the overall HIV incidence rate in the trial, thereby making the trial larger and/or longer. This could add substantially to the cost of the trial. In some instances, it may not be safe, feasible or practical to implement a known efficacious antiretroviral intervention simultaneously in a trial as it may interfere with the new study intervention under investigation. For example, the use of a vaginal ring containing an antiretroviral may be a contra-indication for the simultaneous use with a gel containing the same or a different antiretroviral. In such cases, another option is to include efficacious antiretroviral intervention(s) only in the comparator group.

A trial with active agent(s) in the comparator group can be designed either as a superiority or noninferiority trial. In a superiority trial, the hypothesis tested is that the new intervention under investigation is better, by a clinically relevant amount, than the comparator, which can be either standard of care, placebo or another active intervention. This can be done either as a blinded or open-label trial. The new intervention needs to be even more potent than an active comparator in order to show superiority, that is higher efficacy.

In a noninferiority trial, the hypothesis being tested is that the intervention under investigation is not inferior to, by a predefined clinically relevant amount, or at least as effective as, the comparator [10]. Noninferiority trials are used when the new intervention is assumed to have some benefits (e.g. improved safety, low cost, etc.) over the comparator intervention, but has similar efficacy [11]. Noninferiority trials, however, are not without challenges in data interpretation [12].

The first challenge is that any intervention can easily be shown to be noninferior to a noneffective comparator intervention; whether or not either intervention is superior to placebo or no intervention [10].

The assumption made when an active comparator is used in a noninferiority trial is that the efficacy found in another trial also applies to the population and environment in the current trial; an assumption that cannot, in most instances be readily tested. How similar do two populations need to be before findings in one population can be used to determine the active comparator in another population? For example, can we assume that oral FTC–TDF will be 44% effective in a population of heterosexual women, because it was found to be 44% effective in a population of MSM? [3] The validity of this assumption was challenged by the findings of a trial testing FTC–TDF in a population of heterosexual women (FEM–PrEP) wherein no protection against HIV was shown [4], whereas 73% effectiveness was found in men and women in serodiscordant relationships (Partners PrEP trial) [6].

Consider a hypothetical new PrEP trial testing coital use of oral tenofovir against daily use of oral tenofovir which concluded equivalence; that is concluded that coital use of oral tenofovir is not inferior to daily oral tenofovir. If we drew this conclusion in August 2011, before the recent findings of the VOICE study [5], we would have concluded that coital tenofovir is 62% effective, based on the effectiveness of daily oral tenofovir in the Partners PrEP trial [6]. If, however, we drew this conclusion 3 months later in October 2011, after the recent announcement from the VOICE study, we could conclude that coital tenofovir is not effective. Without a clear and consistent protective efficacy estimate for daily oral tenofovir, it is difficult to provide a meaningful interpretation of a noninferiority trial assessing coital use of oral tenofovir.

The second challenge is that any effect that dilutes the true efficacy of an intervention in a trial such as nonadherence to the study regimen, considerable loss to follow-up or protocol violations, makes it easier for the two interventions to be declared equivalent. Nonadherence is a particular concern in these settings because it is difficult to reliably measure adherence levels in both arms of a PrEP trial. Drug level monitoring is only a partial solution, as this can only be done in the active arm. A placebo with a biological marker of adherence could address this. A poorly conducted trial will be more likely to lead to the false conclusion that the test intervention is noninferior to the active comparator than a well conducted trial [10].

The standard approach to dealing with this problem is to base the primary analysis of a noninferiority trial on the per protocol population and not on the intent-to-treat population, as is standard with a superiority trial. This is because the intent-to-treat analysis is biased toward equality in conditions wherein many participants did not follow study procedures. One of the most important criteria for inclusion in the per protocol analysis is adherence to the study regimen. In the CAPRISA 004 [2], iPrEx [3], Partners PrEP [6] and FEM-PrEP trials [4], self-reported adherence and adherence based on applicator or pill count was high. In the iPrEx trial, there was a poor correlation between reported adherence and drug levels detected. This casts doubts on both the adherence during this trial and on the validity of self-report. If adherence is not accurately measured, an analysis based on adherence or on the per protocol population determined by adherence may lead to incorrect conclusions. More reliable measures of adherence such as drug concentrations in the active arm could go some way toward increasing the likelihood of high adherers being included in the active arm of the per protocol analysis. A biological adherence marker will be needed for the placebo arm for a valid comparison.

The importance of adherence in noninferiority trials should not be underestimated. Suboptimal adherence in a trial with an active comparator has the net effect of making the result ‘flatter’ but does not lead to an erroneous conclusion of noninferiority, provided adherence levels are similar in each of the study arms (Table 1). The exception occurs when adherence is zero or very close to zero in each of the treatment arms; in this case, noninferiority is invariably declared.

Table 1
Table 1:
Projected effectiveness found with different levels of true efficacy of the comparator and new intervention in combination with different adherence levels.

Differential adherence in the study arms is much more complex and, under a range of scenarios, can lead to incorrect conclusions (Table 1). If the true effectiveness of both interventions is the same, the intervention with higher adherence is favored in a noninferiority study. If an inferior, but still efficacious, intervention had higher adherence than the more effective comparator, a conclusion of noninferiority would be made.

Adherence to an intervention is an important determinant of whether a prevention strategy would have a public health benefit; therefore, the adherence to an intervention is an important aspect of the determination of its efficacy. However, if adherence to the comparator is low, one might in effect be comparing a new intervention to a placebo-like effect created by lack of adherence to the intervention, even though this comparator was intended to be an active intervention.

Differential adherence is unlikely in a double-blinded trial of similar interventions wherein neither the study participants nor the investigators are aware of study assignment. However, differential adherence is much more likely when the comparator and the new intervention are substantially different and blinding is less likely. More specifically, if one intervention is an oral formulation and the other intervention is topical, different patterns of use might be likely and adherence might differ substantially. The same holds for different dosing strategies; for example, when once daily dosing is compared with coitally-dependent dosing. Differential adherence is also likely when the interventions have different side-effect profiles. Although no evidence of differential adherence was found in the CAPRISA 004 trial [2], differential adherence was reported in the iPrEx trial in some of the early visits, with lower adherence in the active arm [3], probably due to drug side effects.

Extreme caution should be used when noninferiority studies are planned comparing different dosing strategies, which are likely to lead to different adherence levels. It might not be possible to compare formulations that are very different, such as oral, ring and gel formulations, as they are likely to be used differentially.

The third challenge with noninferiority trials is their large size, much larger than superiority trials [12]. The PrEP trials which have recently announced results were all designed as superiority trials with sample sizes between 1000 and 5000 participants. The topical PrEP trial targeted 92 HIV infections [2], whereas the oral PrEP trial targeted 85 infections. In contrast, a noninferiority trial with 80% power to show an intervention not more than 20% inferior to the active comparator would require about 500 HIV infections. A noninferiority limit of 20% is probably too large if the active comparator is only 40% effective, but small enough if the active comparator is 60–70% effective.


In future efficacy trials of new HIV-prevention interventions, noninferiority designs may become one of the standard approaches. These designs are harder to interpret and care should be taken to ensure that comparison treatments are well understood.

In these study designs, adherence is a critical factor, as it may lead to spurious results. Differential adherence in the treatment arms could lead to incorrect findings about the true effectiveness of the interventions. Investigators conducting noninferiority trials will, of necessity, have to pay special attention to supporting, measuring and maintaining high adherence. The effect of differential adherence between study arms should be considered when interpreting the results.


A.G. and S.S.A.K. conceptualized this work. A.G. wrote the manuscript and S.S.A.K. provided comments and edits.

A.G. received training support from the Columbia University-Southern African Fogarty AIDS International Training and Research Programme (AITRP), funded by the Fogarty International Center (grant D43TW00231).

The CAPRISA 004 Tenofovir gel trial was supported by the United States Agency for International Development (USAID), FHI (cooperative agreement GPO-A-00-05-00022-00 and contract 132119), CONRAD and LIFElab.

Conflicts of interest

There are no conflicts of interest.


1. Padian NS, McCoy SI, Balkus JE, Wasserheit JN. Weighing the gold in the gold standard: challenges in HIV prevention research. AIDS 2010; 24:621–635. doi: 10.1097/QAD.0b013e328337798a.
2. Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, Mansoor LE, et al. Effectiveness and safety of tenofovir gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 2010; 329:1168–1174.
3. Grant RM, Lama JR, Anderson PL, McMahan V, Liu AY, Vargas L, et al.Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med 2010; 363:2587–2599.
4. FHI. FHI to Initiate Orderly Closure of FEM-PrEP. 2011 [Accessed June 2011].
5. National Institute of Allergy and Infectious Diseases. Statement: NIH Modifies ‘VOICE’ HIV Prevention Study in Women Oral Tenofovir Discontinued in Clinical Trial. 2011 [Accessed October 2011].
6. University of Washington International Clinical Research Center. Press release: Partners PrEP Study. Pivotal study finds that HIV medications are highly effective as prophyalxis against HIV infection in men and women in Africa. 2011 [Accessed August 2011].
7. Centers for Diseases Control and Prevention. Press release: CDC trial and another major study find PrEP can reduce risk of HIV infection among heterosexuals. 2011 [Accessed July 2011].
8. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al.Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med 2011; 365:493–505.
9. Kuhn L, Susser I, Stein Z. Can further placebo-controlled trials of antiretroviral drugs to prevent sexual transmission of HIV be justified?. Lancet 2011; 378:285–287.
10. Schall R, Luus H, Erasmus T. Type of comparison. In: Karlberg J, Tsang K, editors. Introduction to clinical trials. Hong Kong: The Clinical Trials Centre; 1998.
11. Lagakos S, Gable A. Methodological challenges in biomedical HIV prevention trials. Washington, DC: Institute of Medicine of the National Academies; 2008.
12. Rothman M, Li N, Chen G, Chi G, Temple R, Tsou H. Design and analysis of noninferiority mortality trials in oncology. Stat Med 2003; 22:239–264.

adherence; HIV prevention; noninferiority trials; placebo-controlled trials; preexposure prophylaxis

© 2012 Lippincott Williams & Wilkins, Inc.