Skip Navigation LinksHome > July 1, 2013 - Volume 63 - Issue > Study Design Considerations for Evaluating Efficacy of Syste...
JAIDS Journal of Acquired Immune Deficiency Syndromes:
doi: 10.1097/QAI.0b013e3182986fac
Supplement Article

Study Design Considerations for Evaluating Efficacy of Systemic Preexposure Prophylaxis Interventions

Donnell, Deborah PhD*; Hughes, James P. PhD; Wang, Lei PhD*; Chen, Ying Q. PhD*; Fleming, Thomas R. PhD

Free Access
Article Outline
Collapse Box

Author Information

*Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA; and

Department of Biostatistics, University of Washington, Seattle, WA.

Correspondence to: Deborah Donnell, PhD, Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue N, M2-A300, Seattle, WA 98109 (e-mail:

The authors have no conflicts of interest to disclose.

Supported by the HIV Prevention Trials Network, sponsored by the National Institute of Allergy and Infectious Diseases, the National Institute of Mental Health, and the National Institute of Drug Abuse, Office of AIDS Research, of the National Institutes of Health, Department of Health and Human Services, grant UM1-AI068617 (D.D.).

Collapse Box


Background: The development of interventions for systemic pre-exposure prophylaxis (PrEP) faces several significant challenges following the US Food and Drug Administration's approval of emtricitabine/tenofovir (FTC/TDF) for HIV prevention. This development is particularly complex because of inconsistency of efficacy results of FTC/TDF PrEP trials for HIV prevention.

Methods: Possible designs for a PrEP phase 3 efficacy trial are obtained by considering scenarios for potential experimental PrEP and control regimens, including consideration of placebo and active controls, longer acting PrEP and alternate dosing schedules.

Results: Noninferiority (NI) trials with hazard ratio NI margins ranging from 1.10 to 1.25 can be justified in the contexts of the 3 PrEP trials demonstrating efficacy of FTC/TDF. However, these HIV endpoint trials may require extremely large number of participants, particularly in settings where FTC/TDF has been shown to reduce the risk of HIV acquisition. NI trials also are often difficult to interpret because they depend on previous placebo-controlled efficacy results. Superiority trials for PrEP are plausible in settings where FTC/TDF efficacy is not yet established, possibly due to low adherence (ie, women at risk as in FemPrEP and VOICE): a new product with potential for higher adherence and potency would be a promising candidate in this setting.

Conclusions: Following Food and Drug Administration's approval of FTC/TDF for PrEP, trials to establish efficacy of new PrEP regimens require stringent design standards, together with rigorous debate about adherence within study populations and many important ethical issues.

Back to Top | Article Outline


The US Food and Drug Administration approved daily dosing of emtricidabine/tenofovir (FTC/TDF) for prevention of HIV acquisition in July 2012. Although this provides an important advance in HIV prevention, there is need for more than 1 option for preexposure prophylaxis (PrEP). In settings where daily FTC/TDF results have been disappointing, there is particular interest in new products that could improve prevention efficacy through improved adherence or potency. Furthermore, FTC/TDF is currently a first-line choice for treatment of HIV infection, raising concerns about potential for resistance and competing resource demands for a drug used for both treatment and prevention.

The development of new interventions for PrEP faces several significant challenges. A study evaluating the effectiveness of a PrEP intervention requires HIV seroincidence endpoints because we do not have evidence to justify a biomarker as a replacement endpoint. To use a biomarker endpoint, we would need to establish that the effect of PrEP on the biomarker reliably predicts whether PrEP has a clinically meaningful effect on the risk of HIV acquisition. Trials with HIV endpoints may be large. This is particularly likely in settings where FTC/TDF has been shown to reduce the risk of HIV acquisition and then would be used as the standard of care (SOC) in the trial of the new intervention because the overall seroincidence rates in the trial could be below 2 per 100 person-years. Even larger sample sizes may be required if we conduct noninferiority (NI) trials, that is, trials designed to determine whether we can rule out that use of a new intervention rather than FTC/TDF would cause an unacceptable increase in the risk of HIV acquisition. The inconsistency of efficacy results for FTC/TDF, across recent PrEP trials in Table 1,1–5 makes NI trials with FTC/TDF as the active control regimen problematic because an NI trial is appropriate only when the effect of the active control in the NI trial setting is (1) large and (2) precisely estimated from previous randomized trials conducted in similar settings. In settings where FTC/TDF has been shown to provide little, if any, reduction in the risk of HIV acquisition, superiority designs could be justified for drugs that could plausibly achieve higher adherence or potency.

Table 1
Table 1
Image Tools

In this manuscript, we will explore potential study designs for randomized efficacy trials evaluating experimental PrEP regimens for prevention of HIV acquisition.

Back to Top | Article Outline


Designs for PrEP phase 3 efficacy trials are obtained by considering potential experimental and control PrEP regimens.

1. Potential experimental PrEP regimens:

a. Oral daily drug(s) other than FTC/TDF.

b. A long-acting formulation (oral or injectable) of an experimental agent(s).

c. Oral FTC/TDF with a different dosing strategy (eg, event-based dosing; event-based dosing + weekly dosing).

2. Potential control regimens:

a. Active control: Daily FTC/TDF is provided as the control; daily FTC/TDF is assumed to be a SOC regimen.

b. Add-on placebo control: Placebo is the control; participants in both arms of the trial have access to background use of daily FTC/TDF with level of use matched to the clinical setting of interest.

c. Placebo control: Placebo is the control, where there is justification that background management does not include use of daily FTC/TDF.

The potential scenarios for a PrEP phase 3 efficacy trial are obtained by combining the choice of experimental and control PrEP regimens (Table 2).

Table 2
Table 2
Image Tools
Back to Top | Article Outline


We discuss the 8 scenarios in Table 2, considering strengths and weaknesses of superiority and NI designs, as well as anticipated level of interest or priority.

Back to Top | Article Outline
Scenario A: Experimental Oral Drug Versus Daily FTC/TDF

In this scenario, daily FTC/TDF is assumed to be SOC for PrEP, most justified where adherence rates and levels of efficacy for that regimen are high, as in the Partners PrEP trial setting of discordant heterosexual couples in committed relationships.1 If an experimental oral drug is of interest, even though its efficacy might be only similar to or modestly better than that of FTC/TDF, it could be evaluated in a FTC/TDF-controlled NI trial. By design, the intention of such an NI trial is to determine whether we can rule out that the efficacy of the experimental oral drug is “unacceptably worse” than daily FTC/TDF in preventing HIV acquisition. The NI trial requires specification of a threshold, called the “NI margin,” for what constitutes unacceptable loss of efficacy. NI is established if the estimated relative effect of the experimental drug to FTC/TDF is sufficiently favorable and precise that the confidence interval for that relative efficacy does not include the NI margin. High adherence to a daily FTC/TDF active control regimen is necessary in an NI trial because justification for the NI margin includes an assumption that the historical randomized trials establishing efficacy of FTC/TDF (where adherence was necessarily high enough to prevent a substantial proportion of infections) provide an unbiased estimate of FTC/TDF's effect (relative to a hypothetical placebo) in the NI trial.6–9

In NI trials in Scenario A, participants randomized to the experimental arm receive the new oral agent instead of the FTC/TDF control, even though there is substantive evidence establishing FTC/TDF meaningfully reduces the risk of HIV acquisition. For such a trial to be ethical, there needs to be strong previous proof-of-concept for efficacy of the new oral agent.9

The sample size of an NI trial can be very large (Table 3). Suppose we decide (1) it is unacceptable for the relative rate of HIV acquisition on the new oral agent to be 1.25-fold higher than on FTC/TDF [ie, the NI margin is selected to be a hazard ratio (HR) of 1.25] and (2) the probability of a false-positive conclusion should be only 2.5% when the HR truly is 1.25. In this setting, if we want to rule out the prespecified NI margin with 90% probability when the experimental drug and FTC/TDF truly have the same effect on HIV acquisition, the sample size and duration of follow-up in the NI trial need to be sufficiently large that 844 trial participants have HIV acquisition events (Table 3). This would be reduced to 211 HIV acquisition events if we believe that the new oral agent is somewhat better than daily FTC/TDF and require 90% probability to rule out the NI margin when the experimental drug to daily FTC/TDF HR is 0.80 (Table 3).

Table 3
Table 3
Image Tools

Scenario A is of interest when adherence rates and levels of efficacy of FTC/TDF are high and an experimental oral drug has strongly favorable evidence from proof-of-concept trials. There is interest in this setting in part because a new oral agent could preserve daily FTC/TDF for first-line HIV treatment and reduce the risk of emerging resistance. However, given the above considerations of ethics and feasibility, Scenario A may be a low priority unless there is favorable evidence for the new oral agent from, for example, a previously completed superiority trial (eg, Scenario D or F.)

Back to Top | Article Outline
Scenario B: Long-Acting Formulation (Injectable or Oral) of Experimental Agent(s) Versus Daily FTC/TDF

In Scenario B, as in Scenario A, daily oral FTC/TDF is assumed as an effective SOC regimen for PrEP. Because participants receiving the experimental long-acting agent are not receiving daily FTC/TDF in a randomized head-to-head comparison trial, the ethics and level of priority of this scenario again is dependent on the strength of evidence from proof-of-concept trials of the long-acting experimental agent.

Even where the daily FTC/TDF control regimen has been proven effective, it might be possible to conduct a superiority trial for a new PrEP drug because exploratory analyses in the Partners PrEP and iPrEx trials suggested that 85% to 90% efficacy might be achieved with higher adherence. To illustrate sample size requirements, a trial with 88 patients having HIV acquisition events would be required to rule out equal effectiveness of experimental and FTC/TDF regimens with 90% “true-positive” probability, when the experimental regimen truly reduces relative risk of HIV infection by 50% (assuming a 2.5% “false-positive” probability if regimens are truly equally effective). If this trial was conducted in settings where HIV acquisition would be approximately 2% per year, then approximately 2250 participants would need to be enrolled, followed up for an average of 2 years.

Alternatively, an NI trial design might be preferred. Suppose we derive the NI margin using the Partners PrEP or the Botswana TDF2 trial to estimate the effect of FTC/TDF (vs. hypothetical placebo) in the NI trial, which assumes adherence levels in the NI trial will be as high as was seen in those trials. Using the “95–95” approach for derivation of the NI margin7,8 and the Partners PrEP result to estimate that FTC/TDF provides a 75% (95% CI: 55 to 87) reduction in the risk of HIV acquisition, and requiring the experimental regimen to preserve at least 75% of FTC/TDF's effect, the NI margin is an experimental to FTC/TDF HR of 1.25. In such a setting, a trial with 108 patients having HIV acquisition events will have 90% power to rule out that the experimental regimen to daily FTC/TDF HR is 1.25 when the experimental regimen truly provides a 33% reduction in the risk of HIV acquisition relative to FTC/TDF [requiring only 2.5% probability of a false-positive conclusion when the HR truly is 1.25 (Table 3)]. Applying the same “95–95” approach in the men who have sex with men setting and using the iPrEX trial to estimate FTC/TDF provides a 42% (18%–60%) reduction in the risk of HIV acquisition; even if the margin ensures preservation of only 50% of FTC/TDF's effect, only a modest 1.10 NI margin for experimental to daily FTC/TDF HR would be justified. In this setting, approximately 171 HIV acquisition events would be required (Table 3).

Suppose, however, that adherence to daily FTC/TDF is not expected to be high. If FTC/TDF still is used as an active control, the experimental regimen needs to be established to be superior to conclude it is effective. This is plausible if the experimental regimen is a long-acting drug. If an experimental regimen is shown to be superior to daily FTC/TDF active control when adherence to FTC/TDF is not high, trial results may not generalize to settings with high adherence to FTC/TDF.

Back to Top | Article Outline
Scenario C: Alternative Dosing of Oral FTC/TDF Versus Daily FTC/TDF

A trial to assess potential superiority of an alternate dosing strategy for FTC/TDF, compared with daily, could be justified if it were thought that the new dosing might substantially reduce the risk of HIV acquisition, perhaps by achieving substantially higher adherence. However, if alternate dosing for FTC/TDF became an established standard regimen in real world settings (eg, it became common practice to use FTC/TDF shortly before and after risky sex) or if alternate dosing would be expected to reduce the risk of serious toxicities, conducting a trial to verify that the alternate dosing was not inferior to the daily regimen could be justified. However, since Scenario C does not achieve replacement of FTC/TDF for PrEP, it likely would not be a priority to conduct a large-scale clinical trial simply to establish that an alternate dosing strategy would be noninferior to daily dosing, keeping in mind that establishing NI does not mean the alternative dosing strategy is “at least as good as” daily FTC/TDF but rather that it is “not unacceptably worse than” daily FTC/TDF.7,8

Back to Top | Article Outline
Scenarios D and E: New Oral Drug, or Long-Acting Experimental Drug (Injectable or Oral) Versus Placebo, Where Trial Participants Have Access to Background Use of Daily FTC/TDF

Daily FTC/TDF is available for use in the clinical setting addressed by these scenarios. However, we want to improve the level of protection against HIV acquisition by developing a long-acting drug or a new drug with mechanisms of action complementary to FTC/TDF. This approach is particularly appealing in settings where adherence to FTC/TDF was not high, such as in iPrEx, and yet relatively high levels of adherence to the new intervention are expected. In this setting, superiority trials would be conducted, with careful assessment of drug–drug interactions. Such trials could be of moderate to high priority, especially Scenario E, because a drug with a long-acting formulation should have a favorable adherence profile.

An important decision is whether daily oral FTC/TDF should be supplied by the trial in Scenarios D and E. The following considerations should guide this decision. First, to enhance the clinical relevance and interpretability of results, the use of FTC/TDF as a background intervention should match that in the clinical setting of interest. Second, by the principle of “distributive justice,”9 the trial should provide results that are relevant and informative for the population in which the trial is conducted. By not supplying daily FTC/TDF to trial participants, efficacy and safety results from the trial become directly relevant to the context of FTC/TDF as it is currently delivered. However, it is appropriate to supply daily FTC/TDF to all trial participants if it is expected that the use of FTC/TDF in clinical practice will evolve so that the level of use in the trial matches what will become standard use of FTC/TDF after the trial is completed.

Back to Top | Article Outline
Scenarios F and G: New Oral Drug, or Long-Acting Experimental Drug (Injectable or Oral) Versus Placebo, Where Background Management Properly Does Not Include Daily FTC/TDF

In settings where patients are not willing or able to adhere to daily FTC/TDF, or where communities are not willing or able to provide or support daily FTC/TDF for prevention, finding safe and effective PrEP regimens that are affordable and feasible is a moderate to high priority. Lack of community willingness to support daily FTC/TDF for PrEP could reflect a desire to reserve this regimen for first-line HIV treatment.

To satisfy the unmet needs in these settings, a new drug has to overcome the barriers encountered by daily FTC/TDF. Hence, drugs with long-acting formulations are of greater interest (Scenario G) than alternative daily oral products (Scenario F). Either scenario requires a superiority trial, with a potential need to establish super-superiority by statistically ruling out modest levels of efficacy, such as a 30% reduction in the risk of HIV acquisition.

Of note, it is ethical to conduct placebo-controlled trials of new PrEP agents only in populations where it is anticipated that the new agent, if effective, would become available for use. In particular, it is not acceptable to evaluate a new agent in a population if the reasons that preclude availability of FTC/TDF would also apply to the new agent.

Back to Top | Article Outline
Scenario H: Alternative Dosing Strategy for FTC/TDF Versus Placebo, Where Background Management Properly Does Not Include Daily FTC/TDF

For this scenario to arise, an alternative dosing strategy for FTC/TDF would be viewed as promising, even though daily FTC/TDF regimen is judged to be unaffordable, impractical, or not effective. For illustration, from a closely related field: clinical trials are ongoing to evaluate effectiveness of coital use of a tenofovir gel microbicide in women,10 even though daily use of the same product was demonstrated to be ineffective.5,11,12

To justify a trial of alternative dosing strategies for FTC/TDF, we would need data establishing proof-of-concept that alternate dosing addresses the deficiencies of daily FTC/TDF. The level of priority of this superiority trial would depend on the strength of those previous data.

Back to Top | Article Outline


An important goal in HIV prevention research is the identification of safe and effective PrEP regimens that are affordable and feasible and do not interfere with the effects of drugs integral to HIV treatment. With initial progress that has resulted in the availability of FTC/TDF for PrEP, it is likely that subsequent trials will require larger sample sizes and longer duration, whether we pursue settings where FTC/TDF has been shown to reduce the risk of HIV acquisition to relatively low levels or settings where FTC/TDF has failed to provide clinically meaningful protection.

NI trials provide a potentially appealing study design in settings where FTC/TDF has been established to be effective, yet there is still interest in alternative PrEP interventions that are better tolerated, more convenient, less costly, or not involving drugs commonly used for HIV treatment, even if these alternative interventions would have efficacy that is only similar to or slightly better than that of FTC/TDF. Unfortunately, NI trials often require large sample sizes and often yield results that are difficult to interpret due to uncertainty about the validity of their strong underlying assumptions. One key assumption is that historical trials can provide an unbiased estimate of the active control's effect for the NI trial. When experimental and FTC/TDF regimens demonstrate similar efficacy in an NI trial, if we cannot rely on the validity of that key assumption, how can we distinguish whether the regimens are similarly effective or similarly ineffective? For illustration, suppose the FemPrEP trial conducted in heterosexual women in South Africa and Kenya4 had been a comparison between an experimental PrEP drug that truly provided no benefit, versus FTC/TDF as the active control. Since daily FTC/TDF provided essentially no benefit in that setting (Table 1), the regimens would have had similar rates of HIV acquisition. If the placebo-controlled evaluation of FTC/TDF provided by the Centers for Disease Control and Prevention's TDF2 PrEP trial conducted in heterosexual men and women in Botswana3 had been used as the historical trial providing an estimate of the effect of FTC/TDF in the setting of this hypothetical FemPrEP trial (Table 1), we would have concluded that this experimental drug was very beneficial even though it was, in reality, ineffective.

The influence of level of adherence will be an important consideration in the design and interpretation of NI trials of PrEP regimens. A requirement of NI trials is that the active comparator regimen has strong efficacy that is precisely estimated by historical trials conducted in settings similar to the setting of the NI trial. As indicated by the evidence from Table 1, the level of adherence has strong influence on efficacy for PrEP regimens, such as FTC/TDF. Hence, it is likely that NI trials of PrEP regimens will be appropriate only when (1) the active comparator (ie, FTC/TDF) is a potent agent, (2) the active comparator's efficacy was established in a setting of high adherence, and (3) there is reliable evidence that the level of adherence to the active comparator will be equally high in the setting of the NI trial. It is problematic that it is unknown why the participants in Partners PrEP and TDF2 were highly adherent to FTC/TDF while the women enrolled in FemPrEP and VOICE were not. The reasons for these differences must be better understood to be able to justify the NI trial assumption that an unbiased estimate of efficacy of the active comparator in the NI trial can be obtained from data in the historical trials. Without that justification, NI trials will not be interpretable.

Superiority trials provide a clearer path to obtaining reliable evidence of clinically meaningful advances. There are important populations at demonstrated high risk of HIV acquisition where daily FTC/TDF has been found to be inadequately effective. A new product with high potential to achieve protection, either because adherence requirements were less onerous or the drug was more potent, may be the highest priority for moving into efficacy evaluation. Conducting such a superiority trial against the evolving background of PrEP SOC would seem to be addressing a particularly relevant public health question.

Back to Top | Article Outline


1. Baeten JM, Donnell D, Ndase P, et al.. Antiretroviral prophylaxis for HIV prevention in heterosexual men and women. N Engl J Med. 2012;367:399–410.

2. Grant RM, Lama JR, Anderson PL, et al.. Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. N Engl J Med. 2010;363:2587–2599.

3. Thigpen MC, Kebaabetswe PM, Paxton LA, et al.. Antiretroviral preexposure prophylaxis for heterosexual HIV transmission in Botswana. N Engl J Med. 2012;367:423–434.

4. Van Damme L, Corneli A, Ahmed K, et al.. Preexposure prophylaxis for HIV infection among African women. N Engl J Med. 2012;367:411–422.

5. Marrazzo J, Ramjee G, Nair G, et al.. Pre-exposure prophylaxis for HIV in women: daily oral tenofovir, oral tenofovir/emtricitabine, or vaginal tenofovir gel in the VOICE study (MTN 003). Paper presented at: 20th Conference on Retroviruses and Opportunistics Infections; March 3-6 2013; Atlanta, GA.

6. Food and Drug Administration, US Department of Health and Human Services. Draft guidance for industry non-inferiority clinical trials. 2010. Available at:

7. Fleming TR. Current issues in non-inferiority trials. Stat Med. 2008;27:317–332.

8. Fleming TR, Odem-Davis K, Rothmann MD, et al.. Some essential considerations in the design and conduct of non-inferiority trials. Clin Trials. 2011;8:432–439.

9. Beauchamp TL, Childress JF. Principles of Biomedical Ethics. 5th ed. New York, NY: Oxford University Press; 2001.

10. Abdool Karim Q, Abdool Karim SS, Frohlich JA, et al.. Effectiveness and safety of tenofovir gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science. 2010;329:1168–1174.

11. National Institute of Allergy and Infectious Diseases (NIAID). NIH discontinues tenofovir vaginal gel in ‘Voice’ HIV prevention study. 2011. Available at:

12. National Institute of Allergy and Infectious Diseases (NIAID). NIH modifies “VOICE” HIV prevention study in women. 2011. Available at:


preexposure prophylaxis; noninferiority; efficacy; phase 3; clinical trials; adherence

© 2013 Lippincott Williams & Wilkins, Inc.


Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.