*Statistical Center for HIV/AIDS Research and Prevention, Fred Hutchinson Cancer Research Center Seattle, WA †Department of Infectious Disease Epidemiology, Imperial College London, UK
B.R.M.: part of this work was supported by the National Institute of Allergy Infectious Diseases of the US National Institutes of Health (5U01AI068615).
To the Editor:
We read with great interest the article by Vickerman et al1 describing the degree to which a microbicide's sexually transmitted infection (STI) efficacy might contribute to the observed HIV effectiveness in a phase 3 microbicide trial. As stated by the authors, the results of their analysis are quantitatively analogous to those found in Desai et al2 where a mathematical modeling approach was also used to address the same question in the context of circumcision effectiveness trial. We would like to highlight several important points that these 2 articles raise.
First, the simulation of randomized control trials, individual based (I-RCTs) or community based (C-RCTs), using transmission dynamics models is a very useful framework not only to help interpret results of completed trials but also to help validate complex trial designs and to inform and clarify important issues relevant for the evaluation and licensing of new products against infectious diseases, including microbicides.
Second, both articles clearly highlight the need to better understand and quantify the mechanisms of protection-or efficacies-against HIV (eg, direct efficacy against HIV through a reduction in HIV susceptibility, indirect protection due to an efficacy against STI) that combine to produce the observed effectiveness in phase 3 microbicide and circumcision trials. This is necessary to provide an assessment of the external validity of the results and to estimate the broader public health impact of these interventions and to guide the licensing process. As suggested in both articles, if a circumcision or microbicide intervention was found to be highly effective against HIV in I-RCT (effectiveness > 50%), then the overall effectiveness was mostly attributable to the direct protection against HIV rather than the indirect protection against STI which suggested a good external validity of trial results to different epidemiological and population contexts. This also implies that an intervention found to be highly effective in I-RCT would also most likely provide a substantial public health impact, and the evidence to license the product would be strong and compelling. However, studies of Vickerman et al1 and Desai et al2 also showed that if the effectiveness of the microbicide or circumcision was less than 50%, the indirect protection against STI could be responsible for a large fraction of infections prevented, highlighting the need to measure STI efficacy independently of HIV efficacy. For a candidate microbicide, this could lead to a potential licensing conundrum: Although a microbicide with an effectiveness of 40% and an HIV efficacy of 40% could be considered a good candidate for licensing, it is unclear whether a microbicide with 40% effectiveness due to 15% HIV efficacy and with a large STI efficacy should be licensed as an HIV prevention intervention. For interventions found to be moderately effective in I-RCT, there are potentially many uncertainties about the broader public health impact of these interventions. I-RCTs do not provide direct estimates of the potential community-level impact of these interventions as several positive and negative indirect longer term effects of the intervention (eg, herd immunity and potential increase in risky sexual behaviors) cannot be derived from relatively short duration I-RCT.
Third, the above issues point to the need for adapting the licensing process to take into account the potential public health impact of new products. Unlike circumcision, a specific candidate microbicide would need to go through the registration/licensure process after positive results from HIV effectiveness microbicide trials before it could be marketed and used as an HIV prevention intervention in a population. The licensure requirements typically do not need to include proof of impact at the public health level. Typically, compelling evidence of effectiveness at the individual level from I-RCT under an intention to treat analysis with a good safety profile is sufficient to warrant licensure. In the context of HIV vaccine, some authors3 have argued that public health impact criteria should be part of the licensing process where they have suggested a framework for public health evaluation of HIV vaccine and proposing a licensing process that takes into account the public health impact of a candidate HIV vaccine. Although this work was developed in the context of HIV vaccine, it is also important to better quantify the effect of a candidate microbicide if regulators are to evaluate adequately the usefulness of new candidates, which necessarily depends on the potential community-level impact of a microbicide across different settings, populations, and conditions of use.
The proposed framework for candidate HIV vaccines by Boily et al3 can be adapted to the microbicide context. Briefly, successful conduct of microbicide I-RCT would be followed by an assessment of the potential population-level impact using mathematical models. Given that (predefined) public health criteria are met, I-RCT would be followed by the assessment of public health impact via the initiation of C-RCTs. New products are approved for licensure based on their “good” benefit-to-risk ratio not solely on their benefits. Microbicides are aimed at healthy women such that a candidate microbicide would need to demonstrate higher benefit-to-risk ratio than treatments aimed at treating sick individuals. Microbicides have potential risks, for instance, second-generation microbicides contain antiretrovirals (ARVs) such that the risk of introducing ARV resistance in a population is of potential concern. I-RCTs typically provide an “asymmetric” look at benefit-to-risk ratio because more resource and attention are devoted to the estimation of benefit than to harm, as it was pointed out in a recent article by O'Neill4. Mathematical models can certainly help rebalance this asymmetry in the evaluation of public health benefits and risks. For instance, little or no information about ARV resistance can be gathered within microbicide I-RCT, but mathematical models combined with uncertainty analysis can help in understanding the potential harm at the community level.
Fourth, the interpretation of results from mathematical models is somewhat subjective such that public health licensing guidelines should be predefined in a unified international manner (ie, where all or most of the interested parties are in agreement). The minimum public health criteria needed for licensure, the minimum public health criteria needed to undertake C-RCT, the minimum effectiveness of the microbicide to achieve in I-RCT, and the minimum total effect of microbicide at the population level are among some of the important benchmarks to be determined. This predetermination process is analogous to the process of predefining the minimum meaningful clinical difference before the conduct of an I-RCT, where this is mainly done to remove the potential subjectivity in interpreting the trial final results. Therefore, it is crucial that public health criteria be defined a priori but, unfortunately, little work has been done toward achieving this goal in the context of microbicides and indeed in the testing of most infectious disease prevention approaches.
Overall, the mathematical modeling approach proposed by Vickerman et al1 is a welcome step toward a better quantification of the microbicide effect on HIV, but we argue that this modeling should be integrated into a much more extensive evaluation process analogous to the one proposed for HIV vaccines which would include an exhaustive mathematical modeling evaluation not only of the benefit (ie, effectiveness/effect) at the community level but also of the potential risks as well, where predefined public health impact criteria would need to be met before licensure.
Benoît R. Mâsse, PhD*
Marie-Claude Boily, PhD†
Kamal Desai, PhD†
*Statistical Center for HIV/AIDS
Research and Prevention,
Fred Hutchinson Cancer Research Center
†Department of Infectious Disease
Epidemiology, Imperial College
1. Vickerman P, Foss A, Watts C. Using modeling to explore the degree to which a microbicide's sexually transmitted infection efficacy may contribute to the HIV effectiveness measures in phase 3 microbicide trials. J Acquir Immune Defic Syndr
2. Desai K, Boily MC, Garnett GP, et al. The role of sexually transmitted infections in male circumcision effectiveness against HIV-insights from clinical trial simulation. Emerg Themes Epidemiol
3. Boily MC, Abu-Raddad L, Desai K, et al. Measuring the public health impact of candidate HIV vaccines as part of the licensing process. Lancet Infect Dis
4. O'Neill R. A perspective on characterizing benefits and risks derived from clinical trials: can we do more? Drug Inf J