Sieve analysis in HIV-1 vaccine efficacy trials

Edlefsen, Paul T.; Gilbert, Peter B.; Rolland, Morgane

doi: 10.1097/COH.0b013e328362db2b
CHANGING ENVIRONMENT IN HIV VACCINE: Edited by Nelson L. Michael and Glenda Gray

Purpose of review: The genetic characterization of HIV-1 breakthrough infections in vaccine and placebo recipients offers new ways to assess vaccine efficacy trials. Statistical and sequence analysis methods provide opportunities to mine the mechanisms behind the effect of an HIV vaccine.

Recent findings: The release of results from two HIV-1 vaccine efficacy trials, Step/HVTN-502 (HIV Vaccine Trials Network-502) and RV144, led to numerous studies in the last 5 years, including efforts to sequence HIV-1 breakthrough infections and compare viral characteristics between the vaccine and placebo groups. Novel genetic and statistical analysis methods uncovered features that distinguished founder viruses isolated from vaccinees from those isolated from placebo recipients, and identified HIV-1 genetic targets of vaccine-induced immune responses.

Summary: Studies of HIV-1 breakthrough infections in vaccine efficacy trials can provide an independent confirmation to correlates of risk studies, as they take advantage of vaccine/placebo comparisons, whereas correlates of risk analyses are limited to vaccine recipients. Through the identification of viral determinants impacted by vaccine-mediated host immune responses, sieve analyses can shed light on potential mechanisms of vaccine protection.

aStatistical Center for HIV/AIDS Research and Prevention, Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington

bU.S. Military HIV Research Program, Walter Reed Army Institute of Research, Silver Spring

cHenry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, Maryland, USA

Correspondenceto Morgane Rolland, Ph.D, Chief, Viral Genetics section, U.S. Military HIV Research Program (MHRP); HJF, 503 Robert Grant Avenue, Room 2A46, Silver Spring, MD 20910, USA. Tel: +1 301 319 2099; fax: +1 301 319 7684; e-mail:

Article Outline
Back to Top | Article Outline


In the last 5 years, the confluence of results from HIV vaccine efficacy trials and improved HIV-1 sequencing capacities have opened the possibility of using HIV-1 genetics to test hypotheses regarding the impact of vaccines on HIV-1 breakthrough infections.

Although sieve analysis as described herein relies on genetic sequencing, the concept of sieve analysis is classical to vaccinology. Numerous pathogens are classified in serotypes, which have historically been used to assess vaccine efficacy. In 2012, a randomized trial in 4002 Thai children showed that a dengue vaccine decreased infection rates by 55–100% against serotypes 1, 3 and 4, but had no effect against serotype 2, bringing the overall efficacy of the vaccine to 30% [1]. Although HIV-1 is not defined by serotypes, particular genetic features have been noted. For example, Env-V3 loop sequences can predict HIV-1 phenotypes. Partly motivated by this, Berman et al. conceived the idea of modern sieve analysis for HIV-1 vaccine trials [2], and Gilbert et al. pioneered a statistical framework [3,4]. Researchers sought to apply comparable methods in the context of phase I/II clinical trials of early candidate HIV-1 vaccines. However, such investigations were limited by the small number of infections in these trials [5,6].

Here we define sieve effects in the context of HIV-1 vaccine efficacy trials. Although more than 200 vaccine candidates have been tested clinically since 1987, only six vaccine efficacy trials, which tested four strategies, have been completed; and immunizations were discontinued in three trials due to futility. This review seeks to describe how investigation of sieve effects can help elucidate the mechanism(s) of action of a vaccine and yield clues for the development and testing of improved HIV-1 vaccine candidates.

Back to Top | Article Outline


The characterization of viruses that evade a vaccine-induced host immune response to establish HIV-1 infections, that is, ‘breakthrough viruses,’ can provide key insights into vaccine efficacy and subsequent immunogen design.

Owing to randomization at study entry and blinding of investigators and study subjects to treatment arms, vaccine and placebo recipients are expected to be exposed to similar circulating HIV-1 strains during a vaccine efficacy trial. Provided that the trial was well-conducted, with efficient blinding and no interference, genetic differences between HIV-1 sequences from the two treatment groups can be attributed to vaccination [3]. Thus, vaccine-induced immune pressure on HIV-1 strains can be ascertained against the background of nonvaccine-induced immune pressure in placebo recipients. The association of specific viral variants with vaccine efficacy can provide powerful clues to the mechanism of viral evasion of vaccine-induced immune pressure. This approach has been termed ‘sieve analysis.’

Sieve analysis is an integral part of the study of immune correlates of vaccine protection, as they allow to develop hypotheses regarding what immune characteristics may have been important in the protection afforded by a vaccine, and confirm that a specific immune response associated with reduced risk from HIV-1 infection had an impact on HIV-1 breakthrough sequences. For hypothesis-generating sieve analysis, if the vaccine partially protects and if sieve analysis detects that a specific HIV-1 region differs between HIV-1 sequences from vaccine and placebo recipients, then this is prima facie evidence for vaccine-exerted immune pressure. This raises the hypothesis that biomarkers measuring immune responses to this specific HIV-1 region are correlates of protection, thus guiding the design of studies to characterize these immune response biomarkers and assess them as correlates of protection. For confirmatory sieve analysis, the HIV-1 genetic comparisons between vaccine and placebo can validate that an immunological measurement is a correlate of protection against HIV-1 infection (i.e. an immune response that reliably predicts the level of vaccine efficacy).

The unique attribute of sieve analyses is that they directly compare vaccine and placebo groups, whereas correlates of risk analyses include only vaccine recipients; in correlates of risk studies, the vaccine recipients who became HIV-1 infected are compared to the vaccine recipients who remained uninfected during the trial period. As such, identified correlates of risk do not necessarily predict vaccine efficacy, because they may merely correlate with an exposure or natural susceptibility factor that actually determines subject differences in infection rates. In contrast, the advantage of vaccine versus placebo comparisons in sieve analyses is that a difference between HIV-1 sequences isolated from the two groups can be causally attributed to the vaccine status because vaccine treatment was randomly assigned at entry and both study subjects and investigators were blinded (provided that it is verified that no corruption occurred) [7].

Back to Top | Article Outline


A vaccine-induced sieve effect has been traditionally understood as the ability of a vaccine to block certain viruses from establishing HIV-1 infection [3,8], arguably the viruses that would be the most similar to the vaccine insert (Fig. 1); then, breakthrough viruses from vaccinees will be more divergent from the vaccine insert than the viruses in placebo recipients. Relatedly, for a vaccine based on amnestic cell-mediated immunity, a cytotoxic T lymphocyte (CTL)-based vaccine, we posit that following vaccination, CTL responses may lead to broader or more rapid CTL escape mutations driving epitopic regions in breakthrough sequences away from the insert sequence [9].

The exclusion of certain HIV-1 variants from establishing infection is defined as an acquisition sieve effect, whereas HIV-1 genetic polymorphisms that are driven by vaccine-induced anamnestic responses correspond to a postinfection sieve effect – influencing the evolutionary outgrowth of specific HIV-1 variants within an infected host. A postinfection sieve effect implies that HIV-1 founder variants are driven to accumulate more vaccine-mediated mutations and/or to accumulate them more rapidly than what is observed in the context of host immune pressure independently of vaccination (Fig. 1).

Sieve effects are traditionally viewed as leading to larger genetic distances from the vaccine insert among sequences from placebo recipients, as was found in a sieve analysis of the HIV-1 Step Trial [9]. However, recent results in the RV144 trial exemplified an opposite scenario in which distances to the insert were shorter among vaccinees (i.e, greater vaccine efficacy against HIV-1 s with certain sites mismatched to the vaccine); we refer to such unexpected sieve effect as atypical. For an acquisition sieve effect, it is plausible that a vaccine effect linked to the presence of antibodies may select against a subset of circulating strains specifically recognized by those antibodies. The excluded viral variants may not necessarily represent the viruses closest to the insert, as the small number of antibody contact residues and their tridimensional organization may supersede the genetic relatedness to the vaccine insert.

The detection of sieve effects hinges on our knowledge of the consequences of host immune pressure in natural infection, particularly the strength of the immune response both within a host (potency to cause escape in one's virus) and among hosts (how common is the response across individuals). In the Step/HVTN502 study, several factors intersected to make site Gag84 remarkable: multiple CTL epitopes span site 84, these responses are often immunodominant in a host, and HLA-A02 alleles that restrict many of these epitopes were the most frequent alleles in the study cohort. Hence, the power of sieve analysis, particularly to detect postinfection effects, improves with the confluence of common HLA alleles, immunodominant responses and ease of escape at a specific site in the context of low background evolutionary noise.

Although acquisition and postinfection sieve effects correspond to different concepts, they are nonetheless difficult to discern, as both are manifested by the presence of selected sequence polymorphisms in vaccine recipients. Further complicating the delineation of each process is the fact that these polymorphisms are not necessarily mutually exclusive in vivo, as vaccine-induced immune responses may be responsible for interdicting specific variants (i.e. an acquisition effect) as well as imprinting the HIV-1 variants that broke through and established a productive infection (i.e. a postinfection effect).

Back to Top | Article Outline


Sieve analysis of founder viruses from both Step/HVTN502 and RV144 showed evidence that vaccination exerted selective pressure on HIV-1 founder viruses, although the suspected sieving mechanisms differed.

The Step/HVTN502 trial, which tested an Ad5 vector with a gag/pol/nef HIV-1 subtype B insert, showed no vaccine-induced reduction in the rate of HIV-1 infections and no overall reduction in viral loads [10,11]. Nonetheless, further analyses suggested a weak and transient suppression of viral loads in acute infection [12] and showed that some variations in viral loads could be linked to specific HLA subgroups [13]. The vaccine was designed to elicit cell-mediated immunity, yet the breadth, strength, and/or focus of these responses were not sufficient to effectively reduce viral loads (in spite of T-cell responses detected by ELISpot in 77% of vaccine recipients [10]).

The hypothesis that vaccine-induced immune responses had an impact on breakthrough infections was tested by comparing in-silico predicted T-cell epitopes in the vaccine versus placebo groups, specifically the distance between epitopes predicted in the vaccine insert and corresponding epitopes in breakthrough sequences. Analysis of 465 genomes from 68 subjects showed that the epitopic distances between breakthroughs and the vaccine insert was larger among vaccinees than among placebo recipients, specifically in Gag [9]. When epitopes corresponding to segments of the proteome (e.g. Env) that were not included in the vaccine insert were tested, there was no vaccine/placebo difference. In addition, 10 signature sites distinguished viruses from the vaccine and placebo groups, including the exemplar, Gag-84, at which 32 of 39 vaccinees presented a mutation compared with 9 of 26 subjects in the placebo group. In sum, in Step/HVTN502, we noted typical sieve effects with larger distances in vaccinees, consistent with the idea of intensified escape mutations following vaccine-induced immune responses.

The RV144 trial, which enrolled 16 000 participants in Thailand between 2003 and 2009, showed modest vaccine protection from HIV-1 acquisition (31%) and prompted efforts to evaluate correlates of risk [14]. As higher antibodies against Env-V1V2 correlated with a lower risk of infection, RV144 sieve analyses were initially geared toward testing for an effect in the Env-V1V2 region of breakthrough infections. A sieve analysis based on over a 1000 HIV-1 genomes from 121 subjects showed that two sites exhibited sieve effects: positions 169 and 181. Vaccine efficacy was increased to 48% against viruses matching the vaccine at position 169 (with a K), and to 78% against viruses mismatching the vaccine at position 181 (residue different from I) [15▪]. The value of sieve analysis is augmented when results can be interpreted together with findings from immunological studies, and several RV144 studies further described the role of binding V2 antibodies [16▪,17,18]; in particular, Liao et al.[19] characterized four V2-specific monoclonal antibodies isolated from RV144 trial participants and found that these antibodies mediated antibody-dependent cellular cytotoxicity in a manner that depended on a vaccine-match at site 169. Thus, together with immunological studies, sieve results supported the hypothesis that the RV144 vaccine regimen specifically blocked certain HIV-1 viruses defined by the two V2 mutations at sites 169 and 181.

Back to Top | Article Outline


Although the ability to deconvolute acquisition sieve effects from postinfection effects depends on the strength, breadth and the rapidity at which the vaccine-induced pressure is exerted on HIV-1 sequences postinfection, it is also crucially affected by the timing of HIV-1 diagnosis: the earlier HIV-1 founder strains are characterized, the more likely one is able to differentiate an acquisition effect from a postinfection effect.

Based on Step/HVTN502 and RV144 data, we calculated the mean pairwise diversity among Env-C2V5 sequences from each subject. Distances were computed only for subjects for whom at least five sequences were available (118 of 121 RV144 subjects and 54 of 68 Step subjects). The median pairwise diversity was significantly higher among sequences from RV144 volunteers (0.42%) than among sequences from Step volunteers (0.13%). This is likely because of a later diagnosis of HIV-1 infections in RV144 volunteers, as HIV-1 diagnosis occurred an average of 181 days after the last negative visit for RV144 participants compared with 105 days for Step individuals (P < 0.0001).

As the interval between visits conditions the ability to diagnose HIV-1 infections in their earliest phase, it is important that efficacy trials are designed to allow viral sequencing as early as possible after infection. Studies of HIV-1 evolution showed few random mutations during the first month of infection before HIV-1 starts evolving rapidly as a consequence of the host-induced immune pressure [20,21]. Hence, scheduling diagnostic visits every month would be beneficial for subsequent sequence analyses but may not be practically feasible. The goal could be to diagnose 50% of HIV-1 infections preseroconversion, and more studies should be done to assess how many infections are identified preseroconversion based on 1, 2 or 3-month visit schedules; results from RV144 showed that a 6-month visit schedule was not optimal, as only six of the 125 cases in the modified intent-to-treat analysis had a nonreactive enzyme-linked immunosorbent assay at diagnosis (based on a test approved in Thailand). Apart from frequent HIV-1 infection assessments, it is also important that sufficient endpoints accrue, preferably through cohorts with high incidence as opposed to large-scale trials or long-running low-incidence trials.

Back to Top | Article Outline


Sieve analyses compare HIV-1 breakthrough infections in vaccinees with the viruses isolated from placebo recipients and can identify viral features that were specific to the vaccines's viruses. Deconstructing the genetic signal linked to vaccine-induced immune responses can help describe qualitatively and quantitatively the immune responses induced by the vaccine and gauge their effectiveness through their impact on the virus.

Back to Top | Article Outline


The authors thank Drs Jerome H. Kim, Nelson L. Michael, James I. Mullins and Robert J. O’Connell for insightful comments and discussions. We thank Rapee Trichavaroj for providing information on the HIV-1 infections diagnosed preseroconversion in the RV144 trial.

Back to Top | Article Outline

Conflicts of interest

This work was supported by a cooperative agreement (W81XWH-11-2-0 174) between the Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc., and the US Department of Defense (DoD), by an Interagency Agreement Y1-AI-2642-12 between the US Army Medical Research and Material Command (USAMRMC) and the National Institute of Allergy and Infectious Diseases, and by the National Institute Of Allergy And Infectious Diseases of the National Institutes of Health under Award Number R37AI054165 (to P.B.G).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the US Army or the Department of Defense.

There are no conflicts of interest.

Back to Top | Article Outline


Papers of particular interest, published within the annual period of review, have been highlighted as:

▪ of special interest

▪▪ of outstanding interest

Additional references related to this topic can also be found in the Current World Literature section in this issue (p. 514).

Back to Top | Article Outline


1. Sabchareon A, Wallace D, Lang J, et al. Efficacy of tetravalent dengue vaccine in Thai schoolchildren: authors’ reply. Lancet 2013; 381:1094–1095.
2. Berman PW, Gray AM, Wrin T, et al. Genetic and immunologic characterization of viruses infecting MN-rgp120-vaccinated volunteers. J Infect Dis 1997; 176:384–397.
3. Gilbert PB, Self SG, Ashby MA. Statistical methods for assessing differential vaccine protection against human immunodeficiency virus types. Biometrics 1998; 54:799–814.
4. Gilbert P, Self S, Rao M, et al. Sieve analysis: methods for assessing from vaccine trial data how vaccine efficacy varies with genotypic and phenotypic pathogen variation. J Clin Epidemiol 2001; 54:68–85.
5. Graham BS, McElrath MJ, Connor RI, et al. Analysis of intercurrent human immunodeficiency virus type 1 infections in phase I and II trials of candidate AIDS vaccines. AIDS Vaccine Evaluation Group, and the Correlates of HIV Immune Protection Group. J Infect Dis 1998; 177:310–319.
6. Connor RI, Korber BT, Graham BS, et al. Immunological and virological analyses of persons infected by human immunodeficiency virus type 1 while participating in trials of recombinant gp120 subunit vaccines. J Virol 1998; 72:1552–1576.
7. Rolland M, Gilbert P. Evaluating immune correlates in HIV type 1 vaccine efficacy trials: what RV144 may provide. AIDS Res Hum Retroviruses 2012; 28:400–404.
8. Berman PW. Development of bivalent rgp120 vaccines to prevent HIV type 1 infection. AIDS Res Hum Retroviruses 1998; 14 (Suppl 3):S277–289.
9. Rolland M, Tovanabutra S, deCamp AC, et al. Genetic impact of vaccination on breakthrough HIV-1 sequences from the STEP trial. Nat Med 2011; 17:366–371.
10. McElrath MJ, De Rosa SC, Moodie Z, et al. HIV-1 vaccine-induced immunity in the test-of-concept Step Study: a case-cohort analysis. Lancet 2008; 372:1894–1905.
11. Buchbinder SP, Mehrotra DV, Duerr A, et al. Efficacy assessment of a cell-mediated immunity HIV-1 vaccine (the Step Study): a double-blind, randomised, placebo-controlled, test-of-concept trial. Lancet 2008; 372:1881–1893.
12. Janes H, Frahm N, DeCamp A, et al. MRKAd5 HIV-1 Gag/Pol/Nef vaccine-induced T-cell responses inadequately predict distance of breakthrough HIV-1 sequences to the vaccine or viral load. PloS one 2012; 7:e43396.
13. Janes H, Friedrich DP, Krambrink AM, Smith RJ, Kallas E, Horton H, et al. Vaccine-induced Gag-specific T cells are associated with reduced viremia after HIV infection. The Journal of Infectious Diseases In press.
14. Rerks-Ngarm S, Pitisuttithum P, Nitayaphan S, et al. Vaccination with ALVAC and AIDSVAX to prevent HIV-1 infection in Thailand. N Engl J Med 2009; 361:2209–2220.
15▪. Rolland M, Edlefsen PT, Larsen BB, et al. Increased HIV-1 vaccine efficacy against viruses with genetic signatures in Env V2. Nature 2012.

This is the first study to tie sieve effects on HIV-1 breakthrough viruses together with an immunological correlate of risk of HIV-1 infection in the context of the RV144 vaccine efficacy trial.

16▪. Haynes BF, Gilbert PB, McElrath MJ, et al. Immune-correlates analysis of an HIV-1 vaccine efficacy trial. N Engl J Med 2012; 366:1275–1286.

This is the first study to identify immunological correlates of risk of HIV-1 infection.

17. Karasavvas N, Billings E, Rao M, et al. The Thai Phase III HIV Type 1 Vaccine trial (RV144) regimen induces antibodies that target conserved regions within the V2 loop of gp120. AIDS research and human retroviruses 2012; 28:1444–1457.
18. Zolla-Pazner S, deCamp AC, Cardozo T, et al. Analysis of V2 antibody responses induced in vaccinees in the ALVAC/AIDSVAX HIV-1 vaccine efficacy trial. PloS one 2013; 8:e53629.
19. Liao HX, Bonsignori M, Alam SM, et al. Vaccine induction of antibodies against a structurally heterogeneous site of immune pressure within HIV-1 envelope protein variable regions 1 and 2. Immunity 2013; 38:176–186.
20. Goonetilleke N, Liu MK, Salazar-Gonzalez JF, et al. The first T cell response to transmitted/founder virus contributes to the control of acute viremia in HIV-1 infection. J Exp Med 2009; 206:1253–1272.
21. Herbeck JT, Rolland M, Liu Y, et al. Demographic processes affect HIV-1 evolution in primary infection before the onset of selective processes. J Virol 2011; 85:7523–7534.

HIV-1 breakthrough infections; sieve analysis; vaccine-induced immune responses; viral genetics

Copyright © 2013 Wolters Kluwer Health, Inc. All rights reserved.