Investigating the utility of the HIV-1 BED capture enzyme immunoassay using cross-sectional and longitudinal seroconverter specimens from Africa

Background: The identification of populations at risk of HIV infection is a priority for trials of preventive technologies, including HIV vaccines. To quantify incidence traditionally requires laborious and expensive prospective studies.

Methods: The BED IgG-Capture enzyme immunoassay (EIA) was developed to estimate HIV-1 incidence using cross-sectional data by measuring increasing levels of HIV-specific IgG as a proportion of total IgG. To evaluate this assay, we tested 189 seroconversion samples taken at 3-monthly intervals from 15 Rwandan and 26 Zambian volunteers with known time of infection and cross-sectional specimens from 617 Kenyan and Ugandan volunteers with prevalent infection.

Results: The BED-EIA-estimated incidence in Uganda was unexpectedly high, at 6.1%/year [95% confidence interval (CI) 4.2-8.0] in Masaka and 6.0%/year (95% CI 4.3-7.7) in Kakira. Prospective incidence data in Masaka from the same population was 1.7%/year before and 1.4%/year after the study. Kenyan estimates were 3.5%/year in Kilifi (95% CI 2.1-4.9) and 3.4%/year in Nairobi (95% CI 1.5-5.3). From the Rwandan and Zambian data, the sensitivity of the assay was 81.2% and the specificity was 67.8%. After approximately one year, subjects misclassified as recently infected tended to have lower plasma viral loads compared with those not misclassified as recent (median copies/ml 14 773 versus 93 560; P = 0.02). Clinical presentation, sex and HIV subtype were not significantly associated with BED-EIA misclassification in seroconverter samples.

Conclusion: These data suggest that this assay does not perform reliably in all populations. Further research is warranted before using this assay to estimate incidence from prevalent HIV samples.

Introduction

Obtaining accurate estimates of the incidence of HIV-1 is essential to determine suitable study populations and to estimate the sample size when designing interventional trials ^[1]. Traditionally, this has meant enrolling at-risk volunteers into expensive, time-consuming longitudinal studies to measure HIV incidence. A newer assay designed to take advantage of the evolving antibody response to HIV-1 after infection, the BED IgG Capture enzyme immunoassay (EIA), was designed to address the shortcomings of previous methodologies, and has been evaluated in individuals infected with subtypes B and E ^[2].

We describe here the application of the BED-EIA on prevalent specimens from cross-sectional studies at four sites in Kenya and Uganda. To assess the reliability of the BED-EIA at detecting recent infections in African populations, we also tested panels of sera from Zambian and Rwandan individuals with a known time of HIV-1 infection.

Methods

Six sites in Africa were selected to participate in epidemiological studies in the context of HIV vaccine trial preparedness. This work received approval from ethics review boards in each country and at the Emory University School of Public Health.

Four sites conducted a cross-sectional study on HIV prevalence. The populations represented rural residents in the Masaka District of Uganda, rural and semi-urban residents in Kilifi, Kenya [including attendees at a voluntary counseling and testing (VCT) clinic], an occupational cohort from a sugar plantation in Kakira, Uganda, and sex workers and clients in the Kangemi district, Nairobi, Kenya. Consenting volunteers, aged 18-60 years, had demographic and risk behavior information collected. Pre and post-HIV test counseling was performed. Prospective data were available from the same study population in Masaka, Uganda, before this survey (ages ≥ 13 years) and after this survey (ages 18-60 years) ^[3,4]. All infected volunteers were referred for care.

Two research units, Projet San Francisco in Kigali, Rwanda, and Zambia-Emory HIV Research Project in Lusaka provide couples with VCT. As part of a study of HIV heterosexual transmission, consenting couples with discordant HIV status are followed quarterly ^[5]; when new infections are identified, this follow up includes additional counseling, support, and care. The date of seroconversion is estimated as the midpoint between the date of the last negative antibody test and detection, plus 28 days. Data were collected on the use of antiretroviral therapy (ART) and the occurrence of symptoms associated with HIV infection. Plasma viral load was measured using the Amplicor HIV-1 MONITOR v.1.5 (Roche Molecular Systems, California, USA). HIV was subtyped by sequencing a conserved region of gp41 (nucleotides 7916-8300 in HXB2 ^[6]), and using the RIP 2.0 program (Los Alamos National Laboratory HIV Sequence Database).

The BED IgG Capture incidence EIA is commercially available (Calypte, Maryland, USA), using gp41 immunodominant regions from subtypes B, E and D. The assay detects increasing levels of HIV-1-specific IgG as a proportion of overall IgG. All specimens had confirmation of antibody status (Axsym HIV-1 1/2 EIA; Abbot, Illinois, USA) before testing. The BED-EIA was performed according to the manufacturer's instructions by technicians trained in the use of the assay. Specimens that registered less than 0.8 normalized optical density (ODn = specimen-OD/calibrator-OD) were termed positive. Positive specimens were considered 'recent infections' and negatives (ODn ≥ 0.8) were considered 'longer term infections'. Rwandan/Zambian specimens that were BED recent but drawn more than 153 days after seroconversion were considered false positive (FP) and specimens that were BED longer term taken at or before 153 days were considered false negative (FN). Specimens that are FP more than one full follow-up window after the 153 day cutoff were considered persistently FP.

Predictors of persistent FP were tested by χ² or Wilcoxon rank sum tests. A logistic regression model was created to explore independent predictors of persistent FP. BED-EIA sensitivity and specificity was calculated with generalized estimating equations using an exchangeable correlation structure to control for multiple time points and multiple data points per volunteer. The average time to crossing the ODn = 0.8 threshold was calculated using a non-parametric product-limit estimate of the 'survival distribution' (i.e. Kaplan-Meier). A receiver operating characteristic (ROC) analysis was conducted to determine optimal assay parameters. Incidence estimates from the prevalent specimens are shown unadjusted, adjusted for missing data, and adjusted using Centers for Disease Control and Prevention incidence adjustment formulae (J. Hargrove et al., 2006, in preparation) ^[7].

Results

Prevalent data

In 2004, 6436 volunteers were enrolled. We identified 694 HIV infections with 617 (88.9%) available specimens; all confirmed HIV antibody positive. The BED-EIA-estimated annual incidence was highest in Uganda: 6.1% in Masaka, 6.0% in Kakira (Table 1) and lower in Kilifi (3.5%) and Nairobi (3.4%). The BED-EIA results from Masaka are three to four times higher than observed data from the same population in 1994-2000 and 2004-2006 (Table 1). CD4 cell count data were available in Kilifi and Kakira in volunteers who returned for care. Among 35 HIV-infected, BED-EIA recent volunteers who returned to have CD4 cell counts performed, four (11%) had counts less than 50 and seven (20%) had CD4 cell counts between 116 and 467 cells/μl.

Table 1

Prospective data

Over the period from July 2003 to February 2005, 41 HIV-infected volunteers were analysed (Table 2). The median age at seroconversion was 31 years (range 19-43). Volunteers provided an average of 4.6 specimens (range 2-8) for a median of 383 days post-seroconversion. The median time from seroconversion to the first visit was 19 days (range 0-85). Three volunteers identified with discrepant rapid tests, but confirmed by a third tie-breaker, were enrolled at day 0. Seventy specimens were collected before the 153 day post-seroconversion assay cutoff, 119 after. All Rwandan infections were subtype A1; all Zambian infections were subtype C.

Table 2

Fifteen of 37 volunteers (40.5%) with adequate follow-up were persistently FP, including two (5.4%) who crossed the ODn = 0.8 threshold and later reverted to FP. Of 30 volunteers with data past one year (median follow-up 458 days), eight remained FP (26.7%).

Persistent FP was not significantly associated with subtype [7/23 (30.4%) in C specimens versus 8/14 (57.1%) in A1; P = 0.11]. The volunteer plasma viral load in 40 volunteers at approximately one year (median of 376 days) was significantly lower among persistently FP volunteers (median viral loads 14 773 versus 93 560; P = 0.02). In multivariate analysis, viral load remained associated with persistent FP (Table 3). Three volunteers who began ART at approximately one year saw their BED-EIA ODn results decline. Volunteers who reported one or more symptoms associated with HIV at baseline, 6 months or 12 months were neither more nor less likely to be persistent FP (P = 0.3, 0.4 and 0.9, respectively).

Table 3

Although the median volunteer time to crossing ODn = 0.8 was similar to the mean provided by the package insert (151.4 days versus 153), there was considerable variability in our estimate [95% confidence interval (CI) 108.5-264.7]. The median time to crossing the ODn threshold was 147 days shorter among the Zambian than the Rwandan volunteers, the difference was not significant (117.7 versus 264.7; P = 0.6; Fig. 1). The same was true across men and women (123.2 days versus 210.8 days; P = 0.3).

Fig. 1

The assay sensitivity in this study was 81.2% and the specificity was 67.8%. There was considerable variability in these estimates (Table 2). Differences across nationality and sex were not statistically significant (P = 0.5 and 0.18, respectively). The optimal assay cutoff values by ROC analysis (Fig. 2) for our data were ODn = 0.5 and 80 days post-seroconversion.

Fig. 2

Discussion

The development of an assay to estimate incidence from prevalent specimens would save considerable money and time. The BED-EIA was designed to be used in multiple populations. Our results suggest that in some populations the BED-EIA overestimates incidence compared with prospectively collected data, partly as a result of poor specificity. Volunteers in this study with a lower viral load tended to be FP by BED-EIA.

Although only three volunteers followed prospectively began ART during the course of this study, their BED-EIA ODn values fell at subsequent visits. The control of viral replication over time may also influence BED-EIA results, although we do not have prospective viral load data. We also reported on two volunteers who saw their BED-EIA results cross the ODn = 0.8 threshold and later returned to FP status (one at approximately 173 days, another at approximately 444 days post-seroconversion). Neither of these volunteers used ART.

AIDS can also lead to the misclassification of chronic infection as FP by BED-EIA, presumably as a result of lower IgG levels ^[8,9]. Thirty-five BED-EIA-positive volunteers from the cross-sectional study had CD4 cell counts performed, and 31% (11) had counts less than 500 cells/μl, suggesting long-term infection. Among those followed prospectively, no volunteers were infected for longer than 2 years, and there was no apparent association between the signs and symptoms of HIV disease and FP. Although published reports are limited, others have noted BED-EIA FP individuals with AIDS or on ART ^[10].

This is, to our knowledge, is the first report of BED-EIA performance using seroconverter panels from subtypes C and A1, and the first comparison of BED-EIA-estimated incidence from an African population compared witho prospective data from the same population. Prospective data are the gold standard to evaluate these results. In Masaka, longitudinal incidence data from the same study villages, collected before and after the cross-sectional study, were considerably less than what the BED-EIA predicted. Prospective data from the other sites are not available; however, a clinical trial in a different population of female sex workers in Nairobi between 1998 and 2002 reported an estimated incidence (3.6 cases/100 person-years ^[11]) similar to our BED-EIA estimate.

A strength of the prospective data presented here is our ability to estimate accurately the date of seroconversion. Volunteers are seen quarterly, enabling us to estimate this date to within a few weeks/days. After the volunteers' first visit, the variability in the BED-EIA ODn results tended to be high, ranging from 0.097 to 3.064 between 70 to 120 days post-seroconversion. Many of the highest FN values at this time fell in subsequent months; however, the variability remained high across the entire period of follow-up.

A significant weakness of this study in the light of this variability was our limited sample size. Although our data suggest differences in sex and subtype, none were statistically significant. We did, however, observe an association between misclassification and viral load. Parekh et al. ^[2] noticed a 30-day difference in the time it took subtype E-infected volunteers to cross the ODn = 0.8 threshold compared with subtype B-infected volunteers (115 versus 145 days). Those volunteers infected with subtype E also had higher average viral loads in the first 3 months of infection than subtype B infections ^[12]. As access to ART improves, and some populations begin to observe a therapy-induced reduction in viral load, the utility of the BED-EIA may be affected. Censoring data from individuals on ART or with AIDS when measuring prevalent specimens may improve the specificity of the assay. This presents challenges, however, in settings where expertise and CD4 cell counts are scarce. Additional unpublished data from work in Ethiopia (subtype C), Zimbabwe (subtype C), Kenya (subtypes A and D) and the Netherlands (subtype B) has shown varying assay window periods from 120 to 180 days (B. Parekh, 2006, personal communication). Our ROC analyses suggested different assay cut-off parameters compared with those provided in the package insert; however, the 80-day window period we report here would require larger sample sizes, and reassessing the BED-EIA with local seroconverter panels for each new cross-sectional study may be more challenging than simply conducting a prospective incidence study.

The BED-EIA produced an estimate of HIV incidence in Uganda three to four times higher than that observed prospectively in the same population before and after this work. Whereas a similarly high estimate was also recorded for the other Ugandan site, our BED-EIA-derived estimate from Kangemi was consistent with prospectively observed data from a similar population also in Nairobi ^[11]. We have attempted to correct these values using an adjustment factor to correct for sensitivity and specificity (J. Hargrove, et al., 2006, in preparation) ^[7]; however, this did not improve the Masaka data. Our prospective data from Rwanda and Zambia also suggest that this assay is not identifying recent infections with adequate accuracy to estimate incidence in similar populations. These results have implications for the use of the BED-EIA on prevalent African specimens. The poor specificity will inflate incidence estimates measured from cross-sectional studies, and without a population-specific understanding of the rates of FP and FN, mathematical corrections are not appropriate. The BED-EIA accuracy may be susceptible to host response, ART, viral load, and possibly sex, and viral subtype. The results from this work and others presented at a UNAIDS Consensus Meeting ^[13] prompted the release of a statement recommending that the BED-EIA not be used for estimating incidence from prevalent specimens ^[14] (http://www.epidem.org/publications.htm).

Acknowledgements

The authors would like to thank the staff and volunteers at the study sites, and the laboratory technicians who performed the assay: Bosco Agaba, Juliet Nsimire, Caroline Ngetsa, Micah Oyaro, John Gachie, Faith Wamalugulu, Brian Magambo, Shila Glass, Sydney Chanda, A Gahoranimana, and Emmanuel Tekirya. They also thank Amanda Tichacek, Lisa Jones and Jane Atkinson for preparing volunteer clinical data, and Paul Farmer, PhD and the Emory Center for AIDS Research Virology Core (National Institutes of Health grant P50 AI-050409) for viral loads and subtyping. The authors also appreciate Dr Bharat Parekh's review, and the assistance of Drs Andrea Kim, John Hargrove and Steve McDougal with the Centers for Disease Control and Prevention adjustment formulae.

The International AIDS Vaccine Initiative (IAVI) Collaborative Seroprevalence and Incidence Study Team consisted of: Gwynn Stevens, International AIDS Vaccine Initiative, New York; Wendy Stevens and Paramesh Chetty, Department of Molecular Medicine and Haematology, University of Witwatersrand, Johannesburg; Patricia Fast, International AIDS Vaccine Initiative, New York; Olivier Manigart, Zambia Emory HIV Research Project, Lusaka; Josephine Birungi, International AIDS Vaccine Initiative, New York; Bashir Farah, Kenyan AIDS Vaccine Initiative, Nairobi, Kenya; and the Rwanda Zambia HIV Research Group.

The Rwandan and Zambian data have previously been presented at the AIDS Vaccine Conference in Montreal, Canada (6-9 September 2005). The Kenyan and Ugandan data have been presented at the UNAIDS Reference Group on Estimates, Modelling, and Projections in Athens, Greece (on 13 December 2005) and at the Microbicides Conference in Cape Town, South Africa (24-26 April 2006) reference no. 255/913.

Sponsorship: This work was made possible through funding from the International AIDS Vaccine Initiative and NIAID grant no. AI-051231.