Secondary Logo

Journal Logo

Original papers

Evaluation of rapid tests for anti-HIV detection in Brazil

Ferreira, Orlando C Juniora; Ferreira, Cristineb; Riedel, Maristelac; Widolin, Marcya Regina Visinonic; Barbosa-Júnior, Aristidesbfor the HIV Rapid Test Study Group

Author Information
doi: 10.1097/01.aids.0000191494.51489.75

Abstract

Introduction

The conventional strategy for the diagnosis of HIV-1 infection in Brazil includes combining two competing enzyme immunoassay (EIA) screening assays plus a confirmatory assay, using Western blot or immunofluorescence (referred to as the ‘gold standard’). This diagnostic system is highly specific and sensitive for the detection of anti-HIV-1 antibodies but has some operational constraints. It requires highly trained laboratory technicians and a developed laboratory infrastructure. In addition, the turnaround time for results to be returned to the client could be 1–2 weeks or more, thus resulting in individuals lost to follow-up, delays in diagnosis and referral to much-needed services.

By 1985, shortly after the introduction of EIA, rapid/simple anti-HIV-1 tests (RT) became increasingly available [1]. In the beginning, the performance of HIV RT assays was poor when compared with EIA. However, in the past few years rapid/simple assays were improved and new assays were developed based on new technologies [2]. As a consequence, some HIV RT assays have comparable performance with EIA [3–5]. They require no laboratory expertise and can be executed in a few steps in less than 20 min.

The World Health Organization recommends a rationalized RT algorithm to diagnose HIV-1 [3]. As the predictive value of a single screening assay depends on the HIV prevalence of the population tested, a single HIV RT assay is not feasible as a tool for a rapid HIV diagnosis. Therefore, the World Health Organization proposed the sequential use of two or three different HIV RT assays for the rapid diagnosis of HIV infection in asymptomatic individuals. Such a strategy has been applied with success in other countries [6–8].

In this project we have evaluated seven commercially available HIV RT assays using whole blood samples from individuals at voluntary counselling and testing (VCT) sites and from pregnant women at antenatal clinics (ANC). Our aim was to create a methodology for the evaluation of HIV RT assays using whole blood in Brazil. This effort will pave the way for the development and future implementation of a three-test algorithm that would rely only on RT assays for the rapid diagnosis of HIV infection in Brazil.

Methods

Ethics Committee approval

This was an anonymous unlinked study approved by the Conselho Nacional de Ética em Pesquisa, CONEP (Brazilian National Ethic Committee for Human Research). Informed consent was obtained from each participant before enrollment.

Sample collection and HIV testing

Between 16 June and 29 August 2003, a total of 1100 samples were collected for the assessment from three different population sites: (i) from the VCT (n = 400); (ii) ANC (n = 500); and (iii) from HIV-infected individuals, as a positive control group (n = 200). After obtaining informed consent, 5 ml whole blood was collected by venous puncture in ethylenediamine tetraacetic acid tubes. Samples were labelled and sent to the Public Health Laboratory of the city of Curitiba where RT and the serological gold standard assays were performed on the same day.

The seven RT evaluated were: Determine HIV-1/2 (Abbott Laboratories, Diagnostics Division, 100 Abbott Park Road, Abbott Park, II 60064-3500, USA); HIV Rapid Check (NDI-UFES, Núcleo de Deonças Infecciosas, Universidade Federal do Espirito Santo, Av. Marechal Campos, 1468 – Maruípe, 29.040-091, Vitória (ES), Brazil); Hexagon HIV 1+2 (Human GmbH, Max-Planck-Ring 21, D 65205, Wiesbaden, Germany); HIV 1/2 STAT PAK (Chembio Diagnostic Systems Inc., 3661 Horseblock Road, Medford, BY 11763, USA); Hema-Strip HIV-1/2 (Saliva Diagnostic Systems, (SDS), 11719 NE 95th Street, Vancouver, WA 98682, USA); Cappillus HIV-1/HIV-2 and Uni-Gold HIV (Trinity Biotech plc, IDA Business Park, Bray, Co., Wicklow, Ireland). All assays were performed and results interpreted according to the manufacturer's packaging insert recommendations.

Four laboratory technicians rotated in performing two RT per day, with whole blood in batches of four samples. Every test result was read by two technicians and by a third in cases of discordant results.

After completion of the RT, the samples were centrifuged in order to recover the leftover plasma. The plasma volume was split into two aliquots: (i) 1–2 ml for the plasma repository; and (ii) 0.5 ml for gold standard screening (EIA and, if necessary, confirmatory Western blot).

Daily, three samples (at least one negative and one positive) were selected for a quality control procedure. All seven RT assays were repeated and in case of discordant results between the initial RT and quality control testing, a third testing with the initial RT sample batch was repeated for the assay that scored the discordant result. The sample final result was the median among the three results obtained.

In order to ensure comparability, all collected specimens that underwent RT testing were also evaluated with the gold standard, including two sandwich EIA from different manufacturers: Vitrus anti-HIV1/2 (Ortho-Clinical Diagnostics, Inc., Rochester, New York, USA) and Axsym HIV-1/2 gO (Abbott GmbH, Wiesbaden-Delkenheim, Germany) and the New Lav Blot I (Bio-Rad, Marnes-la-Conquette, France) as a confirmatory assay. Our criteria for Western blot positivity requires the presence of the p24 gag protein plus any two proteins from the virus envelope (gp41, gp120 or gp160), regardless of band intensity.

In order to address the analytical sensitivity of the assays used in this study, we have used one commercial seroconversion panel from Boston Biomedica Inc. (BBI; panel number PRB931) and an in-house panel from a seroconverting blood donor, originally identified by individual nucleic acid technology screening in Brazil.

Assays were also evaluated for performance against an HIV-1 subtype panel constituted of the main three HIV-1 subtypes circulating in Brazil (B′, C and F) plus the B subtype. Each subtype was represented by four samples.

Criteria used for acceptance and ranking the rapid test performance

The criteria used were: (i) clinical sensitivity (≥ 99.5%); (ii) sensitivity against the subtype panel; (iii) analytical sensitivity against the two seroconversion panels; (iv) clinical specificity (≥ 99.0%); and (v) operational assay performance.

The operational assay performance evaluates some positive (score 1) and negative (score 0) operational characteristics of the assay as follows: (i) number of reagents needed to run the assay (1, only one reagent needed; and 0, more than one reagent needed); (ii) reagent storage temperature (1, ambient temperature possible; and 0, 2–8°C required); (iii) total number of assay steps (1, equal to or less than four steps; and 0, more than four steps); (iv) total performance time (1, equal to or less than 20 min; and 0, more than 20 min); and (v) technical skill needed by the operator (1, no laboratory experience; and 0, laboratory experience recommended). The assay performance was considered good if the assay has scored at least four points and poor if less than four points.

Data management and statistical analysis

Daily results were transferred to an Excel spreadsheet (Microsoft Windows 2000; Microsoft Corp., Redmond, Washington, USA) used for data analysis and 95% confidence interval (CI) calculation.

Results

A total of 1100 samples were screened by seven HIV RT and the two gold standard EIA. Of the 400 samples from the VCT site, 23 were repeat reactive on one or both screening EIA, and of those 18 were Western blot positive (site prevalence of 4.5%; 95% CI 2.5–6.5%). The other five samples were Western blot indeterminant (n = 1) or negative (n = 4). Of the 500 samples screened from the ANC, six were repeat reactive on one or both EIA and three samples were Western blot positive (site prevalence of 0.6%; 95% CI 0.0–1.3%). The other three samples were Western blot indeterminant (n = 2) or negative (n = 1). All 200 samples from the positive control group were repeat reactive on both EIA and Western blot positive.

Clinical sensitivity and specificity evaluation

By using the gold standard assays, 221 samples were characterized as Western blot positive and served for the evaluation of RT clinical sensitivity. All other 879 samples (from VCT and ANC) were considered HIV-1/2 antibody negative, including the 871 samples negative on both screening EIA, five samples negative on Western blot and the three samples with an indeterminant result on Western blot.

Table 1 demonstrates the clinical sensitivity and specificity of the RT and the two EIA. All RT except the HemaStrip and Hexagon detected the 221 Western blot-positive samples (clinical sensitivity of 100%). The Hexagon assay missed two samples and the HemaStrip assay missed five samples resulting in a clinical sensitivity of 99.10% (95% CI 97.85–100%) and 97.74% (95% CI 95.78–99.70%), respectively.

Table 1
Table 1:
Clinical sensitivity and specificity of the seven rapid tests plus the two gold standard enzyme immunoassays.

The UniGold, Capillus and HemaStrip assays had a clinical specificity of 100% followed by the Determine and RapidCheck assays with a clinical specificity of 99.89% (missed one sample), the Stat Pak assay with a clinical specificity of 99.77% (missing two samples), and the Hexagon assay with a clinical specificity of 99.43% (missing five samples). The Vitrus EIA missed three samples (two from ANC) and the Axsym EIA missed six (three from ANC), resulting in clinical specificity of 99.66 and 99.32%, respectively. The nine false-positive results detected by the RT came from nine different samples. However, the Hexagon RT and the Axsym EIA generated false-positive results for the same sample from ANC. This sample was Western blot indeterminant with the presence of p24, a weak p55 and a weak p66 band.

Serconversion panels and HIV-1 subtype panel

All seven RT and the two EIA were tested against two seroconversion panels. The BBI PRB931 is a nine sample panel, with the last four samples carrying antibody against HIV. All RT and EIA but the Capillus RT, gave positive results for the four samples. Capillus scored positive for the last three samples only.

Our in-house seroconversion panel is a six-sample panel with the last four samples carrying antibody against HIV (samples 3–6). In addition to the two EIA, among the RT only, the Determine and RapidCheck assays scored a positive result for the four antibody-positive samples. The UniGold, Stat Pak and Capillus RT scored positive for the last three samples only. The HemaStrip RT scored a weak positive result only for sample number 6. The Hexagon RT showed an irregular reactivity profile. It was weakly positive for samples 1 and 4, positive for sample 5 and negative for sample 6.

All seven RT scored positive results with all 16 samples of the HIV-1 subtype panel.

Operational assay performance

Five characteristics of the RT were used for the evaluation of their operational performance described in the methods section.

Determine, RapidCheck, UniGold, Stat Pak and HemaStrip had a maximum score (5/5) (Table 2). Typically, those assays start with the sample application to test device followed by the addition of a buffer solution, which takes less than 2 min. Results should be read after 10–15 min, but for some assays it can be read up to 20 (RapidCheck and UniGold) or 60 (Determine) min. Any individual with a specific training on these RT could perform the assays. Furthermore, the wide temperature range for reagent storage includes room temperature.

Table 2
Table 2:
Evaluation of the operational assay performance based on five characteristics of the rapid test.

The Capillus assay had a poor performance (scoring 1/5) (Table 2). It has six steps and utilizes three different reagents. One of the steps involves the application of latex beads to the test device and sample mixing, which are critical to a good assay performance. This step is not simple, as it requires good laboratory skills. The test result can be read in approximately 9 min, but it again requires a well-trained or experienced technician for the recognition of latex agglutination. Illumination of the area where the test is being performed can also affect recognition of the agglutination and therefore test interpretation. Reagents should be kept refrigerated (at 2–8°C), limiting test utilization in the field.

The Hexagon assay also had a poor performance (scoring 1/5) (Table 2). It has 11 different steps that can take up 4–5 min before the final incubation that lasts 5 min and after which the results should be read immediately. Every step involves the addition of one out of three different solutions that should be added timely. It is clear that this assay requires a great deal of time and attention. Reading the final result needs good judgement as it involves a comparison between colorimetric changes developed in the test area with the one developed in the control area. As with the Capillus assay, reagents should be kept refrigerated (at 2–8°C).

Quality control procedure

A total of 157 samples (14.3% of the total 1100) had their RT repeated for quality control purposes. Two Western blot-positive samples from the positive control group showed discordant results when the initial RT was compared with quality control testing. In one case, the Hexagon assay was initially negative but quality control and the third repeat testing were positive, thus correcting the initial false-negative result.

The second case involved the Stat Pak and HemaStrip assays. Both assays were positive on initial RT testing but quality control testing was negative for both assays. The third repeat testing was weakly positive for the Stat Pak assay, thus confirming this sample result as positive. However, the HemaStrip assay again scored a negative.

The clinical sensitivity scorings for these tests were adjusted accordingly. The Determine, RapidCheck, UniGold and Capillus RT did not have any discordant results between initial RT and quality control testing.

Cumulative rapid test evaluation

The RT performance against the subtype panel did not serve to discriminate performance as all seven RT detected all four subtypes. Discrimination of assay performance could be seen on the basis of the other four criteria.

Clinical sensitivity criteria divided the seven RT into two groups: the Determine, RapidCheck, UniGold, Stat Pak and Capillus had 100% clinical sensitivity, whereas Hexagon and HemaStrip assays performed less than the threshold of 99.5%.

The Analytical Sensitivity ranking is important because it can help to choose a better assay for screening purposes. Of note is the fact that the Determine and RapidCheck assays performed equally well to the two gold standard EIA.

All RT assays met the study criteria for clinical specificity of 99.0%, three RT assays had a 100% clinical specificity; six had a clinical specificity above 99.5%. The 100% clinical specificity observed for the HemaStrip assay may offset its low sensitivity (97.74%).

The Determine, RapidCheck, UniGold, Stat Pak and HemaStrip had a maximal operational performance score. The Hexagon and Capillus assays failed in this criterion as they scored less than our cut-off value of four points.

Discussion

This study is the first attempt to evaluate systematically the commercially available RT in Brazil and compare their performance with the gold standard testing used in the national algorithm for HIV diagnosis. Five parameters were considered in the evaluation of seven RT: clinical sensitivity, clinical specificity, analytical sensitivity, performance against an HIV-1 subtype panel, and the operational performance of the assay.

All seven RT performed well by these parameters but four RT were notably better: Determine, UniGold, RapidCheck, and Stat Pak. The data for the clinical sensitivity and specificity are consistent with the results from other studies [2,3,7]. However, we should be aware that RT batch-to-batch variations can affect sensitivity and specificity.

At least two RT (Determine and RapidCheck) performed equally to EIA when compared against seroconversion panels. Another two RT (UniGold and Stat Pak) missed only one sample (which would result in a 2-day difference in the detection of anti-HIV antibodies) from one seroconversion panel when compared with EIA. These data provide evidence that the RT is a good alternative to the use of EIA as a screening test, as has been suggested by others [3–5,9].

All RT and the two EIA detected all samples of our HIV-1 subtype panel (B, B′, C, F). As the B′, C and F subtypes are the most important in Brazil [10], we expect that all seven RT would perform well in the country.

In terms of ease of performance, the Capillus and Hexagon received lower scores than others RT as they require more than one reagent that should be kept between 2 and 8°C, involved more than four steps to complete, and required a laboratory skill level on the part of the operator. Although they are rapid assays (performance time is less than 10 min) we found that they were not easy assays to perform, especially outside a laboratory environment. The other five RT assays had a good operational performance with performance time ranging from 10: 30 to 16: 30 (min: s). The characteristics of these five RT make them an excellent option for rapid HIV diagnosis in situations with limited time, laboratory conditions and personnel.

Four RT (Determine, RapidCheck, UniGold and Stat Pak) performed highest according to all evaluation criteria, and will be used to construct an algorithm for the rapid diagnosis of HIV infection in Brazil. This RT diagnostic will subsequently be evaluated to determine its programmatic value.

Our results show that any pair among the four RT would give an accurate test result. On the basis of the analytical sensitivity, Determine and RapidCheck would have the same efficacy as the EIA for the purpose of screening. A discordant result could be correctly resolved by applying a third, tiebreaker test, overcoming the need for a Western blot confirmation. In our case, the UniGold would be an ideal assay for this purpose as it has a clinical specificity of 100%.

The widespread use of two assays for a confirmatory strategy should be regarded with caution as some pairs of assays may be susceptible to the same non-specific effects and thus could generate a false-positive result. We advise that before implementing this strategy additional evaluations on the specificity of a particular pair of assays should be performed using a large number of HIV antibody-negative samples.

In conclusion, we have developed a methodology for the evaluation of RT, and have documented that the RT can provide highly accurate HIV test results that equal the EIA in sensitivity and specificity. The advantage of using an algorithm with RT is its simplicity and rapidity when compared with the algorithms that utilize EIA and Western blot for confirmation. These characteristics can be useful when addressing some high-risk populations in which only one opportunity for counselling and providing test results is possible. Furthermore, the adoption of an appropriate strategy could expand the availability and acceptability of HIV testing and counselling, which may increase the number of individuals reached for encouraging the adoption of risk-reducing behaviour.

Acknowledgements

This study was partially supported by the Centers for Disease Control and Prevention, Global AIDS Program, Atlanta, Georgia, USA and by the Brazilian STD/Aids Programme, Brasilia, Brazil.

The HIV-1 subtype panel was kindly provided to us by Dr. Amilcar Tanuri from the Federal University of Rio de Janeiro. We also want to thank Dr. Mauro Niskier Sanchez from the Brazilian STD/Aids Programme for his collaboration in the early phase of this project.

Finally, our thanks to William Brady, Peter Crippen and Suzanne Westman, from the Centers for Disease Control and Prevention/Global AIDS Program, for their constant support and encouragement.

References

1. Constantine N, Fox TE, Abbatte EA, Woody JN. Diagnostic usefulness of five screening assays for HIV in an east African city where prevalence of infection is low. AIDS 1989; 3:313–317.
2. Branson BM. Rapid tests for HIV antibody. AIDS Rev 2000; 2:76–83.
3. WHO/UNAIDS. Operational characteristics of commercially available assays to determine antibodies to HIV-1 and/or HIV-2 in human sera. Geneva, Switzerland: World Health Organization/the Joint United Nations Programme on HIV/AIDS, Report 11. WHO/BTS/99.1;UNAIDS/99.5; 1999.
4. Wilkinson D, Wilkinson N, Lombard C, Des M, Alan S, Floyd K, et al. On-site HIV testing in resource-poor settings: is one rapid test enough? AIDS 1997; 11:377–381.
5. Ketema F, Zeh C, Edeleman DC, Saville R, Constantine NT. Assessment of the performance of a rapid, lateral flow assay for the detection of antibodies to HIV. J Acquir Immune Defic Syndr 2001; 27:63–70.
6. Kline RL, Dada A, Blattner W, Quinn TC. Diagnosis and differentiation of HIV-1 and HIV-2 infection by two rapid assays in Nigeria. J Acquir Immune Defic Syndr 1994; 7:623–626.
7. Stetler HC, Granade TC, Nuñez CA, Meza R, Terrell S, Amador L, et al. Field evaluation of rapid HIV serological tests for screening and confirming HIV-1 infection in Hounduras. AIDS 1997; 11:369–375.
8. Irwin K, Olivo N, Schable CA, Weber JT, Janssen R, Ernst J. Performance characteristics of a rapid HIV antibody assay in a hospital with a high prevalence of HIV infection. Ann Intern Med 1996; 125:471–475.
9. Respess RA, Rayfield MA, Dondero TJ. Laboratory testing and rapid HIV assays: applications for HIV surveillance in hard-to-reach populations. AIDS 2001; 15(Suppl. 3):S49–S59.

Appendix

Other members of the HIV Rapid Test Study Group are: Tomoko Sasazawa Ito, Rosamaria Megias Ligmanosvski Kuss, and Sara Ferraz Vianti from the Public Health Laboratory of the city of Curitiba; Maria Rita C.B. Almeida from the VCT; Luiz Carlos Beira, Gefersson Alexandre Fernandes de Freitas, Danielle Fontoura Teixeira, Márcia Maria Fantinatti Guerra, Luciana Kusman, Maria Terumi Kami, Silvana Ribeiro Pienta, Paula Graciela Bochkariov, Ligia Fatima Simões, Michele Kessler, Patricia Regina Crozeta, Benedita Almeida dos Santos, Cristiane Ceccon de Souza Martinelli, Cristiane Yumiko Osawa, Solange Dalazoana, Tania Mary Medeiros Karvat, Grizeldi Colla, and Dulce Meri Blitzkow from the Antenatal Clinics.

10. Brindeiro RM, Diaz RS, Sabino EC, Morgado MG, Pires IL, Brigido L, et al. Brazilian Network for HIV Drug Resistance Surveillance (HIV-BResNet): a survey of chronically infected individuals. AIDS 2003; 17:1063–1069.
Keywords:

evaluation of rapid tests; HIV rapid test; rapid serological assays; voluntary and counselling testing and antenatal care; whole blood HIV testing

© 2005 Lippincott Williams & Wilkins, Inc.