HIV genetic diversity tends to hinder the development of genomic amplification techniques, particularly for plasma viral load assays. Commercial kits used to require manual RNA extraction, which was time-consuming and carried a risk of contamination. Abbott Molecular (Rungis, France) and Roche Diagnostics (Meylan, France) have recently developed automated extraction devices coupled to real-time amplification systems for HIV-1 quantification. Before being widely adopted, these new extraction-quantification systems must be tested for their sensitivity and reproducibility on sample panels representing the genotypic diversity of locally circulating strains. We therefore conducted a comparative study of these 2 techniques by testing them against culture supernatants and plasma samples representing the diversity of HIV types, groups, and subtypes.
METHODS
Materials
Table 1 shows the main characteristics of the tests. These automated extraction systems are based on the same principle. The sample undergoes a lysis step, followed by nucleic acid fixation to magnetic beads. Purification is ensured by a series of washing steps in which the beads are captured by a magnet. Finally, the beads are immersed in hot elution buffer to release the nucleic acid. An internal control extracted at the same time as the sample is used to confirm that the reaction is proceeding correctly. The Cobas AmpliPrep extractor (Roche Diagnostics) used for this study automatically distributes the reaction mixture and RNA extract in a multiwell plate. With the Abbott m1000 system, the reaction mix and extract were distributed manually, as recommended by the manufacturer. (The latest version of the m1000 system automatically distributes the reaction mix.)
Amplification is based on real-time polymerase chain reaction (PCR) in both systems, which require 3 controls for each series of 24 samples. The Abbott RealTime system also requires a 6-point calibration step at each change of reagent batch. The results for samples containing HIV-1 group M are expressed in IU/mL, after applying conversion coefficients provided by the manufacturers. The results for HIV-1 group O are expressed in copies/mL because of the lack of an international standard.
Subjects
The study involved 3 sample panels:
Different HIV-1 group M subtypes in 29 culture supernatants reconstituted in HIV-negative human plasma at a dilution of 10-5 and aliquoted and stored at -80°C until extraction: The plasma used for these dilutions came from a blood donor, with the donor's permission. The viral strains in these supernatants had previously been identified by nucleotide sequencing. In case of a conflict, a new aliquot was used to check the result and the Cobas Amplicor HIV-1 Monitor v1.5 (Roche Diagnostics) method was also implemented.
Eight supernatants previously identified as belonging to HIV-1 group O and 7 supernatants corresponding to HIV-2 group A or B: these supernatants were diluted at 10-5 for HIV-1 group O and at 10-3 for HIV-2 and were stored as previously described for HIV-1 group M. The results were compared with those of the HIV-1 group O and HIV-2 PCR reference methods.1,2
Eighty-eight human plasma samples: these samples, aliquoted and stored at -80°C, were provided by the ANRS Observatory of HIV Diversity and had been routinely quantified with the Roche Cobas HIV-1 Monitor v1.5 method. Nucleotide sequencing was done on a minimum of the polymerase gene.
Statistical Analysis
Data on the 3 techniques are expressed as mean (standard deviation), median (interquartiles Q1 and Q3), and range.
Comparison of the Techniques
The Wilcoxon matched-pairs test was used to compare the distribution within the Abbott RealTime and Roche Diagnostics Cobas Taqman techniques in the group M supernatants (n = 28). The mean values of the Roche Cobas Amplicor Monitor v1.5, Abbott RealTime and Roche Cobas TaqMan techniques in the 88 patients' samples were compared using 2-factor analysis of variance (method × subject). If the test result was significant, the techniques were then compared by pairs using the total residual variance.
Agreement Between 2 Techniques
Bland and Altman curves were used to represent the degree of agreement between 2 techniques (Fig. 1). The x axis bore the mean values for each sample obtained by the 2 techniques, and the y axis bore the difference between the values obtained by the 2 techniques. Two methods were used to judge the concordance between 2 techniques, depending on the Bland and Altman curves.
If the individual values of 2 techniques were related to their differences (eg, a larger difference between the techniques for samples with low values), the coefficient of variation (CV) was used.3
If there was no such relation, the intraclass correlation coefficient was calculated from the 2-factor analysis of variance.4
We considered that the agreement between the 2 techniques was acceptable if the CV was <10% or if the intraclass correlation coefficient was >90%.
These calculations were applied to the culture supernatants and patients' samples. A comparison of the differences between the techniques was conducted according to the HIV subtype.
Four subtype classes were created according to their number, their genetic relatedness, and their epidemiologic representativeness: B-D (n = 7), A-CRF01 (n = 16), CRF02 (n = 49), and other subtypes plus circulating recombinant form (CRF [n = 16]: G [n = 4]; C [n = 4]; CRF complex and nontypeable [n = 4]; and D distant, F, CRF11, and CRF18 [n = 1 each]). The Kruskal-Wallis test was used to compare the differences between Abbott RealTime and Roche Cobas TaqMan values according to the HIV subtype. In case of a significant global difference (P < 0.05), the subtypes were compared 2 by 2 with the Wilcoxon 2-sample rank sum test. The Bonferroni method for multiple comparisons (Bonferroni-type α-adjustments) was used with P < 0.008 (0.05/6). These calculations were applied to the panel of clinical plasma samples.
RESULTS
HIV-1 Group M Supernatants
The Abbott RealTime system detected all 29 supernatants diluted 10-5. Two supernatants were not detected by the Roche Cobas TaqMan system. One was a subtype G sample quantified at 4.9 log by the Abbott RealTime system. This supernatant was also tested with the Roche Cobas Amplicor HIV-1 Monitor v1.5, which gave a value of 3 log. This supernatant was retested at a concentration 100 times stronger (10-3 dilution) in the Roche Cobas TaqMan system but was still not detected. It was therefore excluded from the statistical analysis. The second supernatant, a subtype H quantified at 4.3 log by the Abbott RealTime system and at 2.5 log by the Roche Cobas HIV-1 Monitor v1.5, was also undetectable with the Roche Cobas TaqMan method. It was detected at a dilution of 10-4, representing an underestimation of 1.4 log relative to the Roche Cobas HIV-1 Monitor v1.5.
The distributions were significantly different between the 2 techniques (P < 0.001), with the values being higher with the Abbott RealTime system than with Roche Cobas TaqMan system (Table 2). On average, the Roche Cobas TaqMan system gave values 0.51 log lower than those obtained with the Abbott RealTime system. The supernatant underestimated by the Roche Cobas TaqMan system is represented at the top left of the graph (see Fig. 1A). The CV was 10%, indicating acceptable concordance between the 2 techniques.
HIV-1 Group O and HIV-2 Supernatants
None of the 8 group O supernatants was positive using the Roche Cobas TaqMan method. The Abbott RealTime system quantified 7 of these supernatants diluted 10-5. Of note, Roche Diagnostics restricts the indications for the Cobas Taqman system exclusively to HIV-1 group M. The results were compared with those of our in-house technique, which quantified all 8 supernatants. Viral loads of 6 supernatants were higher using the Abbott RealTime system (by 0.2-0.9 log). One supernatant gave a value of 3.8 log with the in-house method and 2.5 log with the Abbott RealTime system. The supernatant not detected by the Abbott RealTime system gave a value of 3.4 log with the in-house method. It was detected by the Abbott RealTime system at a dilution of 10-2 but was underestimated by 2.1 log compared with the in-house technique.
None of the 7 HIV-2 supernatants was positive using the Abbott RealTime or Roche Cobas TaqMan system, which is logical considering the genetic distance of HIV-2 from HIV-1.
Plasma Samples
The Roche Cobas TaqMan and Abbott RealTime systems were compared with the Roche Cobas HIV-1 Monitor v1.5 method. Plasma samples from 88 different patients were tested. The mean values were significantly different when the different techniques were compared 2 by 2 (P < 0.001). The mean value obtained with the Abbott RealTime system was higher than that obtained with the Roche Cobas HIV-1 Monitor v1.5 method and even higher than that obtained with the Roche Cobas TaqMan system (Table 3A).
Comparison Between Roche Cobas TaqMan and Abbott RealTime Systems
Thirty-five plasma samples (40%) were underestimated by at least 0.5 log using the Roche Cobas TaqMan method relative to the Abbott RealTime system. Four samples showed a difference of more than 1 log (1.2, 1.3, 1.3, and 2.4 log), with these patients being infected with CRF02 strains (n = 3) or subtype A (see Fig. 1B). Conversely, 3 plasma samples were underestimated by the Abbott RealTime system, with a difference of more than 0.5 log (0.5, 0.6, and 0.7 log; 2 subtype A and 1 subtype CRF02). The intraclass correlation coefficient between the 2 techniques was 79%. With the 7 subtype B and D plasma samples, viral load values were similar in the 3 systems. As shown in Table 3B, the median of the differences was far lower for subtype B than for the other subtypes. The differences between the techniques were significantly smaller for subtypes B and D than for the CRF02 subtype (P < 0.008).
Comparison Between Abbott RealTime and Roche Cobas HIV-1 Monitor v1.5 Systems
Values obtained with the reference method (Roche Cobas HIV-1 Monitor v1.5), were 0.15 log lower, on average, than the Abbott RealTime system values but were similar in terms of the distribution around the mean of the difference (see Fig. 1C): only 15 patients had more than a 0.5-log difference between 2 viral load measures, with only 3 having a difference of more than 1 log. The major discrepancy of 2.4 log was not found between the Roche Cobas HIV-1 Monitor v1.5 (5.2 log) system and the Abbott RealTime (5.5 log) system. The intraclass correlation coefficient between the 2 techniques was 90%.
Comparison Between Roche Cobas TaqMan and Roche Cobas HIV-1 Monitor v1.5 Systems
Viral loads obtained with the Roche Cobas TaqMan system were, on average, 0.28 log lower than those obtained with Roche Cobas HIV-1 Monitor v1.5 system: 23 patients had more than a 0.5-log difference in the 2 methods, with 9 having a difference exceeding 1 log (see Fig. 1D). The intraclass correlation coefficient between the 2 techniques was 83%.
DISCUSSION
The genomic variability of HIV hinders the development of universal primers and probes for genomic hybridization. This holds for all HIV strains, which form a quasispecies.5 The genomes available in sequence banks for primer and probe selection are biased toward subtype B, with non-B strains representing a minority. Even within subtype B, a simple synonymous mutation may lead to detection failure. The risk of mutations at primer and probe target sites is higher for other subtypes.
Importance of Expressing Viral Load in International Units
To be able to compare the results of the different techniques, we expressed our results in international units, using conversion factors provided by each manufacturer. The Roche Cobas TaqMan and Abbott RealTime systems were calibrated with World Health Organization (WHO) international standard 97/656. The Roche Cobas HIV-1 Monitor v1.5 method was developed before this WHO standard, and its conversion factor has not been updated; calculations were therefore based on the coefficient proposed in the user manual. The impact of these conversion factors on the results is small, because the mean discrepancy between the 2 Roche Diagnostics techniques increased only from 0.22 log when the results were expressed in copies/mL to 0.29 log when the results were expressed in IU/mL.
Sophisticated Extraction-Amplification Systems
We did not separately analyze the quality of the amplification and extraction systems. The evaluation of these extraction systems is therefore based on their simplicity, openness, and reliability. The 2 systems seemed to be acceptable. The Abbott m1000 system is based on a robust sample distribution technology adapted to molecular biology uses by Tecan (Palm Springs, CA), and it can be used to extract DNA and RNA from a variety of samples. The m2000 reverse transcriptase amplifier is manufactured by Applied Biosystems (Foster City, CA) but is not yet available as an open system because of software limitations. Conversely, the Roche Cobas AmpliPrep system can only be used to extract nucleic acids from plasma. It is specifically designed to be part of a fully automated procedure for high-throughput laboratories, with the possibility of automatic transfer of the plates from the extractor to the amplifier. Neither system is technically superior to the other, and both methods have their respective advantages and disadvantages, chief among which are the need for multiple reagents, volume, and high purchase and maintenance costs. These extractors are therefore used mainly by teams doing large numbers of tests. To explain the discrepancies observed between the Abbott RealTime and Roche Cobas TaqMan systems, a crossed study between the products of extraction and PCR of the 2 assays would be of interest. Unfortunately, an independent analysis of the performance of the extraction is not practicable. The internal control added during the extraction procedure of the Roche Cobas TaqMan system cannot be recognized during the amplification procedure of the Abbott RealTime system, impairing the viral load determination by the software. Moreover, the recovery of the product extracted by the Roche Cobas AmpliPrep system is limited because of the mix of the different reagents by the system itself before the end of the extraction cycle.
Difference in Quantification Between HIV-1 Group M Supernatants
In absolute terms, the Roche Cobas TaqMan system was less sensitive than the Abbott RealTime system when tested on supernatants of different types, groups, and subtypes. On average, viral load values obtained with the Roche Cobas TaqMan system were 0.51 log lower than those obtained with the Abbott RealTime system, but the CV (10%) indicated a good agreement between the 2 systems. The 2 tests gave similar results when we excluded a supernatant that was not detected by the Roche Cobas TaqMan system. Of the 3 subtype G supernatants in our panel, 1 was not detected and 2 were underestimated (by 0.6 and 0.9 log) by the Roche Cobas TaqMan system relative to the Roche Cobas HIV-1 Monitor v1.5 method. One subtype H strain was also strongly underestimated by the Roche Cobas TaqMan system. Our panels did not contain enough subtype G through H strains to conclude that these subtypes are systematically underestimated by the Roche Cobas TaqMan system.
Quantification of HIV-1 Group O by Abbott RealTime System
The HIV-1group O supernatants could be only quantified by the Abbott RealTime system. The lack of an international standard for HIV-1 group O may explain the difference relative to our in-house method. For 6 of the 7 samples, the difference with the in-house method was <1 log, which can be considered acceptable given the major genetic variability of group O. Paradoxically, the efficient detection of HIV-1 group O by the Abbott tests may lead to less attention being paid to infections with this variant. HIV-1 group O strains used to be suspected (usually too late) in subjects with paradoxic viral load and CD4 cell count values. The use of the Abbott RealTime system therefore raises the need for a systematic grouping of strains from patients originating from areas that are endemic for HIV-1 group O, given the natural resistance of group O strains to nonnucleoside reverse transcriptase inhibitors (NNRTIs).6
Plasma Viral Load Correlation Between Abbott RealTime, Roche Cobas TaqMan, and Roche Cobas HIV-1 Monitor v1.5 Systems
The underestimation by the Roche Cobas TaqMan system compared with the Abbott RealTime system was also found in the study of 88 plasma samples, with an intraclass correlation coefficient of only 79%. The difference was usually <1 log, but 4 plasma samples, including 3 CRF02 strains, were underestimated by more than 1 log. It is not possible to conclude that there is a specific problem with CRF02 strains, however. Indeed, the underestimation may be punctual, as shown with our panel of well-identified CRF02 strains. Observed differences reflect the frequency of a given subtype as much as its variability, because the risk of encountering a mutation increases strongly with the prevalence of the subtype and with its genetic distance from the prototype strains used to develop a particular quantitative assay method.
The reference method (Roche Cobas HIV-1 Monitor v1.5) correlated better with the Abbott RealTime system than with the Roche Cobas TaqMan system (intraclass correlation coefficients of 90% and 83%). This seems paradoxic, but it confirms the results obtained with the supernatant panel: the Roche Cobas HIV-1 Monitor v1.5 method quantified non-B HIV-1 subtypes better than the Roche Cobas TaqMan technique. Conversely, the Roche Cobas Amplicor v1.5, Roche Cobas Taqman, and Abbott RealTime systems gave similar results for B and D plasma samples, indicating that the primers and probes are suitable for these strains. In contrast, the large differences observed with other subtypes, particularly CRF02, underline the importance of the choice of primers and probes. These data, and the lack of detection of G and H supernatants, indicate that it is indeed the genetic diversity of HIV strains, combined with real-time PCR amplification conditions, that is mainly responsible for nondetection or underestimation. The primers and probes of the Abbott RealTime system, chosen in the highly conserved region of the integrase region of the pol gene, amplified all HIV-1 strains, including distant strains belonging to HIV-1 group O.
Complex Choice of an Automated Extraction-Amplification System
The choice of a viral load assay depends on the overall qualities of available systems. Sensitivity for the different HIV subtypes is a major criterion but is not the only consideration. Our study has certain limitations, such as the reconstitution of viral supernatants with seronegative plasma, which might have influenced the efficiency of the extraction and amplification reactions. In addition, we used plasma samples that had been stored frozen for several weeks. Finally, we did not examine the possible variability of reagent batches. Thus, our results offer only a snapshot of the situation. The quality of successive reagent batches, use in real-life situations, the reliability of the extraction systems, the cost of consumables, and the laboratory's specific needs are all important factors in the decision-making process. Finally, manufacturers are continually improving their products in response to health agency guidelines, as illustrated by the modified Roche Cobas HIV-1 Monitor system.7
Our results nonetheless provide useful information on the limitations of real-time PCR systems for non-B HIV subtypes. The diversity of HIV can at least be partially overcome, as shown by the choice of primers and probes capable of detecting groups M and O. Independent monitoring is needed to improve the sensitivity of these tests and to ensure reliable treatment monitoring for the emergence of resistant strains. Our results also underline the importance of harmonizing viral load assays and data expression in therapeutic trials.
ACKNOWLEDGMENTS
Abbott Molecular and Roche Diagnostics provided all the reagents and lent us the automatons. Sylvie Lambert, Jean Robert Mountou, and Jean Michel Dupré performed the study. The Etablissement Français du Sang Normandie provided HIV-negative human plasma.
REFERENCES
© 2007 Lippincott Williams & Wilkins, Inc.