Mushanski, Linda M. BS*; Brandt, Ken BS*; Coffin, Nicolette MSc, MBA*; Levett, Paul N. PhD*; Horsman, Gregory B. MD*; Rank, Elliot L. PhD†
From the *Saskatchewan Disease Control Laboratory, Regina, Saskatchewan, Canada; †BD Diagnostics, Sparks, MD
The authors acknowledge BD Diagnostics—Diagnostic Systems for the donation of the BD reagents and test materials used during the course of this study.
Any views expressed in this publication are the views of the authors and do not necessarily represent the views of the Government of Saskatchewan.
Current address: Elliot L. Rank, PhD, (D)ABMM, Director of Molecular Clinical Infectious Diseases, St. Luke's-Roosevelt Hospital Centers, 1111 Amsterdam Avenue, New York, NY 10025.
Correspondence: Paul N. Levett, PhD, (D)ABMM, FCCM, Saskatchewan Disease Control Laboratory, 5 Research Drive, Regina, Saskatchewan S4S 0A4, Canada. E-mail: firstname.lastname@example.org.
Received for publication August 8, 2011, and accepted February 7, 2012.
Background: Performances of the BD ProbeTec Chlamydia trachomatis (CT)/Neisseria Gonorrhoeae (GC) Qx Amplified DNA Assay reagents on a BD Viper System with XTR Technology and APTIMA COMBO 2 Assay reagents on a TIGRIS DTS platform, for detection of both CT and GC were compared.
Methods: A total of 1018 first-void urine specimens were tested for the presence of CT and GC DNA using the 2 assays.
Results: CT was detected in 143 specimens (14%). Eight specimens exhibited discordant results, and they were divided equally between the 2 assays. Based on the original results, the overall agreement for CT was 99.2%, with 97.1% and 99.5% in agreement with positive and negative specimens, respectively. Cohen's Kappa was 0.967. GC was detected in 27 specimens (2.6%). Two specimens exhibited discordant results, and they were divided equally between the 2 assays. Based on the original results, the overall agreement was 99.8%, with 96.2% and 99.9% in agreement for positive and negative specimens, respectively. Cohen's Kappa was 0.961.
Conclusions: There was a high level of agreement between the systems for both CT and GC detection.
Automated nucleic acid amplification tests (NAATs) for the detection of Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (GC) have become widely used during the past 2 decades for diagnosis of these common sexually transmitted infections.1 NAATs have been shown to offer greater sensitivity and at least equal specificity to conventional methods and, as a result, have largely replaced culture as the standard for diagnostic testing.2 Numerous specimen types have been validated for use in NAAT testing.3–6 The availability of highly sensitive NAATs that can be applied to urine7 and other minimally invasive specimens8,9 has led to an increase in testing. The reported rate for chlamydial infection per 100,000 individuals has risen steadily during the past decade (1999 to 2009) from 247.2 cases to 409, whereas the rate for gonococcal infection has progressively decreased during the same period.10–12
The BD Viper System with XTR Technology (BD Diagnostics, Sparks, MD) is a third-generation platform that, when operating in extracted mode, provides automated DNA extraction using ferric oxide (FOX) and strand displacement amplification.13
The Gen-Probe TIGRIS DTS System (Gen-Probe, Inc., San Diego, CA) is a totally integrated nucleic acid amplification testing system that fully automates all steps necessary to perform the APTIMA transcription-mediated amplification assays from sample processing through amplification, detection, and data reduction.1,14
The purpose of this evaluation was to compare the performance of the 2 systems for the detection of CT and GC from urine specimens collected from male and female patients.
Specimen Collection, Transport, and Storage
This evaluation was conducted from July to November 2010. First-void urine specimens were collected in sterile urine containers from male and female patients attending 4 sexual health clinics in the Province of Saskatchewan. Immediately after collection, the urine specimen was dispensed into 2 proprietary collection devices: a urine preservative tube (UPT) for testing with the BD system, and the urine specimen transport tube (USTT, volume 2 mL) for testing with the Gen-Probe system. Tube caps were removed aseptically, and the appropriate volume of urine was transferred by sterile plastic pipette to the UPT (2–3 mL) and the USTT (2 mL). Urine specimens in BD and Gen-Probe transport tubes were labeled and transported to the Saskatchewan Disease Control Laboratory (SDCL). Specimens were received in the laboratory within 24 to 120 hours after collection and kept at room temperature before testing.
Urine specimens in USTT were tested with the Gen-Probe system within 24 hours of arrival. On the BD system, the UPT urine samples were accessioned and subjected to the prewarming step (114°C for 15 minutes.) on a daily basis and then stored at room temperature before testing; batches were tested at least once per week to reduce reagent waste. Specimens were tested on the 2 platforms, with strict adherence to the manufacturers' instructions.
The algorithm in use by the SDCL required retesting of all specimens initially positive in the Gen-Probe system for GC, all specimens positive for CT with a relative light unit value of <900 units, and all positive specimens with medico-legal implications, using the Gen-Probe system supplementary assays, which use different targets. Any samples that were retested with either the Gen-Probe CT or GC supplementary assay were also repeated with the BD system.
All discordant samples were retested on both platforms; in the case of positive Gen-Probe specimens, the supplementary assay using a different target was run, whereas on the BD system, samples were retested using the same assay.
A total of 1023 urine specimens were submitted for testing, which were obtained from 503 females and 520 males. Five of those specimens (4 female and 1 male) were of insufficient volume for parallel testing with the BD system and were excluded from the study. Of the 1018 specimens analyzed, 942 (92.5%) specimens were tested on the Gen-Probe system on the day of receipt into the laboratory, 75 (7.4%) were tested on the next day, and one specimen (0.1%) was tested 3 days after receipt in the laboratory. As batch sizing allowed, specimens were tested on the BD system a median of 4 days after receipt in the laboratory (range: 1–6 days).
Table 1 shows results for 1018 urine specimens tested on both systems for the presence of CT and GC. Using the Gen-Probe assay, 139 specimens were positive for CT (13.7% prevalence) and 26 were positive for GC (2.6% prevalence). Using the BD assay, 139 specimens were also positive for CT (13.7% prevalence) and 26 were also positive for GC (2.6% prevalence). Eleven specimens were positive for both CT and GC in both systems. Detection rates from male and female urine specimens were not significantly different.
There were concordant CT-positive results for 135 specimens (13.3%) and concordant negative results for 875 specimens (86.0%). There were 8 specimens (0.8%) that were discordant for CT, split evenly between the 2 systems: 4 specimens were positive for CT with the Gen-Probe system but negative for CT with the BD system, and 4 were positive for CT with the BD system but negative for CT by the Gen-Probe system (Table 2).
Table 2-a. Discordan...Image Tools
Of the 139 CT-positive specimens observed with the Gen-Probe system, 30 were subjected to the alternate target supplementary test, with identical findings. Discordant results for 4 specimens (A, B, D, E; Table 2) were observed with the BD system, and all 4 remained negative with repeat testing. Of the 139 CT-positive specimens observed with the BD system, 4 (C, F, G, and H) were negative with the Gen-Probe system. Three of the 4 specimens (C, G, and H) were not confirmed with repeat testing on the BD system or with the supplementary Gen-Probe assay. One specimen (F) was confirmed with repeat testing in the BD system but remained negative on the Gen-Probe system (Table 2).
Table 2-b. Discordan...Image Tools
There were concordant GC-positive results for 25 specimens (2.5%), and concordant negative results for 991 (97.3%) specimens. Two specimens (0.2%) were discordant for GC and were evenly split between the 2 systems: one specimen (I) was positive for GC with the initial and supplementary Gen-Probe tests but negative with the initial and repeat BD tests, and the other specimen (J) was positive with one of the 2 BD tests but negative with both Gen-Probe tests (Table 2). Using the initial results, the overall agreement for positive specimens was 97.1% (135/139 specimens) for CT and 96.2% (25/26 specimens) for GC. Overall agreement for negative specimens was 99.5% for CT and 99.9% for GC. Cohen's Kappa coefficient was 0.967 for CT and 0.961 for GC.
In this study, the performance of the BD Viper system was compared with the performance of the Gen-Probe TIGRIS system. This study compared urine specimens on both systems. Urine is recommended as the preferred specimen type for men7 and is an acceptable specimen type for women.5 Urine represents an ideal specimen for evaluations because it is a noninvasive specimen. There is usually residual urine available after initial testing. Target nucleic acids in urine are also readily stabilized by the transport tubes supplied by each manufacturer.
There were 1018 urine specimens from which were generated a total of 2036 CT and GC test results. There were 10 discordant specimens: 8 discordant specimens for CT and 2 discordant specimens for GC. Discordant results were equally split between the 2 systems. Thus, there were 0.4% discordant CT results and 0.1% discordant GC results. All discordant results were retested using a Gen-Probe second target assay and a repeat of the BD assay. Resolution of discrepant results would ideally be performed by collecting additional specimens from the patients; however, this could not be done by this study design using specimens from patients visiting sexual health clinics. We believe retesting all discordant results with a third test method (i.e., Gen-Probe test using an alternative target) is an acceptable method to resolve discrepancies. Of the 10 discrepant results, 9 of the Gen-Probe tests were confirmed by the alternative Gen-Probe test. It is recognized that use of the same nucleic acid amplification technique for the initial and supplementary Gen-Probe assays can introduce a bias in the data analysis; therefore, it is difficult to classify these as true-positive or true-negative results. An alternative approach to resolving discrepancies would be to repeat the initial tests as was done with the BD tests. Of the 10 discrepant results, only one specimen (F) repeated as a positive result. Again, it is difficult to classify the other 9 specimens as false-positive or false-negative because this may represent statistical variation at the threshold for detection of positive specimens. This can only be resolved by collecting additional specimens from the patients.
In 2002, the Centers for Disease Control and Prevention recommended 4 options for supplementary or confirmation testing.15 The shortcomings of these recommendations have been previously discussed.16,17 Some clinical investigators have opted for alternate-site confirmation testing7,18 or repeat testing.19–21 The Association of Public Health Laboratories and Centers for Disease Control and Prevention Expert Consultation Meeting recently concluded that routine repeat testing of NAAT-positive screening specimens is not recommended (http://www.aphl.org/aphlprograms/infectious/std/Documents/CTGCLabGuidelinesMeetingReport.pdf). However, the Public Health Agency of Canada continues to recommend either confirmatory testing using a different target or repeat testing in a different laboratory for medico-legal cases.22
There were several limitations of this study design. Because the SDCL tests urine specimens almost exclusively,1 the study was designed to address this specimen type only. In addition, the patients tested in the sexual health clinics represent a high-risk prevalence population. Further evaluations are necessary in lower-risk populations.
This was an investigational study in which the Gen-Probe system was used to test patient specimens and to report results, and the tests on the BD system were run by the same staff after the tests were run on the Gen-Probe system, with a median delay of 4 days. Because of the delay in testing, specimens received in the BD UPT were heated on the day of arrival and stored at room temperature until tested. This delay in testing on the BD system was unavoidable, but urine is stable in the BD UPT for up to 30 days at 2–30°C.
The BD Viper system runs batches of up to 92 specimens (184 CT and GC results) using prepackaged consumables, whereas the Gen-Probe Tigris platform has higher total throughput and more flexible batch sizes. Because this was not a direct, side-by-side comparison study, we did not analyze specimen throughput in the 2 systems, although both platforms are capable of high-volume testing.
Based on the study of 1018 urine specimens, the 2 systems have similar performance. Selection of a NAAT platform for detection of CT and GC in clinical microbiology laboratories should be made after consideration of daily specimen throughput, the ease of use, potential efficiency gains in the laboratory, the environmental footprint, and the test costs.
1. Levett PN, Brandt K, Olenius K, et al.. Evaluation of three automated nucleic acid amplification systems for detection of Chlamydia trachomatis and Neisseria gonorrhoeae in first-void urine specimens. J Clin Microbiol 2008; 46:2109–2111.
2. Centers for Disease Control and Prevention, Association of Public Health Laboratories. Volume and type of laboratory testing methods for STDs in public health laboratories 2007. Atlanta, GA. 2011. Available at: http://www.cdc.gov/std/general/LabSurveyReport-2011.pdf
. Accessed August 8, 2011.
3. Ota KV, Tamari IE, Smieja M, et al.. Detection of Neisseria gonorrhoeae and Chlamydia trachomatis in pharyngeal and rectal specimens using the BD Probetec ET system, the Gen-Probe Aptima Combo 2 assay and culture. Sex Transm Infect 2009; 85:182–186.
4. Cook RL, Hutchison SL, Ostergaard L, et al.. Systematic review: Noninvasive testing for Chlamydia trachomatis and Neisseria gonorrhoeae. Ann Intern Med 2005; 142:914–925.
5. van Dommelen L, Dukers-Muijrers N, van Tiel FH, et al.. Evaluation of one-sample testing of self-obtained vaginal swabs and first catch urine samples separately and in combination for the detection of Chlamydia trachomatis by two amplified DNA assays in women visiting a sexually transmitted disease clinic. Sex Transm Dis 2011; 38:533–535.
6. Chernesky M, Jang D, Portillo E, et al.. Abilities of APTIMA, AMPLICOR, and ProbeTec assays to detect Chlamydia trachomatis and Neisseria gonorrhoeae in PreservCyt ThinPrep liquid-based Pap samples. J Clin Microbiol 2007; 45:2355–2358.
7. Chernesky MA, Martin DH, Hook EW, et al.. Ability of new APTIMA CT and APTIMA GC assays to detect Chlamydia trachomatis and Neisseria gonorrhoeae in male urine and urethral swabs. J Clin Microbiol 2005; 43:127–131.
8. Schachter J, Chernesky MA, Willis DE, et al.. Vaginal swabs are the specimens of choice when screening for Chlamydia trachomatis and Neisseria gonorrhoeae: Results from a multicenter evaluation of the APTIMA assays for both infections. Sex Transm Dis 2005; 32:725–728.
9. Skidmore S, Horner P, Herring A, et al.. Vulvovaginal-swab or first-catch urine specimen to detect Chlamydia trachomatis in women in a community setting? J Clin Microbiol 2006; 44:4389–4394.
10. Centers for Disease Control and Prevention. Sexually transmitted disease surveillance 2009. Atlanta, GA: U.S. Department of Health and Human Services, 2010.
11. Jayaraman G, Totten S, Perrin M, et al.. Report on sexually-transmitted infections in Canada: 2008. Ottawa: Public Health Agency of Canada; 2010.
12. Burckhardt F, Warner P, Young H. What is the impact of change in diagnostic test method on surveillance data trends in Chlamydia trachomatis infection? Sex Transm Infect 2006; 82:24–30.
13. Taylor SN, Van Der Pol B, Lillis R, et al.. Clinical evaluation of the BD ProbeTec Chlamydia trachomatis Qx Amplified DNA Assay on the BD Viper system with XTR technology. Sex Transm Dis 2011; 38:603–609.
14. Chernesky MA, Jang DE. APTIMA transcription-mediated amplification assays for Chlamydia trachomatis and Neisseria gonorrhoeae. Expert Rev Mol Diagn 2006; 6:519–525.
15. Centers for Disease Control and Prevention. Screening tests to detect Chlamydia trachomatis and Neisseria gonorrhoeae infections—2002. MMWR Morb Mortal Wkly Rep 2002; 51 (RR15):1–38.
16. Schachter J, Chow JM, Howard H, et al.. Detection of Chlamydia trachomatis by nucleic acid amplification testing: Our evaluation suggests that CDC-recommended approaches for confirmatory testing are ill-advised. J Clin Microbiol 2006; 44:2512–2517.
17. Moncada J, Donegan E, Schachter J. Evaluation of CDC-recommended approaches for confirmatory testing of positive Neisseria gonorrhoeae nucleic acid amplification test results. J Clin Microbiol 2008; 46:1614–1619.
18. Munson E, Boyd V, Czarnecka J, et al.. Evaluation of Gen-Probe APTIMA-based Neisseria gonorrhoeae and Chlamydia trachomatis confirmatory testing in a metropolitan setting of high disease prevalence. J Clin Microbiol 2007; 45:2793–2797.
19. de Silva T, Kudesia G, Joall A, et al.. Significance of low positive scores obtained with a method other than acceleration in the BDProbeTec-Strand displacement amplification test for detection of Neisseria gonorrhoeae. J Clin Microbiol 2006; 44:4628–4629.
20. Hadgu A, Dendukuri N, Hilden J. Evaluation of nucleic acid amplification tests in the absence of a perfect gold-standard test: A review of the statistical and epidemiologic issues. Epidemiology 2005; 16:604–612.
21. Hadgu A, Sternberg M. Reproducibility and specificity concerns associated with nucleic acid amplification tests for detecting Chlamydia trachomatis. Eur J Clin Microbiol Infect Dis 2009; 28:9–15.