Background: Reverse sequence screening for syphilis, in which an automatable treponemal assay (enzyme immunoassay [EIA]/chemiluminescence assay [CIA]) is performed first and followed by a nontreponemal test for reactive specimens, has been used increasingly in the United States. The EIA is objective, efficient, and believed to be more sensitive than the rapid plasma reagin (RPR) because treponemal antibodies appear before nontreponemal antibodies. We sought to compare the sensitivity of a commonly used EIA, the Trep-Sure EIA (TS-EIA), to the RPR in cases of suspected primary syphilis infection in our clinic.
Methods: A retrospective medical record review of patients with sexually transmitted infection clinic visits from January 2009 to December 2011 was conducted, and 52 patients met the following inclusion criteria: suspected primary syphilis symptoms, at least 1 positive syphilis test result at visit, and no history of syphilis. Sensitivity analyses compared the TS-EIA and RPR, using the reference standard of concordantly positive/reactive TS-EIA/RPR or positive fluorescent treponemal antibody absorption test (FTA-ABS) result. We considered equivocal TS-EIA results to be positive for sensitivity calculations because such results typically reflex to additional testing and therefore may still result in identifying new infections.
Results: Twenty-eight (53.8%) of the 52 patients had a positive or equivocal TS-EIA. Twenty-five (89.3%) of those were RPR reactive; the remaining 3 (10.7%) were RPR nonreactive, FTA-ABS positive. Forty patients (76.9%) had a positive RPR, including 15 patients (37.5%) with negative TS-EIA results; all 15 were FTA-ABS positive. Nine additional patients were TS-EIA negative and RPR nonreactive but had a positive FTA-ABS result. The RPR was significantly more sensitive than the EIA (76.9% vs. 53.8%, P = 0.005). Trep-Sure EIA positivity was also significantly associated with higher median RPR titer (P = 0.011).
Conclusions: Use of the TS-EIA may result in underdetection of primary syphilis compared with the RPR. Further evaluation of the sensitivity of the TS-EIA in high-morbidity settings is warranted before the adoption of reverse sequence screening algorithms.
An evaluation of diagnostic syphilis serological results among patients with primary syphilis symptoms suggests that reverse sequence algorithms using the Trep-Sure enzyme immunoassay suffers from reduced sensitivity relative to the rapid plasma reagin.
From the *Howard Brown Health Center, Department of Research, Chicago, IL; †Ann & Robert H. Lurie Children’s Hospital of Chicago, Center for Gender, Sexuality and HIV Prevention, Chicago, IL; ‡Division of Community Health Sciences, School of Public Health, University of Illinois at Chicago, Chicago, IL; §Howard Brown Health Center, Department of HIV/STI Prevention, Chicago, IL; and ¶John H. Stroger Hospital, Division of Infectious Diseases, Chicago, IL
Financial support: No financial support was received for this study.
Conflicts of interest: None.
Correspondence: Beau Gratzer, MPP, Howard Brown Health Center, 4025 N Sheridan Rd, Chicago, IL 60613. E-mail: email@example.com.
Received for publication June 18, 2013, and accepted March 5, 2014.
High-volume laboratories in the United States increasingly use a reverse sequence algorithm to screen for syphilis.1–3 Reverse sequence algorithms use a treponemal test (e.g., enzyme immunoassay [EIA] or chemiluminescence assay [CIA]) for initial screening and reflex to a nontreponemal test (e.g., rapid plasma reagin [RPR]) for assessment of disease activity. If an RPR is discrepantly negative in a patient without known history of syphilis, the US Centers for Disease Control and Prevention (CDC) recommends performing the Treponema pallidum–passive particle agglutination assay for confirmation of infection.1,4,5 Although it is no longer recommended by CDC, many public health and commercial laboratories continue to use the fluorescent treponemal antibody absorption test [FTA-ABS] as the secondary treponemal test to resolve discrepant results.
Reverse sequencing has become popular because EIA/CIA tests are automatable, thereby increasing throughput, and are objectively read. Treponemal tests are reported to have superior sensitivity relative to nontreponemal tests in both early and latent syphilis infection.4,6–11 Furthermore, nontreponemal tests suffer from high cross-reactivity with a range of other infections and conditions that may cause false-positive results. Controversy exists regarding the use of reverse sequencing, however, because of difficulty in clinical interpretation of discrepant serological results and the abundance of presumed false-positive treponemal tests that require substantial follow-up efforts of surveillance and field staff.1,5,12
One of the most commonly used EIAs for reverse sequence screening is the Trep-Sure EIA (TS-EIA; Trinity Biotech, Jamestown, NY). The TS-EIA is a third-generation T. pallidum total antibody sandwich assay that detects treponemal antibodies to both immunoglobulins G and M using a proprietary recombinant antigen.2,13 The TS-EIA produces qualitative results via quantitative index values based on optical density (OD) signal to cutoff ratios. An OD index value greater than 1.2 is positive and less than 0.8 is negative. Optical density index values between 0.8 and 1.2 are equivocal and should trigger retesting and reflex to additional testing with other technologies; patients with consecutively equivocal results and negative/nonreactive reflex test results are recommended to retest in 2 to 4 weeks because this may be an indication of recent syphilis infection.2,11,13
In December 2007, the Illinois Department of Public Health (IDPH) State Laboratory conducted a validation of the TS-EIA in anticipation of transitioning to a reverse screening algorithm in early 2008. During the IDPH validation of the TS-EIA, a patient seeking testing at the “STI Walk-in Clinic” at Howard Brown Health Center (HBHC) with no known history of syphilis had an anomalous result: TS-EIA negative, RPR reactive (titer 1:2), and FTA-ABS reactive. A second patient, also with no known history of syphilis, presented for care in early 2008 with a penile lesion that had a reported duration of approximately 1 month. The initial serological result for the second patient was TS-EIA negative, RPR reactive (titer 1:8), and FTA-ABS reactive. This patient underwent a punch biopsy on the lesion during a follow-up visit that was positive for treponemes. A portion of the original serum specimen was sent to the manufacturer for further evaluation, and retesting also produced a negative TS-EIA result; the TS-EIA OD index value for this specimen was 0.215, which is well below the index value for an equivocal or positive result. The manufacturer performed 2 additional treponemal tests, the Inno Lia and the Trep ID, and both produced positive results. This led the manufacturer to conclude that this was likely a very early infection and that the TS-EIA was falsely negative. Several additional discrepant results were detected among suspected primary syphilis cases at our facility throughout the remainder of 2008. The medical staff of HBHC implemented a protocol in January 2009 to request both RPR and FTA-ABS testing for all STI Walk-in Clinic patients exhibiting symptoms suggestive of primary syphilis when the TS-EIA result was negative.
We conducted a medical record audit of patients with suspected primary syphilis infection presenting at the HBHC STI Walk-in Clinic between January 2009 and December 2011. Cases were included in the review if the patient had clinical signs and symptoms suggestive of primary syphilis (e.g., chancre or genital/perianal/oropharyngeal ulcer of unknown etiology), at least 1 positive/reactive syphilis serological test result, and no reported and/or known history of syphilis infection. Data examined for the analysis included the following: demographic (age, racial and ethnic identity, sex identity), HIV status, sex of sex partners, symptom location and duration, and syphilis serological results at the time of visit.
Serological Testing and Syphilis Case Definition
All syphilis serological testing was performed by the IDPH State Laboratory, which uses a reverse sequence algorithm beginning with the TS-EIA. Positive TS-EIA specimens were subsequently tested with the RPR; if the RPR was discrepantly nonreactive, an FTA-ABS was performed to confirm infection. Treponemal antibody absorption tests were not performed if the TS-EIA and RPR were concordantly positive/reactive. All patients with suspected primary syphilis symptoms and an initially negative TS-EIA also received RPR and FTA-ABS testing; in addition, all equivocal TS-EIA specimens were subsequently tested with both RPR and FTA-ABS.
We defined the reference standard for considering a patient to be infected with syphilis as follows: concordantly positive/reactive TS-EIA/RPR results or a positive FTA-ABS result. We also considered equivocal TS-EIA results as “positive” for the purposes of this analysis. Equivocal TS-EIA results typically reflex to additional testing in the reverse sequence algorithm, meaning that the TS-EIA equivocal specimens would have been further evaluated and likely found to be infected using this algorithm. We conducted 2 supplemental analyses, one in which we excluded all cases with a TS-EIA equivocal result and another in which we considered the TS-EIA equivocal result to be negative to determine if reclassifying the TS-EIA equivocal results impacted the sensitivity comparison. No direct detection methods for T. pallidum (e.g., dark field microscopy, direct fluorescent antibody, or polymerase chain reaction) were available for use at our clinic during the study.
Demographic data, HIV status, and serological data were routinely exported into an MS Access database from patients’ electronic medical record per standard procedures as part of HBHC’s participation in the CDC-sponsored STD Surveillance Network (SSuN). Sex of sex partners, lesion location, and self-reported lesion duration were abstracted from HBHC Disease Intervention Specialist interviews using CDC’s Interview Record. All data were de-identified for analysis. The institutional review board at HBHC reviewed and approved all study methods.
Characteristics of TS-EIA–negative and TS-EIA–positive patients were compared using χ2 tests for categorical variables and Wilcoxon tests for nonnormally distributed continuous variables. Sensitivity was calculated for TS-EIA and RPR as the number of true positives divided by the sum of true positives and false negatives, using the reference standard described above as the basis for classifying syphilis infection. Exact confidence limits for sensitivity estimates were calculated based on the binomial distribution. Trep-Sure EIA and RPR sensitivity values were compared using McNemar test for paired proportions. Statistical analysis was performed using SAS version 9.2 (SAS Institute, Cary, NC).
Between January 2009 and December 2011, 72 patients at the HBHC Walk-in STI Clinic had clinical symptoms consistent with primary syphilis combined with at least 1 positive/reactive serological test result for syphilis. Twenty (28%) had a history of syphilis and were excluded from this analysis, leaving 52 patients who met the inclusion criteria. Fifty-one (98%) identified as cisgender male and 1 (2%) identified as cisgender female. All patients reported sex with men as their primary sexual risk for syphilis infection (i.e., all 51 cisgender men were men who have sex with men). Median age was 29.5 (range, 17–56) years. The majority (53.8%) identified as white; 25% identified as African American or black; 19.2% identified as Hispanic or Latino; and 3.8% identified as another race/ethnicity. Five (9.6%) patients were HIV positive, and 3 (5.8%) had an unknown HIV status; the remaining 44 (84.6%) had confirmed negative HIV test results. Forty-eight patients (92.3%) had penile lesion(s), 3 (5.8%) had perianal lesion(s), and 1 (1.9%) had labial lesion(s), all of which were examined by experienced medical personnel.
The diagnostic serological results of the 52 patients included in this analysis are presented in Table 1. Twenty-eight patients (53.8%) had an initially positive or equivocal TS-EIA result. Twenty-five (89.3%) of these patients had a reactive RPR and were considered to be infected, including both patients with TS-EIA equivocal results who were also positive by FTA-ABS. The remaining 3 patients (10.7%) had discrepantly nonreactive RPR results requiring use of the FTA-ABS; all 3 were FTA-ABS positive and were considered infected.
Fifteen patients (28.8%) had an initially negative TS-EIA result but had a reactive RPR. All were FTA-ABS reactive and were considered infected. The remaining 9 patients (17.3%) had negative TS-EIA and nonreactive RPR results, but had a reactive FTA-ABS. These were considered primary syphilis infections for the purposes of this analysis because of the strong clinical suspicion of infection combined with a positive treponemal test and absence of syphilis history.
We did not observe any serological results that were either TS-EIA positive or TS-EIA equivocal and both RPR nonreactive and FTA-ABS negative among patients with primary symptoms. We also did not observe any RPR reactive results where both the TS-EIA and FTA-ABS were negative.
Trep-Sure EIA–positive/equivocal and TS-EIA–negative patients did not differ by age, race/ethnicity, sex, or HIV status (Table 2). The median lesion duration was 14 days (interquartile range [IQR], 9–21 days) for TS-EIA–positive/equivocal patients and 10 days (IQR, 7–16 days) for TS-EIA–negative patients; this difference was not statistically significant (P = 0.184). The median RPR titer was higher among TS-EIA–positive/equivocal patients (1:16; IQR, 1:8–1:32) than among TS-EIA–negative patients (1:8; IQR, 1:2–1:8; P = 0.022), suggesting that there may be a relationship between titer and TS-EIA seroconversion.
Sensitivity estimates comparing the TS-EIA and RPR for the 52 patients are shown in Table 3. The TS-EIA identified 28 of 52 cases based on our reference standard, for a sensitivity of 53.8% (95% confidence interval [CI], 39.5–67.8). The RPR identified 40 of 52 cases, yielding a sensitivity of 76.9% (95% CI, 63.2–87.5). The sensitivity of the RPR was significantly higher than the TS-EIA at detecting primary syphilis in our clinic (P = 0.005). The RPR resulted in detection of a net of 12 additional cases of primary syphilis, or 23.1% of all cases during the study period, compared with the TS-EIA.
We conducted 2 supplemental sensitivity analyses to ensure that the inclusion of TS-EIA equivocal results as positive did not bias our findings. First, we excluded the 2 patients with equivocal TS-EIA results from the analysis altogether, and second, we considered the TS-EIA equivocal patients to be negative (Table 3). In the first scenario, the TS-EIA identified 26 of 50 cases, a sensitivity of 52% (95% CI, 37.4–66.3). The RPR identified 38 of 50 cases for sensitivity of 76% (95% CI, 61.8–86.9). The TS-EIA identified 26 of 52 cases in the second scenario for a sensitivity of 50% (95% CI, 35.8–64.2), whereas the RPR identified 40 of 52 cases for a sensitivity of 76.9% (95% CI, 63.2–87.5). The RPR performed significantly better than the TS-EIA in both of these supplemental analyses (P = 0.005 and P = 0.002, respectively); reclassifying or excluding the equivocal TS-EIA results did not alter our conclusions.
To our knowledge, this is the first analysis evaluating the performance of the TS-EIA compared with the RPR in patients with symptoms suggestive of primary syphilis infection in routine clinical practice. The prospective collection of additional serological results via the institution of a standing order allowed us to consistently compare serological results for all patients in a high-morbidity clinic setting. Contrary to expectation, our data suggest that the TS-EIA was significantly less sensitive in diagnosing primary syphilis compared with the RPR. The lower sensitivity was not trivial clinically or epidemiologically; a net of 12 additional cases of primary syphilis (23.1% of all suspect cases included here) were identified by the RPR compared with the TS-EIA. Because the TS-EIA uses a proprietary recombinant antigen, it is difficult to explain the test’s performance and provide substantive reasons for our findings.
The synergies between the syphilis and HIV epidemics among men who have sex with men make identifying primary syphilis infections a crucial public health priority.14,15 Because many lesions go unrecognized, are mistaken for other conditions, may be occult, or simply may not be reported to clinicians during medical visits, using screening tests with superior sensitivity for primary syphilis infection is essential for disease detection and control. For that reason, the data presented here pose important questions about the rapid transition to reverse sequence algorithms without conducting more in-depth analyses of test performance for specific diagnostics. The main focus of conversation about the reverse sequence algorithm thus far in the United States has centered on discrepantly false-positive EIA/CIA results.1,2,5,12 We believe discrepantly false-negative results present a potentially more alarming scenario for some populations.
These data, however, do not provide enough information to determine which algorithm should be used for screening purposes. The reverse sequence algorithm diagnosed more total syphilis infections in our setting because of the TS-EIA’s ability to identify latent infections that have serologically reverted to RPR nonreactive (data not shown). In addition, financial and human capital efficiencies may outweigh the clinical impact of reduced sensitivity reported here.3,16 These considerations may justify the continued preference for reverse sequence algorithms in some settings.
There are several important limitations to our findings. Our analysis is limited to evaluation of the TS-EIA, which is the EIA used at the IDPH Laboratory. We cannot infer anything about the sensitivity or performance of other commercially available EIA/CIA tests used in reverse sequence algorithms elsewhere.
The STI Walk-in Clinic patient population of HBHC is not representative of all persons at risk of primary syphilis infection. This may lead to overestimation or underestimations of the TS-EIA’s sensitivity in broader practice. One group not represented well in our data is HIV-infected patients. In our sample, only 9.6% of patients had HIV, compared with more than 60% of the total syphilis diagnoses in our practice. At our facilities, most HIV-positive patients are screened for syphilis in primary care, and specimens are submitted to private laboratories for evaluation, all of which used the RPR as the initial diagnostic test during this period. We do not know how these results would differ in a screening population with higher HIV prevalence.
All serological tests have limitations in accurately diagnosing syphilis infections (especially primary stage infections), and a direct detection method would have conclusively met the laboratory criteria for syphilis diagnosis. Unfortunately, we were not able to supplement this analysis with confirmation by dark-field microscopy, direct fluorescent antibody, or polymerase chain reaction testing. These technologies are not used widely in clinical practice because of their required technical expertise, cost, timeliness, and the invasive nature of specimen collection. Despite the absence of additional confirmatory testing, we strongly believe that the combination of clinical judgment and expertise, and the available serological results presented, are highly suggestive of active syphilis infection. Howard Brown Health Center medical providers diagnose and treat more cases of syphilis than any other clinic in Illinois and have extensive experience evaluating primary syphilis chancres/lesions and differentiating ulcers caused by syphilis infection from ulcers of some other etiology. Forty-three (82.7%) of the 52 cases reported in this study also have 2 documented positive serological test results and clinical presentation compatible with primary syphilis infection. We are hopeful that future studies will incorporate such gold-standard methods to supplement this analysis.
Finally, we were not able to present individual OD index values for the TS-EIA–negative cases in our study. Wong and colleagues2 identified a similar instance of a false-negative result when comparing the TS-EIA to the Venereal Disease Research Laboratory test) and suggested laboratories conduct additional testing for TS-EIA results that had high nonreactive index scores (>0.200). The second case we described in the background section of this study had an OD index value of 0.215 and would have been reevaluated given this standard, which may have led to correctly diagnosing the infection. It is possible that some of the additional TS-EIA–negative cases reported here had OD values between 0.200 and 0.800 and would have benefitted from further scrutiny in the reverse sequence algorithm.
Syphilis diagnosis and serological interpretation is notoriously difficult and even more so for primary stage infections. Patients with no known history of syphilis who present with signs symptoms suggestive of active syphilis may benefit from receiving multiple tests, including both treponemal and nontreponemal diagnostics. It is also critical that medical providers conduct in-depth sexual and clinical history examinations in high-risk populations, including discussion of recent sexual exposures and history of lesions, as well as a thorough physical examination. Aggressive empiric treatment and frequent repeat testing in these groups are also warranted. Requesting additional tests and aggressively treating may not be possible, however, when primary lesions are occult or are not mentioned to medical providers, and in such scenarios, use of a reverse sequence screening algorithm may miss cases that would have been identified by the traditional algorithm. Early identification of primary syphilis infections has important public health implications for interrupting further syphilis transmission, due to the infectiousness of primary symptoms, and HIV transmission, given the synergistic nature of ulcerative syphilis and HIV infection.
Further research evaluating diagnostic sensitivity and specificity is therefore an urgent need for the field. We believe that these observational data provide compelling evidence that the TS-EIA has a lower sensitivity than the RPR in the detection of primary syphilis. Complementary research should be conducted with the use of a microbiologic gold-standard test (e.g., dark-field microscopy or polymerase chain reaction) that builds on these findings.
1. Centers for Disease Control and Prevention. Discordant results from reverse sequence syphilis screening—Five laboratories, United States, 2006–2010. MMWR 2011; 60: 133–137.
2. Wong EH, Klausner JD, Caguin-Grygiel G, et al. Evaluation of an IgM/IgG sensitive enzyme immunoassay and the utility of index values for the screening of syphilis infection in a high-risk population. Sex Trans Dis 2011; 6: 528–532.
3. Owusu-Edusei K, Peterman TA, Ballard RC. Serologic testing for syphilis in the United States: A cost-effectiveness analysis of two screening algorithms. Sex Trans Dis 2011; 38: 1–7.
4. Young H, Moyes A, Seagar L, et al. Novel recombinant-antigen enzyme immunoassay for serological diagnosis of syphilis. J Clin Microbiol 1998; 36: 913–917.
5. Centers for Disease Control and Prevention. Syphilis testing algorithms using treponemal tests for initial screening–Four laboratories, New York City, 2005–2006. MMWR 2008; 57: 872–875.
6. Larsen SA, Steiner BM, Rudolph AH. Laboratory diagnosis and interpretation of tests for syphilis. Clin Microbiol Rev 1995; 8: 1–21.
7. Larsen SA, Pope V, Johnson R, et al., eds. A manual of Tests for Syphilis, 9th ed. Washington, DC: American Public Health Association, 1998.
8. Ratnam S. The laboratory diagnosis of syphilis. Can J Infect Dis Med Microbiol 2005; 16: 45–51.
9. Sena AC, White BL, Sparline PF. Novel Treponema pallidum
serologic tests: A paradigm shift in syphilis screening for the 21st century. Clin Infect Dis 2010; 51: 700–708.
10. Schmidt BL, Edjlalipour M, Luger A. Comparative evaluation of nine different enzyme-linked immunosorbent assays for determination of antibodies against Treponema pallidum
in patients with primary syphilis. J Clin Microbiol 2000; 3: 1279–1282.
11. Binnkicker MJ, Jespersen DJ, Rollins LO. Treponema-specific tests for serodiagnosis of syphilis: Comparative evaluation of seven assays. J Clin Microbiol 2011; 49: 1313–1317.
12. Park IU, Chow JM, Bolan G, et al. Screening for syphilis with the treponemal immunoassay: Analysis of discordant serology results and implications for clinical management. J Infect Dis 2011; 204: 1297–1304.
14. Buchacz K, Greenberg A, Onorato I, et al. Syphilis epidemics and human immunodeficiency virus (HIV) incidence among men who have sex with men in the United States: Implications for prevention. Sex Trans Dis 2005; 32: S73–S79.
15. Zetola NM, Klausner JD. Syphilis and HIV infection: An update. Clin Infect Dis 2007; 44: 1222–1228.
16. Chuck A, Ohinmaa A, Tilley P, et al. Cost effectiveness of enzyme immunoassay and immunoblot testing for the diagnosis of syphilis. Int J STD AIDS 2008; 19: 393–399.