Mishra, Sharmistha MD, MSc*†; Boily, Marie-Claude PHD†‡; Ng, Victoria PHD§; Gold, Wayne L. MD*; Okura, Tom BSc¶; Shaw, Marsha BSc¶; Mazzulli, Tony MD¶∥; Fisman, David N. MD, MPH***
The accurate diagnosis of syphilis continues to elude—-and amidst a multitude of test interpretations—-perplex many clinicians. In the absence of a reliable gold standard, the diagnosis of syphilis requires epidemiologic exposure and characteristic symptoms and signs together with laboratory tests, the mainstay of which is serology. Traditionally, screening begins with the rapid plasma reagin (RPR), a nontreponemal test which, if positive, is confirmed by a treponemal test (Treponema pallidum particle agglutination, fluorescent treponemal antibody absorption, or microhemagglutination assay T. pallidum). Whereas the RPR titer correlates with disease activity (and is thereby advantageous for monitoring treatment response and reinfection), treponemal tests only confirm lifetime exposure to syphilis.1
Recently, many laboratories have started to use treponemal enzyme immunoassays (EIA) for screening because they can be performed on an automated platform and facilitate high-throughput screening. 2–4 Implemented to ease laboratory capacity, EIA-based screening has increased detection of RPR negative syphilis,2 but little is known about the effect of altering syphilis testing algorithms on physicians' interpretation of serology, and consequently, future testing. In August 2005, the Public Health Laboratory–Toronto (PHLT), the sole centralized syphilis screening site for the Greater Toronto Area (GTA) in Ontario, Canada, moved from the conventional RPR-based algorithm to EIA-based screening. Both before and after August 2005, screen positive results were confirmed with an alternate treponemal test and since August 2005, all samples positive on EIA screening underwent RPR testing. Herein, we evaluate the impact of this algorithm change on laboratory results including its effect on the relationship between test positivity and future testing patterns to inform ongoing practice and incorporation of treponemal tests for high throughput syphilis screening.
We conducted a retrospective, population-based study of laboratory syphilis diagnoses to assess patterns of testing and reporting of results following implementation of EIA screening in the GTA. Laboratory data between August 1, 1998 and July 31, 2008, were collected on all serum samples submitted for syphilis screening from testing centers within the GTA. Demographic and limited clinical information were available for submissions between August 1, 2005 and July 31, 2008. Before August 1, 2005, demographic variables and clinical data were restricted to screen positive samples. Individual-level data were extracted from the PHLT database; outliers and missing data were reconciled with the stored hardcopy requisition and laboratory specimen data at the time of analysis.
Serum samples were excluded if they had been submitted via the Canadian Blood Services as blood donor syphilis screening procedures are conducted under a separate protocol and samples are referred to the PHLT only for confirmatory testing.
We defined a screen positive sample on the basis of a reactive RPR during the RPR period (August 1, 1998–July 31, 2005). During the EIA period (August 1, 2005–July 31, 2008), a screen positive sample included a positive EIA (Enzygnost Syphilis, Dade Behring, Germany) or an indeterminate EIA on >1 of duplicate testing on the same sample.
Samples were defined as confirmed positive in the case of a positive screening test if at least one of the treponemal tests used for confirmation (microhemagglutination assay T. pallidum; T. pallidum particle agglutination; fluorescent treponemal antibody absorption test) was positive. To estimate the rates of submissions and positives, we excluded repeat submissions after an initial positive result per patient. The submission rate per 100,000 persons was calculated using the population of the GTA during the study period5; monthly population estimates were derived using linear interpolation. For an estimate of nontreponemal seroconversion among patients with EIA(+)/RPR(−)-confirmed positive tests during the EIA period, we evaluated repeat submissions up to 2 months following the initial sample (based on estimated duration of incubating and primary syphilis).6,7 During both screening periods, the submission report returned to physicians included information on all tests conducted on the sample.
To assess the effect of algorithm change on test positivity, we conducted interrupted time-series analysis using segmented linear regression by dividing the series into RPR and EIA segments.8,9 We evaluated the following per capita (per 100,000 population) outcomes: (1) rate of confirmed positives, (2) rate of confirmed positives with reactive RPR, (3) rate of confirmed positives with RPR titer ≥1:8 (high-titer). We similarly evaluated per capita submission rates and further included yearly segments to characterize short-term changes in submission. Because linear regression models include 2 parameters (level and slope), we tested for a change in both parameters to quantify the difference in positivity rates between time-periods. A change in level between periods indicates a step-change, and a change in slope indicates a change in trend. We corrected for autocorrelation as described in the Appendix.
We then sought to evaluate whether the introduction of the EIA, as a treponemal screening test for syphilis, resulted in an increase in future submissions (hypothesized to occur as a result of heightened clinician awareness of syphilis or testing of sexual contacts of individuals diagnosed as infected). The impact of EIA screening on the monthly per capita submission rate (response variable) was evaluated using the Granger test for causality with linear autoregressive models,10 incorporating the following exposure variables: (1) confirmed positives (both periods), (2) confirmed positives with reactive RPR, (3) confirmed positives with RPR titer ≥1:8 (both periods), and (4) confirmed positives with negative RPR (EIA period). Granger methods can be used to evaluate the possibility of causal relationships between closely correlated time series; a causal exposure to an independent variable should be associated with a lagged change in the dependent variable, and this relationship should be absent when the temporal sequence of exposure and response is reversed.10–16 A more detailed description of time-series methods used is provided in the Appendix. Analyses were repeated by city and yearly intervals as appropriate.
During the EIA period, associations between demographic and reported clinical covariates and the RPR status of confirmed positives were assessed with univariate Poisson regression. Age was converted into a categorical variable with 2 subgroups (10–50 and >50 years, based on the median age among individuals with a diagnosis of late latent syphilis as reported to the Toronto Public Health Department),17 and symptoms were dichotomized on the basis of sexually transmitted infection (STI) symptoms (vs. non-STI symptoms or absence of symptoms). Risk factors were evaluated as categorical variables (intravenous drug use, males who have sex with males, antenatal testing [females], intravenous drug use, and males who have sex with males [males], no risk factor).
Analysis was conducted using Stata (Version 9.2, StataCorp). The study protocol was approved by the University of Toronto Research Ethics Board and PHLT Research Committee.
After excluding repeat submissions on the basis of a prior confirmed positive result, a total of 3,092,938 submissions remained for analysis. As shown in Figure 1, 0.59% and 2.24% of samples, tested under the RPR and EIA periods, respectively, screened positive. During the RPR period, 0.46% of patients tested were confirmed positive compared with 1.98% of patients during the EIA period. During the latter period, 1.38% of confirmed positives were associated with a negative RPR, comprising 69.6% of all confirmed positives. In the central, urban core of the GTA (City of Toronto) the incidence rate ratio for confirmed positivity was 1.69 (95% confidence interval [CI]: 1.35–1.88) during the RPR period, and 1.80 (95% CI: 1.67–1.91) during the EIA period, relative to surrounding suburban areas.
The per capita rate of detecting confirmed positives are shown in Figure 2. Following EIA implementation, the monthly rate of confirmed positives increased by approximately 3-fold from 3.2 to 13.5 per 100,000 population (P < 0.001). The corresponding results of the segmented regression analysis for rates of screen and confirmed positives are summarized in Table 1. Although confirmed positive rates with reactive RPR or high-titer RPR increased over time, the trends were unaffected by change in algorithm (P = 0.12 and 0.11, respectively), nor was there evidence of a step-change (P = 0.13 and 0.10, respectively).
As depicted in Figure 3, the per capita submission rate increased over the period of study (on average, by 1.26 per 100,000 population each month after adjusting for autocorrelation, P < 0.001). The algorithm change did not significantly impact on change in level (step-change, P = 0.39) or change in slope (P = 0.22) when analyzed over the entire study period. First-order autocorrelation was evident (P = 0.03). However, comparison of short-term trends (series segmented into 1-year blocks relative to August 2005) demonstrated a steeper increase during the first year of EIA screening (increasing by 3.89 submissions per 100,000 population every month (95% CI: 2.99–4.79) as compared with the previous year under RPR screening (0.93 submissions [95% CI: 0.57–1.29]) and the second year of EIA screening (1.17 submissions [95% CI: 0.45–1.89]).
Findings from tests of Granger causality are shown in Table 2. Each of the 3 laboratory results were unidirectional Granger causal for future submission rates, suggesting that reporting of confirmed positives, RPR positives, and high-titer RPR positives, are all predictive of per capita submission rates over the subsequent 4 months. When assessed at yearly intervals (relative to August 2005), the directional relationship persisted throughout both RPR and EIA periods, suggesting that RPR positive and high-titer RPR positive syphilis remained relatively stable in predicting future submissions post-changeover (P > 0.10). No geographic variation was observed between the City of Toronto and surrounding suburban regions. Confirmed positives in the City of Toronto were predictive (unilateral Granger causal) of submission rates in each of the surrounding suburbs within the GTA (P < 0.05), but the inverse was not significant (P > 0.1). During the EIA period, the rate of confirmed positive with negative RPR was Granger causal for predicting submission rate, but only during the first 12 months after the changeover (R2 = 0.44, P = 0.01).
Patient Characteristics Associated With Confirmed Positive and RPR Negative Results During the EIA Period
Table 3 lists the demographic and reported clinical features associated with RPR negative samples among patients with a confirmed positive result during the EIA screening period, as compared to those with both EIA and RPR positivity. Older age, male sex, and the absence of reported STI symptoms were associated with a negative RPR result. Among the 9137 patients with a EIA(+)/RPR(−) result on initial testing, 0.59% subsequently seroconverted to RPR reactive on repeat samples submitted within 8 weeks, indicating a diagnosis of early syphilis. Among the 560 individuals on whom lesion scrapings for direct fluorescent antibody testing were submitted alongside sera during the EIA period, 3.2% with EIA(+)/RPR(−) and 2.6% with EIA(+)/RPR(+) confirmed positive results also had direct fluorescent antibody positive lesion scrapings (P = 0.09).
Laboratories in many industrialized nations are increasingly using automated treponemal screening tests such as the EIA for the serological diagnosis of syphilis—driven primarily on the need for high-volume screening and to limit manual flocculation required of screening RPR. Our experience in the GTA revealed 2 interesting consequences of this changeover; an increased serological detection of syphilis and a transient predictive relationship between RPR negative syphilis and future submission rates after EIA implementation.
We observed an increase in the diagnosis of confirmed positive syphilis during the EIA period by approximately 3-fold compared with the RPR period, predominantly because of EIA(+)/RPR(−) results. In the absence of detailed clinical data, this would correspond to increased detection of latent syphilis of unknown duration, prior treated syphilis, early syphilis, or false positive tests (because of false positivity of both screening and confirmatory treponemal assays) including nonsyphilitic treponemal disease. Together, the excess primarily represents infection that would have been missed with RPR screening, including the identification of a small group of patients with infectious (transmissible) syphilis. Indeed, trends in cases of late latent syphilis reported to the Toronto Public Health Department increased dramatically after August 2005 and peaked in 2006.17 Detection of early syphilis among 0.59% of 9137 patients who may have been screen negative under the RPR-screening algorithm suggests that EIA screening could result in decreased syphilis transmission at a population level if the initial result leads to treatment. Yet detection of primary syphilis using treponemal tests (when compared with nontreponemal tests) has been inconsistent across immunoassays.18,19 Our findings are similar to those observed in New York, where 3% of all submitted samples were EIA positive and RPR negative during the first 14 months after implementation of EIA screening—although the decision to use confirmatory testing with another treponemal assay in this setting varied across laboratories, and follow-up serology (seroconversion to RPR reactive) was not reported.2
In addition to the expected increase in overall identification of syphilis, our use of Granger time series methods permitted the identification of positive “feedbacks” in syphilis testing at the community level, which to our knowledge have not been previously documented. Our findings could be in keeping with the adage the more we find, the more we search—with utilization of the EIA further facilitating detection. However, our forecasting analyses are limited by the lack of other possible explanatory variables for future submission rates, such as public health awareness, screening campaigns, rates of other STIs, and details of contact tracing patterns.
The exclusion of repeat submissions following an initial confirmed positive result essentially “removed” a pool individuals from future testing and may explain the decrease in submission rates noted after the first year of EIA relative to preceding year. However, it fails to account for the transient positive correlation between EIA(+)/RPR(−) results and subsequent testing rates after the introduction of EIA screening. The increase may have been due to identification of individuals with prevalent latent syphilis (as only a small percentage of EIA (+)/RPR(−) results would have represented early syphilis). We postulate 2 possible reasons for the transient nature of this phenomenon. In the early period following the changeover, there was little clinical experience with EIA(+)/RPR(−) results—and physicians and public health authorities may have performed contact tracing and partner notification on individuals with these test results, leading to increased submissions, despite local public health consensus that patients with late latent syphilis need not undergo contact tracing.20 As experience grew, clinicians may have revised their approach to individuals with this constellation of serological results, managing them instead as having prior treated syphilis, nonsyphilitic treponemal disease, or false-positive test results (making partner testing unnecessary). The second explanation involves testing recent or current partners of individuals who were diagnosed with syphilis based on a positive EIA and negative RPR. If the partner's result was similar (or negative), the chain of contacts may have stopped there (especially if either individual's infection was felt to be remote)—that is, there was a surge in testing of individuals based on their partnership status, the results of which did not lead to further testing.
We also found that within the GTA, positive tests appeared to cluster in the region's urban core. This could reflect any of the following: (a) patients sought testing in the City of Toronto but resided in other regions, such that partner tracing—and subsequently, future testing—occurred in the other cities, or (b) most infections were diagnosed in testing centers in Toronto, and resultant public awareness led to heightened testing in the other cities. As the geographic unit of measurement was defined by the place of testing, this pattern of prediction does not necessarily invoke Toronto as the site of syphilis acquisition with onward transmission in other cities at a later time.
The use of laboratory data from a large metropolitan area with centralized syphilis testing constitutes a major strength of this study, as it permits evaluation of the complex interplay between disease epidemiology, evolving testing methods, and testing by clinicians. However, because of the laboratory-based nature of this study, our analysis is limited by the lack of clinical information for syphilis staging and longitudinal clinical follow-up of patients with RPR negative results. Given the multitude of diagnostic possibilities conferred by a laboratory report of EIA(+)/RPR(−) serology, further clinical investigation into the diagnostic properties of isolated EIA(+) or EIA(+) in conjunction with another positive treponemal test are needed. To date, most studies have relied on stored or prospectively collected sera from individuals within different stages of syphilis as well as nonsyphilitic sera, with both sets (except perhaps in primary syphilis) compared alongside other treponemal tests as the reference standard.18,21,22 Furthermore, treponemal tests have different levels of agreement and often a consensus of more than one test is required for reference.
After introducing a treponemal test as the initial screening tool for syphilis, health-care providers and public health teams will need to prepare themselves and their patients for an influx of patients with EIA positive results alongside negative RPR and/or negative confirmatory test. Reports of isolated EIA positive samples among high-risk populations suggest the result confers a diagnosis of prior or latent infection, but these studies are restricted to a patient population with a high pretest probability of syphilis.23,24 In addition to reconciling discordant serological results with the absence/presence of syphilis among patients with EIA(+)/RPR(−) results, the decision to investigate for concomitant neurosyphilis in these cases must be addressed. Although the probability of neurologic complications among patients with RPR-negative latent syphilis of unknown duration is likely small, the utility of lumbar punctures (and neuroimaging) on elderly patients with dementia and asymptomatic HIV-positive patients is unclear (although both would meet recommendations for lumbar punctures based on current Centers for Disease Control and Public Health Agency of Canada guidelines25,26). These patients, under an RPR-based screening algorithm, would otherwise not have been identified with possible syphilis.
Laboratories that experience a high volume of submissions for syphilis screening are moving from the conventional RPR to automated treponemal screening tests. The downstream consequence of this changeover in screening methodology can have a significant impact on the detection of syphilis, leading to an increase in the diagnosis of latent syphilis of unknown duration or prior treated syphilis, and a small but notable increase in the detection of early syphilis. To what extent these changes impact on the dynamics of syphilis in the community warrant further study. On one hand, enhanced detection (and therefore treatment) of early syphilis may interrupt onward transmission, yet on the other hand—increased treatment of early syphilis and noninfectious latent syphilis may increase vulnerability to reinfection (in the absence of behavior change). Laboratories, public health systems, and clinicians should be aware and prepared for the potential effects of moving from RPR to treponemal screening tests, as increases in testing capacity, medication supply (for treatment of latent syphilis of unknown duration), public health resources (for case follow-up and partner notification), and the need for preemptive guidance on serological interpretation for front-line physicians should be anticipated.
Interrupted time-series analysis can be used to evaluate the effect of external factors such as interventions (policy change) on time series data. To evaluate the impact of algorithm change in test positivity rates, we used segmented linear regression, which divides a time series into pre- and post-changeover segments. We assessed for changes in the level and slope between the 2 segments. Where Yt is the outcome variable (test positivity),
Yt = β0 + β1 × timet + β2 × changet + β3 × (time after change)t + εt
β0 estimates the baseline level of the outcome at the beginning of the time series; β1 estimates the prechange trend where time is a continuous variable indicating the time in months at time t from start of the study period. β2 estimates the change in level post-changeover where changet = 0 during the RPR period and changet = 1 during the EIA period. β3 estimates the change in post-changeover trend where time after change is a continuous variable indicating the number of months after the start of EIA screening at time t and is coded as zero pre-changeover. εt includes random error and autocorrelation. Time series data are often correlated (events closer in time series tend to be more similar than events further apart in time). Hence, in our linear regression model, the residuals may not be independent. To correct for this, lagged residuals are included in the regression model (the residual of the regression model is shifted to the previous time points in the time series).8,9
Granger causality tests are used to evaluate the forecasting properties of previous (lagged) values of exposure covariates (e.g., confirmed positive rate) in predicting the current outcome (submission rate) using time-series data, provided that the lagged values of the covariate jointly improve the model fit over and above the lagged values of the outcome and the contemporaneous values of the exposures. If Yt is the outcome at time (t), then Xt is said to be Granger causal for Yt if Xt helps predict Yt at some stage in the future—a potentially useful model so long as there is no duality (i.e., Yt does not also Granger cause Xt). Such models have been recently utilized in the health sector.13–15
First, the prerequisites were verified. We found no graphical (visual) evidence of periodicity for each variable, formally confirmed by Walter and Elwood test (P > 0.5). After assessment for nonstationary time-series data, first-order differencing enabled stationarity (Dicky-Fuller unit root test, P < 0.001) of each covariate and submission rate, and was conducted separately for the pre- and post-changeover periods. Investigation for a structural break (short-period of rapid change) in the per capita submission rate over time was not evident by the Zivot-Andrews method.10,16
We then built linear autoregressive models of the per capita submission rate and separately for the following per capita exposures: (a) confirmed positives with reactive RPR (both periods), (b) confirmed positives with negative RPR (EIA period), and (c) confirmed positives with high-titer RPR (both periods). Each variable was found to be stationary after first-order differencing. If we represent the first-order difference of the submission rate as Yt = the submission rate at time (t) − submission rate at time (t-1), and the first-order difference of the result rate Xt = first time true positive with RPR positive rate at time (t) − first time true positive with RPR positive at time (t-1), the model can be represented with the following 2-dimensional time-series vector:
The superscript displays the number of the lagged term (not an exponential term) up to k lags. The error terms in each equation were uncorrelated (Durbin-Watson h test, P > 0.1). The data were fit to an autoregressive model of order 4 (4 lags; 1 lag = 1 month) for each of the 3 models by measuring iterative AIC criterion. The coefficients (βyx1 … βyxk) were jointly tested for significance by the F test (5% α-level) for the first and second (βxy1 … βxyk) equations separately.
Equation (Uncited)Image Tools
The final univariate equations (excluding individual nonsignificant [P > 0.1] estimated coefficients) are shown for confirmed positives with reactive RPR (both periods).
Yt = 76.9 + 0.12Yt−1 + 0.48Yt−2 + 0.35Xt−2 + 1.93Xt−3 + 1.47Xt−3
Xt = 0.24 + 0.09Xt−2 + 0.12Xt−3 + 0.06Xt−3
Hence, there was evidence of unidirectional granger causality of Xt (first-order difference of confirmed positives with reactive RPR) predicting future Yt (first-order difference of submission rate).
1.Larsen SA, Steiner BM, Rudolph AH. Laboratory diagnosis and interpretation of tests for syphilis. Clin Microbiol Rev 1995; 8:1–21.
2.Centers for Disease Control and Prevention. Syphilis testing algorithms using treponemal tests for initial screening—four laboratories, New York City, 2005–2006. Morb Mortal Wkly Rep 2008; 57:872–875.
3.Egglestone SI, Turner AJ; PHLS Syphilis Serology Working Group. Serological diagnosis of syphilis. Commun Dis Public Health 2000; 3:158–162.
4.Amin AK, Manuel RJ, Ison CA, et al. Audit of laboratory diagnostic methods for syphilis in England and Wales. Sex Transm Infect 2009; 85:88–91.
6.LaFond RE, Lukehart SA. Biological basis for syphilis. Clin Microbiol Rev 2006; 19:29–49.
7.Sparling PF, Swartz MN, Mushner DM, et al. Clinical manifestations of syphilis. In: Holmes KK, Sparling PF, Stamm WE, et al, eds. Sexually Transmitted Diseases, 4th ed. New York, NY: McGraw-Hill, 2008:661–684.
8.Wagner AK, Soumerai SB, Zhang F, et al. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002; 27:299–309.
9.McDowall D, McCleary R, Meidinger E, et al. Interrupted Time Series Analyisis. Thousand Oaks, California: Sage Publications, 1980.
10.Gujarati D. Basic Econometrics. 4th ed. New York, NY: McGraw-Hill, 2003.
11.Granger C. Investigating causal relations by econometric methods and cross-spectral methods. Econometrica 1969; 34:424–438.
12.Sims C. Money, income and causality. Am Econ Rev 1972; 62:540–552.
13.Moineddin R, Nie JX, Domb G, et al. Seasonality of primary care utilization for respiratory diseases in Ontario: A time-series analysis. BMC Health Serv Res 2008; 8:160.
14.Upshur RE, Moineddin R, Crighton EJ, et al. Interactions of viral pathogens on hospital admissions for pneumonia, croup and chronic obstructive pulmonary diseases: Results of a multivariate time-series analysis. Epidemiol Infect 2006; 134:1174–1178.
15.Pitard A, Viel JF. A model selection tool in multi-pollutant time series: The Granger-causality diagnosis. Environmetrics 1999; 10:53–65.
16.Zivot E, Andrews DW. Further evidence on the Great Crash, the oil-price shock, and the unit-root hypothesis. J Bus Econ Stat 2002; 20:25.
18.Schmidt BL, Edjlalipour M, Luger A. Comparative evaluation of nine different enzyme-linked immunosorbent assays for determination of antibodies against Treponema pallidum in patients with primary syphilis. J Clin Microbiol 2000; 38:1279–1282.
19.Singh AE, Wong T, De P. Characteristics of primary and late latent syphilis cases which were initially non-reactive with the rapid plasma reagin as the screening test. Int J STD AIDS 2008; 19:464–468.
20.Provincial Infectious Diseases Advisory Committee. Sexually Transmitted Infections Case Management and Contact Tracing Best Practice Recommendations. Toronto, Canada: Ontario Ministry of Health and Long-Term Care, 2009.
21.Castro R, Prieto ES, Santo I, et al. Evaluation of an enzyme immunoassay technique for detection of antibodies against Treponema pallidum. J Clin Microbiol 2003; 41:250–253.
22.Young H, Pryde J, Duncan L, et al. The Architect Syphilis assay for antibodies to Treponema pallidum: An automated screening assay with high sensitivity in primary syphilis. Sex Transm Infect 2009; 85:19–23.
23.Woznicova V, Valisova Z. Performance of CAPTIA Select Syph-G enzyme-linked immunosorbent assay in syphilis testing of a high-risk population: Analysis of discordant results. J Clin Microbiol 2007; 45:1794–1797.
24.Ooi C, Robertson P, Donovan B. Investigation of isolated positive syphilis enzyme immunoassay (ICE Murex) results. Int J STD AIDS 2002; 13:761–764.
26.Public Health Agency of Canada. Canadian guidelines on sexually transmitted infections 2009. Available at: http://www.phac-aspc.gc.ca/std-mts
. Accessed December 8, 2009.