Recent advances in syphilis testing technologies have led to the use of automated treponemal tests that can reduce labor costs for large laboratories.1 However, the new tests involve a different testing sequence that identifies more individuals who need clinical follow-up. The net effect of these different costs and effects, beyond the laboratory, might help guide the choice of tests in different situations.
Serologic tests for syphilis fall into 2 categories: nontreponemal (or reagin-based tests) and treponemal tests. The nontreponemal tests (such as, rapid plasma reagin [RPR] and venereal disease research laboratory [VDRL]) detect antibodies to cardiolipin and are not specific for treponemal infections.1,2 Although nontreponemal tests have relatively lower sensitivity and specificity, they have a higher correlation with disease activity3,4 and consequently are more likely to become nonreactive after treatment than treponemal tests.1 False-positive nontreponemal tests can occur in individuals with conditions such as viral and rickettsial diseases, malaria, connective tissue disorders, pregnancy, intravenous drug use, and advanced malignancy.5
In contrast, treponemal tests, such as enzyme immunoassays (EIAs), chemiluminescence assays (CAs), the Treponema pallidum passive particle agglutination assay (TP-PA) and fluorescent treponemal antibody absorbed (FTA-ABS) tests, detect antibodies that are specific to treponemes, which cause syphilis and other treponemal infections such as yaws, pinta, and bejel.6 Nonveneral treponematoses are rare in the United States,7 but there are other causes of false-positive results such as Lyme borreliosis and autoimmune disease.8,9 Most individuals who are treated for syphilis continue to have reactive treponemal tests for the rest of their lives,10,11 so reactive treponemal tests do not differentiate between adequately treated and untreated syphilis infections.1
In the United States, the classic testing algorithm comprises an initial screen with a nontreponemal test. If this test is positive, a second (treponemal test) is done.1 This algorithm has been shown to be more cost-effective12 because the cost of the second test (TP-PA, $18) was about 3 times the cost of the first test (RPR, $6).13 The recent introduction of automated treponemal EIA/CA tests has led some high-volume laboratories to adopt the new reversed algorithm. The new algorithm starts with an automated treponemal test and all reactive specimens are confirmed with a nontreponemal test (RPR).1 The RPR-negatives are further tested using another treponemal test such as a TP-PA test. This new algorithm identifies persons who were not previously identified—those with reactive treponemal tests and nonreactive nontreponemal tests. Evaluation and follow-up of these persons bring added costs to the new algorithm.
The primary purpose of this study was to compare the total costs and effects (cases treated) of the 2 syphilis testing algorithms using estimated summary measures - cost-effectiveness ratios.
MATERIALS AND METHODS
We constructed a decision analysis model for a cohort of 200,000 individuals to compare the costs and effects of the traditional algorithm (Nontreponemal-First) and the new algorithm (Treponemal-First). Previous cost-effectiveness studies that assessed the performance of syphilis tests divided the population into 2 basic categories based on syphilis serostatus – infected and uninfected.14–16 However, because of the persistent nature of syphilis antibodies even after adequate treatment,1,6,10,11 the performance of available treponemal tests is also influenced by treponemal infections that have previously been treated. Thus, following Blandford et al.,17 we constructed the decision tree such that the population was divided into 3 mutually exclusive categories–infected, never-infected, and previously infected. The proportion of the cohort that was previously infected was assumed to have been adequately treated and currently not infected.
We assumed a prevalence (current infections) of 0.5%, which was mostly late latent (may include reinfections and/or relapsed cases); 5% of the population was previously infected, and the remaining 94.5% had never been infected. For those who were infected, the likelihood of developing tertiary syphilis if untreated was 28%18 if they were EIA/CA-positive and RPR-positive, and 2.8% if they were EIA/CA-positive but RPR-negative. The risk of advanced disease was lowered further by assuming that 70% of infected persons with no known history of treatment were unknowingly cured by antibiotics taken for other reasons.19 The cost of disease for persons who were EIA/CA and RPR-positive was $610 (which includes the expected cost of advanced stages of syphilis for the fraction that will progress20) and $60 for individuals who were EIA/CA-positive and RPR-negative, because the risk of progression to tertiary syphilis for this group was assumed to be 10-times lower. The specificity of EIA/CA was 0.99 for the never-infected individuals, and TP-PA concordance with a false-positive EIA/CA result was assumed to be 50%. The TP-PA concordance with a false-positive EIA/CA was varied from 5% to 95% because it was found to be influential in determining the number of overtreatment, and because expert opinion on its magnitude varied. Also, because individuals with previous syphilis continue to have treponemal antibodies for the rest of their lives,1,11 it was assumed that the treponemal tests (EIA/CA and TP-PA) would have very low specificities (for current infections) for this category. Thus, the specificities of EIA/CA and TP-PA for current infection among previously infected persons were assumed to be 5% and 0%, respectively (we varied these numbers in a sensitivity analysis to examine their influence as well).
Per patient reminder cost ($19), obtained from primary data in an earlier study,21 was used as the cost to determine whether a person had been previously treated (follow-up cost). The cost of treating syphilis was $50 (range, $25–$100) for early infections and $100 (range, $50–$150) for late latent. Details of other estimates of sensitivity and specificity were obtained from the literature and from expert opinion (Table 1). We used a health-care system perspective. All costs were adjusted to 2008 United States dollars using the medical care component of the Consumer Price Index for All Urban Consumers.26
For the Nontreponemal-First algorithm, individuals were followed up if they were reactive to both RPR and EIA/CA. For the Treponemal-First algorithm, individuals were followed up if they were reactive to EIA/CA and RPR, or nonreactive to RPR, but reactive to TP-PA. For the purpose of this study, we assumed that perfect information was obtained from the interview process for all categories. In other words, all those who belonged to the previously treated category were not treated because it was determined through the interview process that they had been adequately treated. However, those who were never infected and were asked whether they had been previously treated, were treated because they confirmed through the follow-up process that they had never been treated for syphilis. Finally, all those who were infected and identified for follow-up were treated.
We estimated and compared the total net costs (total cost of program minus cost of diseases prevented), total number of individuals who were identified for follow-up, total number of true cases of syphilis treated, as well as the total number of persons overtreated for each algorithm. Following cost-effectiveness analysis guidelines,27–29 we computed cost-effectiveness ratios for the 2 algorithms as the net cost divided by the number of cases treated or prevented (it was assumed for the purpose of this study that all cases identified and treated were successfully cured). We then conducted comprehensive sensitivity analyses on all variables in the model using ranges presented in Table 1 and determined the threshold values (i.e., the critical value at which the final decision switched from one algorithm to the other). We used DATA Professional version 4.0 (TreeAge Software, Williamstown, MA) to construct the decision tree and conduct the comprehensive sensitivity analyses. Microsoft Excel, version 2003 (Microsoft Corporation, Redmond, WA), was used for summary analyses and result presentation.
Using the baseline values for a cohort of 200,000 individuals (including 1000 infected and 10,000 adequately treated previously infected individuals), results indicated that the net costs were $1.6 million for the Treponemal-First algorithm and $1.4 million for the Nontreponemal-First algorithm (Table 2). The Treponemal-First algorithm identified 11,450 people for follow-up (to determine whether they were previously treated) and resulted in the treatment of 1950 individuals (Fig. 1). Most of those treated were persons who were EIA/CA-positive, RPR-negative, and TP-PA-positive. Of the number treated in the Treponemal-First algorithm, a total of 986 actual cases (≈99% of the cases) were identified and treated (see Table 2). However, approximately 49% (n = 964) of those treated were not currently infected. The Nontreponemal-First algorithm identified 3756 for follow-up (to determine whether they were previously treated) and resulted in the treatment of 906 (Fig. 1). Of the number treated, a total of 868 actual cases (≈87% of the cases) were identified and treated (Table 2). However, approximately 4% (n = 38) of those treated were not currently infected. Thus, the Treponemal-First algorithm treated approximately 118 more infected individuals. We estimated that treating them would prevent one case of tertiary syphilis because most people will have at least partial treatment for other reasons. The estimated cost-effectiveness ratio was slightly lower for the Nontreponemal-First than the Treponemal-First algorithm ($1621 vs. $1671 per case treated) and the incremental cost-effectiveness ratio was $2042 per an additional case treated.
One-Way Sensitivity Analyses
In the one-way sensitivity analyses, we varied 1 variable using the ranges presented in Table 1 while keeping the others constant and computed the cost-effectiveness ratios. Summary results of the comprehensive sensitivity analyses are presented in Table 3. We found threshold values within the ranges used for 10 variables. In other words, over the ranges used, the relative magnitude of the cost-effectiveness ratio changed such that the algorithm with the initially lower (or higher) cost-effectiveness ratio became the algorithm with the higher (or lower) cost-effectiveness ratio beyond a critical value. The 10 variables were proportion with previous infections, cost of EIA/CA, RPR, and TP-PA, follow-up cost, sensitivity of RPR (late latent), specificities of RPR, EIA/CA, and TP-PA for the never infected, and specificity of RPR for previous infections. Figure 2 depicts the magnitudes of cost-effectiveness ratios for the 2 algorithms over the specificity ranges used for TP-PA (for the never infected proportion) and RPR (for previous infections). The results show that as the specificity of the TP-PA increases from 5% to 95%, the cost-effectiveness ratio of the Treponemal-First algorithm decreased from $1761 to $1592, whereas that of the Nontreponemal-First remained unchanged at $1621. At a specificity of 72%, the cost-effectiveness ratios are the same and when the specificity of TP-PA for the never infected individuals was greater than this critical value (72%), the Treponemal-First algorithm was more cost-effective, i.e., its cost-effectiveness ratio was lower than the cost-effectiveness ratio for the Nontreponemal-First algorithm (see Fig. 2, panel A). Our results also indicated that as the specificity of RPR (for previous infections) increased from 50% to 75%, the algorithm with the lower cost-effectiveness ratio changed from Treponemal-First to Nontreponemal-First, with a threshold value at 59% (see Fig. 2, panel B).
Two-Way Sensitivity Analyses
In the 2-way sensitivity analyses, we varied 2 variables while keeping all the other variables constant and then estimated and compared the cost-effectiveness ratios. Results of the 2-way sensitivity analyses are presented in Figure 3. Shaded regions imply lower cost-effectiveness ratios for the combination of values of the 2 variables and the dotted lines depict threshold value(s). Our results indicated that keeping all other variables constant, the cost-effectiveness ratio of the Nontreponemal-First algorithm was lower than that of Treponemal-First over the entire range of the follow-up cost ($10–$30) as long as the cost of EIA/CA was more than $4.1 (Fig. 3, panel A). Our results also show that if follow-up cost was at least $14.4 and the proportion of previous infections was greater than, or equal to 1.3%, then the Nontreponemal-First algorithm was more cost-effective, all else remaining constant (see Fig. 3, panel B). If the cost of RPR remained below $3.2 and that of EIA/CA was greater than $7.6, then the Nontreponemal-First option was more cost-effective, all else remaining constant (see Fig. 3, panel C). Finally, we found that if the cost of EIA/CA was greater than $5 then over the range of previous infection values used (1%–10%), the Nontreponemal-First option remained more cost-effective, all else remaining constant (see Fig. 3, panel D).
Technological advances have prompted some laboratories to reverse the traditional order of screening for syphilis. We assessed the health and economic outcomes of this new algorithm in comparison with the traditional algorithm–Treponemal-First versus Nontreponemal-First. For the baseline analysis, our results indicated that even though the Treponemal-First algorithm was more effective (resulting in the treatment of 99% of the syphilis cases), it was more expensive overall because it resulted in a substantially higher number of follow-ups and overtreatment. The Treponemal-First algorithm resulted in over 3 times as many persons for follow-up as did the Nontreponemal-First algorithm (11,450 vs. 3756) and resulted in the treatment of almost as many uninfected persons as actual cases treated—964 of the 1950 treated were not infected. The overtreatment rate was substantially lower when applying the Nontreponemal-First algorithm than the Treponemal-First algorithm (≈4% vs. ≈50%). Thus, the estimated net cost per case treated was slightly lower for the Nontreponemal-First than the Treponemal-First algorithm ($1621 vs. $1671). The estimated incremental cost-effectiveness ratio indicated that it would cost $2042 to increase the effectiveness by one syphilis case treated. Thus, a change from the Nontreponemal-First algorithm to the Treponemal-First algorithm would cost relatively more for an additional case treated.
The number of overtreatment was a result of the lack of independence of the EIA/CA and TP-PA. Thus, we varied the specificity of TP-PA widely to show its effect on the final results. Our results indicated that when the TP-PA was independent of the EIA/CA (i.e., higher specificity), it reduced overtreatment and lowered the cost-effectiveness ratio. In contrast, when it was concordant with the EIA/CA results (low specificity), the overtreatment was high, resulting in a high cost-effectiveness ratio. Second, the higher number of follow-ups required for the Treponemal-First option was a result of the relatively low specificity of the treponemal tests (EIA/CA and TP-PA) for persons who were previously infected. On the contrary, using the RPR as an initial test reduced substantially the number of follow-ups required because it has a higher performance in differentiating current from previously infected (and adequately treated) individuals.
There are no other studies that have compared these specific algorithms so discussion of our results in relation to the findings in other studies is not possible. However, a recent cost-effectiveness study conducted in Canada16 recommended the use of EIA/CA as an initial test and Inno-Lia as the confirmatory test so as to replace the RPR and TP-PA and/or FTA-ABS algorithm. Their recommendation suggests using one type of serologic test (treponemal) instead of 2 (a treponemal and a nontreponemal) as suggested by earlier reports,10,30 but did not account for the low specificity of treponemal tests for previous infections. Second, the potential independence (or the lack thereof) of EIA/CA and Inno-Lia was not addressed.
Limitations of the Study
As with all models, the validity of the results depends, to a large extent, on the availability and reliability of data. We based our parameter values on estimates from peer-reviewed journals, expert opinion and conclusions from peer-reviewed scientific studies which varied somewhat. However, our comprehensive sensitivity analyses largely addressed this limitation. Our model did not account for events over time that would have included changes in behavior and transmission effects. Accounting for these events would require more assumptions because of the lack of adequate data on these parameters. It is difficult to assess how this omission affected our final results. Additionally, we ignored syphilis complications resulting from fetal and maternal transmission. To the extent that fetal and maternal transmission complications increase the expected cost of a case of syphilis, the Treponemal-First may be more cost-effective because it results in the treatment of almost all syphilis cases. Because of the lack of data, we used the cost of treatment as the cost of overtreatment. Thus, our study did not account for the full cost of overtreatment because using the cost of treatment ignored opportunity costs to the clinician and patients. Additionally, treatments have been shown to have adverse drug-related effects such as anaphylaxis from penicillin allergy.5 We also ignored costs associated with false diagnosis and treatment such as social stigma and the potential for tension and distrust in relationships. A higher cost of overtreatment would make the Nontreponemal-First algorithm relatively more cost-effective because it had a lower number of overtreatment. The average follow-up cost used may be too low, given that in some cases, additional costs resulting from activities such as medical record retrieval and health department registry queries would be incurred. A higher cost of follow-up would make the Nontreponemal-First algorithm relatively more cost-effective based on the estimated relative number of follow-ups.
Strengths of the Study
One of the strengths of this study is the inclusion of previous infections in the cohort. Although the values of performance of the tests for previous infections used were obtained from expert opinion, their inclusion provided a good reflection of the associated costs to the health care system because of the high number of follow-ups resulting from this category of the cohort. Second, we conducted comprehensive sensitivity analyses to assess how each of the variables might affect the final decision. Also, when compared with other cost-effectiveness studies that assessed syphilis testing algorithms, our study provided information on follow-ups and overtreatment rates. This is important because, in some settings, different entities (such as laboratories and health departments) may be responsible for different levels of the health services delivery process. Information on how the different algorithms affect the delivery of services and the associated cost structure can guide decision-making across laboratories and health departments.
Under our assumptions, the Treponemal-First algorithm has a higher net cost because it resulted in a higher number of follow-ups, in addition to substantially more overtreatment. The overtreatment was a function of the specificity of the EIA/CA for active disease and the lack of independence of the EIA/CA and TP-PA. The performance (specificity) of the available serological tests for syphilis depends on the history of the disease within the cohort, among others. To this end, future analyses of syphilis screening tests or algorithms should consider including or accounting for the proportion of the population with a history of syphilis or treponema disease. Although the results of our study are not entirely generalizable, the comprehensive sensitivity analyses provide insight into the potential qualitative (and quantitative) outcomes which can guide decision-making in different settings and under a wide range of scenarios. Given the data limitations mentioned in this study, more data are needed on the performance of the tests and the proportion of the population with a history of syphilis or treponematoses for future studies. Additionally, more research is needed to assess these algorithms in detail, accounting for the dynamic nature of the population, ongoing transmission, and the potential long-term outcomes. In this study, we assessed the algorithms using low prevalence as would be found in the United States, which implies that the positive predictive values are relatively low. Thus, further analyses should be conducted to assess the relative operational utilities of these algorithms in populations where prevalence is high.
1. Centers for Disease Control and Prevention. Syphilis testing algorithms using treponemal tests for initial screening–four laboratories, New York City, 2005–2006. Morb Mortal Wkly Rep 2008; 57:872–875.
2. Ebel A, Vanneste L, Cardinaels M, et al. Validation of the INNO-LIA syphilis kit as a confirmatory assay for Treponema pallidum
antibodies. J Clin Microbiol 2000; 38:215–219.
3. Aberle-Grasse J, Orton SL, Notari E, et al. Predictive value of past and current screening tests for syphilis in blood donors: Changing from a rapid plasma reagin test to an automated specific treponemal test for screening. Transfusion 1999; 39:206–211.
4. Rusnak JM, Butzin C, McGlasson D, et al. False-positive rapid plasma reagin tests in human-immunodeficiency-virus infection and relationship to anticardiolipin antibody and serum immunoglobulin levels. J Infect Dis 1994; 169:1356–1359.
5. Calonge N. United States Preventive Services Task Force. Screening for syphilis infection: Recommendation statement. Ann Fam Med 2004; 2:362–365.
6. Larsen SA, Pope V, Johnson RE, et al, eds. A manual of Tests for Syphilis, 9th ed. Arlington, VA: Am Public Health Association, 1998.
7. Egglestone SI, Turner AJ; PHLS Syphilis Serology Working Group. Serological diagnosis of syphilis. Commun Dis Publ Health 2000; 3:158–162.
8. Larsen SA, Steiner BM, Rudolph AH. Laboratory diagnosis and interpretation of tests for syphilis. Clin Microbiol Rev 1995; 8:1–21.
9. Young H. Syphilis–new diagnostic directions. Int J STD AIDS 1992; 3:391–413.
10. Centers for Disease Control and Prevention. Sexually transmitted disease treatment guidelines, 2006. MMWR Recomm Rep 2006; 55:1–94.
11. Lukehart SA. Biology of treponemes. In: Holmes KK, Sparling PF, Stamm WE, et al, eds, Sexually Transmitted Disease. New York, NY: McGraw Hill, 2008:647–659.
12. Owusu-Edusei K Jr, Koski KA, Ballard RC. The tale of two syphilis serologic tests–treponemal and nontreponemal: does the order matter? Paper presented at: 11th International Union against Sexually Transmitted Infections (IUSTI) World Congress, 2009, Cape Town, South Africa.
13. Clinical Diagnostic Laboratory Fee Schedule. Washington, DC: Centers for Medicare & Medicaid Services, 2009. Available at: http://www.cms.hhs.gov/apps/ama/license.asp?file=/ClinicalLabFeeSched/downloads/09CLAB.ZIP
. Accessed December 18, 2009.
14. Rydzak CE, Goldie SJ. Cost-effectiveness of rapid point-of-care prenatal syphilis screening in Sub-Saharan Africa. Sex Transm Dis 2008; 35:775–784.
15. Institute of Health Economics, Anderson C, Jacobs P, et al. Cost-Effectiveness in the Detection of Syphilis. Alberta, Canada: Institute of Health Economics, 2007.
16. Chuck A, Ohinmaa A, Tilley P, et al. Cost effectiveness of enzyme immunoassay and immunoblot testing for the diagnosis of syphilis. Int J STD AIDS 2008; 19:393–399.
17. Blandford JM, Gift TL, Vasaikar S, et al. Cost-effectiveness of on-site antenatal screening to prevent congenital syphilis in rural Eastern Cape Province, Republic of South Africa. Sex Transm Dis 2007; 34:S61–S66.
18. Clark EG, Danbolt N. The Oslo study of the natural history of untreated syphilis; an epidemiologic investigation based on a restudy of the Boeck-Bruusgaard material; a review and appraisal. Med Clin North Am 1964; 48:613.
19. Kraut JR, Haddix AC, Carande-Kulis V, et al. Cost-effectiveness of routine screening for sexually transmitted diseases among inmates in United States prisons and jails. In: National Commission on Correctional Health Care, ed. The Health Status of Soon-to-be Released Inmates: A Report to Congress. Chicago, IL: Commission on Correctional Health Care, 2002:81–108.
20. Chesson HW, Blandford JM, Gift TL, et al. The estimated direct medical cost of sexually transmitted diseases among American youth, 2000. Perspect Sex Reprod Health 2004; 36:11–19.
21. Gift TL, Malotte K, Ledsky R, et al. A cost-effectiveness analysis of interventions to increase repeat testing in patients treated for gonorrhea or chlamydia at public sexually transmitted disease clinics. Sex Transm Dis 2005; 32:542–549.
22. Clinical Laboratory Fee Schedule. Baltimore, MD: Centers for Medicare and Medicaid Services, 2009. Available at: http://www.cms.hhs.gov/ClinicalLabFeeSched/02_clinlab.asp#TopOfPage
. Accessed December 22, 2008.
23. Centers for Disease Control and Prevention. Sexually transmitted diseases treatment guidelines 2002. Centers for Disease Control and Prevention. MMWR Recomm Rep 2002; 51:1–78.
24. Blandford JM, Gift TL. The cost-effectiveness of single-dose azithromycin for treatment of incubating syphilis. Sex Transm Dis 2003; 30:502–508.
25. American Academy of Family Physician, Woodard C, Fisher MA. Drug Treatment of Common STDs: Part I. Herpes, Syphilis, Urethritis, Chlamydia and Gonorrhea. American Academy of Family Physician, 2008. Available at: http://www.aafp.org/afp/991001ap/1387.html
. Accessed February 10, 2008.
26. Consumer Price Indexes–all Urban Consumers. Washington, DC: United States Department of Labor. Available at: http://www.bls.gov/cpi/home.htm
. Accessed January 15, 2010.
27. Siegel JE, Weinstein MC, Russell LB, et al. Recommendations for reporting cost-effectiveness analyses. JAMA 1996; 276:1339–1341.
28. Siegel JE, Weinstein MC, Russell LB, et al. Consensus statement: Recommendations for reporting cost-effectiveness analyses. Pediatr AIDS HIV Infect Fetus Adolesc 1997; 8:130–134.
29. Weinstein MC, Siegel JE, Gold MR, et al. Recommendations of the panel on cost-effectiveness in health and medicine. JAMA 1996; 276:1253–1258.
30. Young H. Syphilis–serology. Dermatol Clin 1998; 16:691–698.