Secondary Logo

Journal Logo

Original Studies

Challenges of Rapid Plasma Reagin Interpretation in Syphilis Screening in Uganda: Variability in Nontreponemal Results Between Different Laboratories

Hamill, Matthew M. MBChB, PhD*; Mbazira, Kimeze J. DipClinMed; Kiragga, Agnes N. PhD; Gaydos, Charlotte A. DrPH, MPH*; Jett-Goheen, Mary MT (ASCP)*; Parkes-Ratanshi, Rosalind MBBS, PhD‡§; Manabe, Yukari C. MD*; Nakku-Joloba, Edith MBChB, PhD; Rompalo, Anne MD, ScM*

Author Information
doi: 10.1097/OLQ.0000000000000883
  • Free

The World Health Organization (WHO) estimated that in 2012 there were 5.6 million cases of syphilis, in 2015, the median case rate of syphilis per 100,000 adults was 25.1 (range, 0.1–1664). In Uganda, that rate was reported as 372.8 in 2014.1 There were an estimated 350,000 annual global adverse pregnancy outcomes due to mother-to-child transmission of syphilis.1 Consequently, the diagnosis, treatment, and prevention of syphilis are global health priorities.

Syphilis is a major global infectious disease with huge individual and public health consequences2 including congenital syphilis and increased potential for sexual transmission of human immunodeficiency virus (HIV),3 Ugandan national guidelines specifically highlight the additional risk of HIV infection posed by genital ulcer disease4; such “epidemiological synergy” increases the transmission potential for HIV.5 Without screening for and treatment of syphilis it is estimated between 53–82% of infected women will have poor pregnancy outcomes including neonatal death, and congenital syphilis6; this is 4 times the rate in syphilis uninfected women.6

Treponema pallidum (TP) has remained fully sensitive to penicillin therapy, therefore, failures to control the spread of infection, both sexually and mother-to-child,7 largely represent failures in public health and health systems approaches. Like other Sub-Saharan African (SSA) countries Uganda has high rates of syphilis with correspondingly high prevalence of antenatal syphilis. Although many nations in SSA have seen decreases in the antenatal syphilis prevalence, Uganda remains a high-prevalence country.8

In a 2015 study of 43 SSA countries, it was estimated that Uganda had a syphilis prevalence of 3.0% (based on 2004 data),9 with estimated adverse pregnancy outcomes of 10,670 (5982–19,633) attributable to missed cases of active syphilis.

Ugandan 2016 National guidelines recommend rapid plasma reagin (RPR) testing followed by TP Hemagglutination (TPHA) to confirm a positive nontreponemal test.4 The WHO highlight the benefits of treponemal rapid diagnostic tests,10 for example, treponemal lateral flow to screen for syphilis, and this approach is increasingly used in Antenatal (ANC) and Sexually Transmitted Disease (STD) clinic settings. In this context the treponemal rapid diagnostic test is confirmed using the RPR nontreponemal test. Rapid plasma reagin is used in the diagnosis and assessment of treatment response in syphilis infection in health settings globally, and is currently the best available test for assessing serological response to treatment and indicating reinfection and/or treatment failure. It is based on antigens that contain cardiolipin, lecithin and cholesterol which cause visible flocculation of positive sera containing IgM and/or IgG antibodies to lipoidal antigens that are putatively released from bacteria. However, since these antigens are also normal components of human cells it is not possible to determine if the RPR measures products released by TP subspecies pallidum, the causative agent of syphilis, or damaged host cells or both.11 Seroconversion from RPR-negative to -positive generally occurs within 3 weeks of exposure to TP but may be delayed up to 6 weeks.12 The advantages of the RPR is that it is relatively inexpensive and easy to perform, and can provide titers (by dilution) to aid initial diagnosis of disease activity and against which to monitor treatment response. There are several disadvantages particularly in low and middle income country (LMIC) settings, specifically, the tests require an electricity supply, refrigeration for antigen storage, a light source, constant humidity,13 temperature monitoring, and trained personnel.13 Laboratory variation14 and false-positive reactions are relatively common in the context of pregnancy, aging, autoimmune conditions, and other infectious disease, for example, HIV.15 Rapid plasma reagin may vary by 1 serial dilution on repeated testing, so titer changes of less than 2 serial dilutions (a 4-fold change) are, according to Hook and Marra, rarely clinically “meaningful.”16 The RPR is prone to false positive results15; for this reason it is recommended that a specific treponemal test (e.g., TP Particle- or Hem- Agglutination TPPA/TPHA) is used in an algorithmic approach with RPR to avoid false positive12 and minimize false-negative results.

An RPR product insert highlights difficulties in initial interpretation; there are only 2 categories: reactive and nonreactive (rather than allowing for inferences made about titers), it is only with dilution that a titer is generated. However, it is occasionally necessary to repeat the test on a different platform or to report as “indeterminate” pending further evaluation. It also cautions about the susceptibility of the antigen to bright sunlight and freeze-thaw cyles.17 There are reports of wide variations between and within laboratories; a 2009 article by Gupta et al. examined 26 microbiology laboratories testing 138 sera panels and demonstrated interlaboratory and intralaboratory variations of 58.7% and 53.1%, respectively, by RPR when compared with the reference laboratory.18 In the described large external quality control assurance (EQA) study, the authors identified 7 key errors ranging from improper interpretation of RPR results to delayed testing and laboratory temperatures.18

The analysis presented in this article compared the performance of RPR both against a treponemal test and across different laboratories in Kampala, Uganda and in Baltimore, MD.

METHODS

A full description of the study from which the sera were drawn can be found in Nakku-Joloba et al.19 Briefly, the study was undertaken between February 2012 and June 2013. The population were adults identified in ANC and STD clinics at Mulago National Tertiary Referral Hospital, Kampala, Uganda. The parent study offered inclusion and enrollment of a minimum of 100 participants with negative followed by 100 participants with positive syphilis serological tests (Fig. 1). Potential participants were offered information about the study by trained research assistants and written informed consent was obtained.

Figure 1
Figure 1:
Flow chart of samples and testing.

Participants were screened using the Ugandan national standard of care algorithm for syphilis which consisted of nontreponemal RPR testing (Carbon, Cypress Diagnostics, Langdorp Belgium) followed by treponemal test (TPAb, ABON Syphilis Ultra Rapid Test) when the RPR was reactive. The treponemal test uses a rapid chromatographic immunoassay platform for the qualitative detection of antibodies (IgG and IgM) to TP. Blood was tested at the local onsite ANC/STD clinic (A) for RPR reactivity and sera were batched for repeat RPR testing with titers at laboratories B, C, and D (Fig. 1). Other than the local ANC/STD onsite laboratories, the other laboratories included; an International STD reference laboratory (in the USA) and 2 College of American Pathologist (CAP) Certified clinical laboratories (in Uganda). Technicians who perform RPR testing undergo a period of 6 months initial training in serological techniques, are then assessed using CAP EQA RPR samples followed by annual assessment. Laboratories B and D initially diluted sera up to a maximum dilution of 1:16 therefore, there may have been higher RPR titers within the samples. One laboratory served as a ‘tie-breaker’ following discrepant results in initial RPR results and when RPR titers were > 2 fold different between the other laboratories (Fig. 1).

Analysis

The primary outcome for this analysis was the difference between the RPR positivity between laboratories A, B, and C; when discrepant results arose, samples were tested at laboratory D. A χ2 statistic was used to test for differences in the primary outcome. Secondary outcomes were agreement in RPR titer, allowing for 2-fold or less variation, between the sets of tests, the difference and equivalence between laboratories B and C results with those from reference laboratory D.

Because of the nature of the data, a log2 transformation was conducted to normalize the data; when RPR is “negative”, the titer score is 0, therefore, a very small pseudo-number (0.0001) was added to these values. χ2 Test was used to evaluate whether scores from these 3 clinics were statistically different. This allowed us to test if the difference in transformed scores is different from zero. Secondary endpoints were further analyzed using the 2 one-sided test (TOST) approach, which allows testing for equivalence, as distinct from difference.

The t-test looks at whether the confidence interval (CI) includes 0, and if it does not, the t-test states that the 2 clinics give different scores. However, the equivalence test looks at whether the confidence interval goes beyond the acceptable range (log2(2) = 1, therefore, is [−1, 1]).

RESULTS

Of the participants, 144 (66.9%) were women, and their median age was 26 years (interquartile range, 22-32). In a separate analysis, 110 of 215 sera were determined to be RPR/TPAb, ABON/TPHA positive, and 105 RPR/TPHA negative as part of the parent study (Fig. 1); these tests were conducted at the Makerere University Medical Microbiology Laboratory19 (data not shown).

Primary Outcome

Of 215 sera, 97 (45.1%), 81 (37.7%), and 65 (30.2%) were RPR reactive in laboratories A, B, and C, respectively. All reported RPR positives in laboratory C were positive in laboratories A and B (Table 1). χ2 Test was highly significant (χ2 = 150.35, df = 1, P = <0.001) for difference. χ2 Test was highly significant (P = <0.001) for difference between each dyad of laboratories (A and B, A and C, and B and C) RPR results (Table 1).

TABLE 1
TABLE 1:
Comparison of Laboratories A, B, and C Results

A χ2 test was conducted to evaluate the differences in RPR positivity: A and B gave very different RPR scores (χ2 = 121.4, df = 1, P <2.2e-16), as did A and C (χ2 = 92.2, df = 1, P <2.2e-16) and B and C (χ2 = 150.35, df = 1, P <2.2e-16).

Of samples, with discrepant results 49 were rerun in laboratory D, laboratory D was selected as a ‘tie-breaker’ to adjudicate discrepant results after comparisons between laboratories A, B, and C. Discrepancies were classed as lack of RPR positivity between laboratories A, B, and C or a 2-fold or greater difference in RPR titer between laboratories B and C (Table 2).

TABLE 2
TABLE 2:
Comparison of Discrepant Laboratories A, B, C Compared With Laboratory D Results

Agreement between laboratories A/D, B/D, and C/D was 47 (95.9%) of 49, 48 (98.0%) of 49, and 34 (69.4%) of 49, respectively. Laboratory C was statistically different from the others (P < =0.001), whereas laboratories A, B, and D were not.

Secondary Outcomes

There were significant differences between RPR titers by paired t test and Wilcox rank test (P = <0.001) between laboratories B/C, and C/D; laboratory B RPR being 3.2 times higher, on average, than laboratory C. The TOST approach demonstrated nonequivalence. Using the TOST approach, we demonstrated that the laboratories were not equivalent; equivalence defined as within 2-fold change (Fig. 2).

Figure 2
Figure 2:
Comparing difference and equivalence of laboratory RPR titer scores—Panels I, II, and III.

If we can accept that as long as the difference is within 2-fold change, the equivalence test states the 2 clinics (B and C) did not give equivalent scores (Fig. 2), I. Laboratories B and D gave equivalent titer scores: paired t test and Wilcox rank test showed that there were no significant differences between these 2 results. Tests for equivalence test indicated that the 2 gave equivalent titers (Fig. 2), II. Laboratories C and D reported significantly different RPR titers, paired t test and Wilcox rank test (P < 0.001). Further, the 2 clinics were not equivalent (Fig. 2), III.

DISCUSSION

Syphilis diagnosis and treatment are urgent and ongoing priorities for improving maternal and child outcomes and those with STD in SSA. The discrepancies in RPR results highlight the need for ongoing regular EQA, staff training, and quality standards as well a need to better understand underpinning host immune responses to syphilis. The 3.2 -fold differences in RPR titers between laboratories B and C is of potential clinical relevance particularly because higher titers increase the risk of congenital syphilis and poor fetal/infant outcomes6 and more serious syphilis sequelae20; in practice, this might change an RPR profile from 1:2 to >1:16. Such inconsistency in titers is a key limitation of the RPR test because it may result in either overtreatment of adequately treated past infection or, more problematically, undertreatment of active disease or treatment failures. Of the positive treponemal-specific TPHA between 65 and 97/110 (59.1%–88.2%) were nontreponemal test (RPR) reactive depending on the laboratory. This may indicate:( i) poor performance of either test; (ii) false-positive treponemal results; (iii) false-negative nontreponemal tests; (iv) late/inactive syphilis with negative RPR; (v) prior infection, treated or untreated (with RPR titer reversion); and (vi) training and quality assurance issues. Although this analysis is not able to distinguish the cause of these discrepancies, the data highlight the need for combined treponemal and nontreponemal testing in the ANC/STD clinic setting.

There has been a call to automate nontreponemal testing to overcome some of the issues seen in RPR variability and labor costs. Some argue that the problem of objectivity and reproducibility are inherent to the manual method of RPR that might be overcome using automation21 but others describe inconsistent results with automated RPR assessment22 particularly as a way of monitoring treatment response; comparative large scale trials are lacking.23 Other suggestions include the use of paired RPR testing of acute and convalescent samples in the context of repeated infection and for follow up.24 This may overcome some of the intralaboratory variation but lacks utility in the diagnosis of new infections. It may also be cost prohibitive in resource limited settings, such as Uganda.

Possible explanations for the discrepancies in RPR data in general include the use of different RPR card tests (BD Macro-Vue RPR and Carbon; Cypress Diagnostics) in different laboratories, environment (temperature and humidity), personnel, endemic treponemal disease, HIV status, laboratory variation and the effects of freeze-thawing of serum; storage and transportation. Because there is only 1 laboratory (C) that had a statistically significant difference to the others, it could be argued that this is an outlier. However, laboratories B, C, and D are all CAP certified/accredited and undergo external proficiency testing and on-going daily internal quality assurance procedures. This should minimize the effects of environmental conditions and laboratory variation; however, this study suggests that other factors are having an impact on these results. As highlighted by the RPR manufacturers, the reading of these tests has a degree of subjectivity. The laboratories in the capital city, Kampala, all have good access to water, electricity, highly qualified personnel, and good laboratory infrastructure. If the RPR is underperforming in these settings, it is highly unlikely to give reproducible results in other less well-resourced laboratories outside of Kampala.

Limitations of the current study include a lack of dark field microscopy or TP PCR data which meant raising the possibility of missed early infections before serological positivity. The parent study19 did not set out to compare laboratory or test performance, so we did not have data on all samples collected. The sample size was relatively small, the study population lacked descriptive characteristics, such as HIV prevalence or assessment of nonsyphilis treponemal disease which may have resulted in false positive results. Stored sera underwent freeze/thaw cycles which may have affected results, and not all samples had contemporaneous treponemal testing. Because these results are from patients attending ANC and STD clinics in Kampala, they are not generalizable to other populations; therefore, larger studies in various populations are warranted to ascertain performance of RPR tests in different settings. It would also be interesting to characterize the phenotype of patients with an isolated positive RPR and ascertain follow-up to assess maintenance, evolution or regression of syphilis serology, development of connective tissue disease or other infection, and rates of syphilis infection in their sexual contacts. Future work could involve repeat, simultaneous testing of all samples with treponemal/nontreponemal tests in a reference laboratory.

To conclude, these data demonstrate significant differences in RPR results (reactive vs nonreactive) between all 3 laboratories measuring RPR and also demonstrate significant difference in laboratories where RPR titer was measured. Further work is required to better understand these differences and ensure the most accurate and reproducible results in the future. They also serve to remind the global health community of the limitations of the RPR as a test for syphilis diagnosis. This is especially important in LMIC where there is a high burden of syphilis associated morbidity, in an environment with minimal access to equipment and laboratory support. Important future developments could include a better, more reliable, and affordable dual point-of-care tests (POCT) for syphilis diagnosis than those currently available which can have reduced sensitivity compared to serological tests. This would have the double advantage of fewer false negatives in early syphilis and reduced overtreatment of previously treated infection. A recent meta-analysis of dual path platform POCT assessing both treponemal and nontreponemal antibodies for syphilis is encouraging with good sensitivity and specificity25; however, data are urgently needed in ANC/STD settings in Africa and in HIV-positive populations generally. Although the diagnosis issue may be resolved with a dual treponemal/nontreponemal syphilis POCT, the evaluation of treatment response and possible reinfection (including need for retreatment during pregnancy) is a challenging one, and RPR titers are routinely used globally to determine if retreatment is needed. The data generated from this analysis suggest that RPR results need to be interpreted with utmost caution in a Ugandan setting. In recent years, driven by the WHO, there has been a renewed interest in syphilis diagnostics and POCT technology, this should bode well for the future development of more sensitive, reliable and affordable near patient syphilis tests in LMIC.26

REFERENCES

1. World Health Organization (WHO). Report on global sexually transmitted infection surveillance 2015. 2016. Available at: http://apps.who.int/iris/bitstream/10665/249553/1/9789241565301-eng.pdf?ua=1. Accessed December 13, 2017.
2. Hook HW III. Syphilis. Lancet 2017; 389:1550–1557.
3. Greenblatt RM, Lukehart SA, Plummer FA, et al. Genital ulceration as a risk factor for human immunodeficiency virus infection. AIDS 1988; 2:47–50.
4. Uganda Clinical Guidelines 2016: National Guidelines for Management of Common ConditionsUganda: 2016269-76. Available at: http://health.go.ug/sites/default/files/Uganda%20Clinical%20Guidelines%202016_FINAL.pdf. Accessed December 13, 2017.
5. Wasserheit JN. Epidemiological synergy. Interrelationships between human immunodeficiency virus infection and other sexually transmitted diseases. Sex Transm Dis 1992; 19:61–77.
6. Gomez GB, Kamb ML, Newman LM, et al. Untreated maternal syphilis and adverse outcomes of pregnancy: A systematic review and meta-analysis. Bull World Health Organ 2013; 91:217–226.
7. Newman L, Kamb M, Hawkes S, et al. Global estimates of syphilis in pregnancy and associated adverse outcomes: Analysis of multinational antenatal surveillance data. PLoS Med 2013; 10:e1001396.
8. Kenyon CR, Osbak K, Tsoumanis A. The global epidemiology of syphilis in the past century—a systematic review based on antenatal syphilis prevalence. PLoS Negl Trop Dis 2016; 10:e0004711.
9. Kuznik A, Habib AG, Manabe YC, et al. Estimating the public health burden associated with adverse pregnancy outcomes resulting from syphilis infection across 43 countries in Sub-Saharan Africa. Sex Transm Dis 2015; 42:369–375.
10. World Health Organization (WHO). WHO Guidelines for the treatment of Treponema pallidum (syphilis). 2016.
11. Stamm LV. Syphilis: Antibiotic treatment and resistance. Epidemiol Infect 2015; 143:1567–1574.
12. Seña AC, White BL, Sparling PF. Novel Treponema pallidum serologic tests: A paradigm shift in syphilis screening for the 21st century. Clin Infect Dis 2010; 51:700–708.
13. Pope V, Ari MD, Schriefer ME, et al. Immunologic Methods for Diagnosis of Spirochetal Disease. In: Detrick B, Hamilton RG, Folds JD, eds. Manual of Molecular and Clinical Laboratory Immunology. Washington, D.C.: ASM Press, 2006:477–492.
14. Centers for Disease Control and Prevention (CDC). Discordant results from reverse sequence syphilis screening–five laboratories, United States, 2006–2010. MMWR Morb Mortal Wkly Rep 2011; 60:133–137.
15. Lafond RE, Lukehart SA. Biological basis for syphilis. Clin Microbiol Rev 2006; 19:29–49.
16. Hook EW 3rd, Marra CM. Acquired syphilis in adults. N Engl J Med 1992; 326:1060–1069.
17. Becton Dickinson. (BD). Macro-Vue TM RPR Card tests 18 mm Circle Qualitative and Quantitative Brewer Diagnostic Kit for the Serological Detection of Syphilis, 2015.
18. Gupta SM, Bala M, Muralidhar S, et al. Evaluation of test results of microbiology laboratories of North India for standard tests for syphilis under an external quality assurance scheme. Eur J Clin Microbiol Infect Dis 2009; 28:461–468.
19. Nakku-Joloba E, Kiragga A, Mbazira JK, et al. Clinical evaluation of 2 point-of-care lateral flow tests for the diagnosis of syphilis. Sex Transm Dis 2016; 43:623–625.
20. Marra CM, Maxwell CL, Smith SL, et al. Cerebrospinal fluid abnormalities in patients with syphilis: Association with clinical and laboratory features. J Infect Dis 2004; 189:369–376.
21. Onoe T, Honda M, Matsuo K, et al. Examination of the correlation between the manual and automated serological testing methods for syphilis. J Dermatol 2012; 39:355–361.
22. Kim YS, Lee J, Lee HK, et al. Comparison of quantitative results among two automated rapid plasma reagin (RPR) assays and a manual RPR test. Korean J Lab Med 2009; 29:331–337.
23. Osbak K, Abdellati S, Tsoumanis A, et al. Evaluation of an automated quantitative latex immunoturbidimetric non-treponemal assay for diagnosis and follow-up of syphilis: A prospective cohort study. J Med Microbiol 2017.
24. Kenyon CR, Osbak KK, Van Esbroek M, et al. What is the role of paired rapid plasma reagin testing (simultaneous testing of acute and convalescent samples) in the diagnosis of repeat syphilis and the follow-up of syphilis? Sex Transm Dis 2017; 1.
25. Marks M, Yin YP, Chen XS, et al. Metaanalysis of the performance of a combined treponemal and nontreponemal rapid diagnostic test for syphilis and yaws. Clin Infect Dis 2016; 63:627–633.
26. Gliddon HD, Peeling RW, Kamb ML, et al. A systematic review and meta-analysis of studies evaluating the performance and operational characteristics of dual point-of-care tests for HIV and syphilis. Sex Transm Infect 2017; 93:S3–S15.
Copyright © 2018 American Sexually Transmitted Diseases Association All rights reserved.