Approximately 1 in 5 US residents have a sexually transmitted infection (STI), with an estimated annual incidence of 26 million new cases.1 Trichomonas vaginalis (TV) infections were estimated to be at 2.6 million,1 Chlamydi a trachomatis (CT) and 616,392 Neisseria gonorrhoeae (NG) infections were reported in 2019.2 1
Nucleic acid amplification tests (NAATs) are the Centers for Disease Control and Prevention (CDC)–recommended standard to detect these STI pathogens.3 4,5 6,7
An accurate, rapid, point-of-care (POC) polymerase chain reaction (PCR) diagnostic test has been identified as a key innovation needed to mitigate the increasing rates of STIs.8–11 12
In this study, we investigated its potential to reduce misdiagnosis, inappropriate treatment, and delay of treatment in clinical practice. We also examined its acceptability, ease of use, and impact on staff time.
MATERIALS AND METHODS
Visby Medical sponsored a noninterventional, cross-sectional single-visit study of women aged 18 to 56 years, offering enrollment to consecutively presenting asymptomatic and symptomatic women. The study (ClinicalTrials.gov
Inclusion criteria were as follows: (1) symptomatic or asymptomatic for STIs, (2) willing and able to give voluntary written informed consent (or parent/legal guardian consent for minors), (3) female at birth (including pregnant/breastfeeding women), (4) age 14 years or older at the time of enrollment, (5) able to read and understand procedural information provided, and (6) able and willing to follow all study procedures, including self-collection of one vaginal swab. Exclusion criteria were as follows: (1) having a medical condition, serious intercurrent illness, or any other circumstance that, in the investigator’s judgment, could jeopardize the woman’s safety or could interfere with study procedures; (2) previous enrollment in this study, (3) and use of antiperspirants, deodorants, or certain vaginal products in the genital area within 48 hours before enrollment.
Investigational Materials
The Visby STI Panel houses a PCR assay for the qualitative detection of DNA from CT, NG, and TV from a single vaginal swab sample.12
Each patient was provided the Visby Medical Sexual Health Vaginal Specimen Collection Kit, which contains easy-to-follow instructions, a swab, and collection buffer vial. The patient self-collected a vaginal swab sample without the support of clinical personnel. The swab was placed into the provided collection vial and returned to the clinic staff who ran the Visby STI Panel.
Upon completion of the test, a light on the device turned green indicating that the results were ready. The results could be read immediately or up to 2 hours. An internal positive control monitored for effective sample preparation, PCR amplification, and detection. The test result was considered valid if the control produced a positive signal, and a green light was visible indicating that the test ran without error.
Clinical Procedures and User Evaluations
Standard-of-Care Tests
A licensed healthcare provider recorded signs of infection and, based on clinic practice, collected a vaginal swab or had the patient self-collect urine or vaginal swab. Swabs and urine were stored and transported to the external laboratory (LabCorp, Burlington, NC) for standard-of-care (SOC) testing. Urine was tested for CT/NG/TV using a NAAT, whereas vaginal swabs were tested using Vaginitis Plus (VG+) NuSwab (LabCorp) on the Panther System (Hologic, Marlborough, MA). Results from these SOC tests were compared with the Visby STI Panel results.
Visby STI Panel
Patients meeting the enrollment criteria were asked to provide consent. Consenting patients were further assessed for inclusion/exclusion criteria. Eligible participants self-collected vaginal swab samples after samples required for the SOC tests. An untrained clinical operator transferred the sample into the Visby STI Panel and ran the test using the provided Quick Reference Guide without coaching or instruction. Operators were instructed not to discuss results with clinical staff and were not allowed to observe or coach each other. Tests were run within 2 hours of sample collection. If the initial result was invalid (no signal in control window) or a device error occurred (no green check), the sample was rerun on a new test. A single retest was permitted. Test results and time to complete each test were recorded. The prescribing clinician was blinded to the Visby test results.
Workflow
On the day of the visit, presumptive diagnoses based on patient symptoms were recorded and physicians prescribed syndromic treatment. The clinic staff recorded the (1) number and type of clinical staff involved in STI SOC activities, (2) staff involved in explaining sample self-collection for the POC test, (3) visit duration and portion of visit spent on the setup of the Visby STI Panel (excluding run time for PCR analysis), and (4) time needed to explain sample collection, return the sample to clinic staff, and produce results. Subsequently, they recorded (1) turnaround time to receive SOC results, (2) time to review findings, and (3) number of attempts and time until findings were reported to the patient.
Participants were asked to complete a written survey on ease of use of the collection kit and on the perceived value of a fast POC test. Clinicians were surveyed twice about the perceived value of a fast POC test—before seeing the first enrolled patients and after study completion. Finally, operators were queried about the ease of use of the Visby STI Panel.
Statistical Methods
For each organism, we calculated the sensitivity, specificity, and overall agreement of the Visby STI Panel against the patient-infected status based on SOC with 95% Wilson confidence intervals. The study was not powered to meet 95% with a lower confidence bound of 90% as is typical. Calculations were made simply for illustration. Full analyses of sensitivity, specificity, and accuracy of the assay were performed previously.12
RESULTS
Fifty-five eligible patients participated in this study. Participant characteristics are shown in Table 1 . Based on clinic practice, vaginal samples (clinician or self-collected) or urine from all patients were submitted to LabCorp to obtain SOC test results (data from one SOC sample were not available). After samples for the SOC test were collected, an additional self-collected vaginal swab was collected by each patient using the provided collection kit and analyzed by clinic staff on the Visby STI Panel. Comparison of the results found that among the 162 paired analytical findings (54 per infection type), the Visby STI Panel recorded 2 CT false positives, and 1 NG and 1 TV false negative compared with the SOC device. The sensitivity, specificity, and accuracy of the Visby STI Panel are shown in Supplemental Table S1 (https://links.lww.com/OLQ/A778
TABLE 1 -
Characteristics of Enrolled Study Participants
n
Race
Asian
1
Native Hawaiian/Pacific Islander
1
Black/African American
37
White
13
Multiracial
2
Unknown
1
Age, y
18–25
38
26–35
13
36–45
2
46–55
1
>55
1
We investigated whether use of the Visby STI Panel, which provided test results during a patient's visit, could result in more appropriate treatment decisions than if the patient were presumptively treated. Table 2 shows the number of cases for which treatment was or was not provided for the 3 organisms. Seven CT, 2 NG, and 6 TV cases were not presumptively treated at the clinic visit. These cases were later shown to be positive by SOC laboratory results. In 13 of these 15 cases, the Visby STI Panel correctly identified the undertreated patients as infected at the time of the visit.
TABLE 2 -
Syndromic Treatment and SOC Laboratory Result
Syndromic Treatment vs. SOC*
SOC
Visby STI Panel Predictions of Overtreatment and Undertreatment
% Over/Undertreated Compared With SOC
Positive
Negative
CT
Syndromic treatment
Yes
3
14
Negative in 14/14 cases of overtreatments
82.4
No
7
30
Positive in 7/7 cases of undertreatments
18.9
NG
Syndromic treatment
Yes
1
11
Negative in 11/11 cases of overtreatments
91.7
No
2
40
Positive in 1/2 cases of undertreatments
4.8
TV
Syndromic treatment
Yes
1
8
Negative in 8/8 cases of overtreatments
88.9
No
6
39
Positive in 5/6 cases of undertreatments
13.3
*Laboratory results were missing for 1 patient.
Treatment decisions based on clinical symptoms were compared with SOC laboratory results for CT, NG, and TV. For the 3 organisms, treatment based on the rapid POC Visby STI Panel results could have prevented 33 of 33 cases of overtreatment and 13 of 15 cases of undertreatment.
CT indicates Chlamydi a trachomatis ; NG, Neisseria gonorrhoeae ; SOC, standard of care; STI, sexually transmitted infection; TV, Trichomonas vaginalis.
In addition, 14 CT, 11 NG, and 8 TV cases were treated presumptively at the clinic visit, before the availability of SOC results. The subsequent SOC results confirmed that these patients were not infected with CT, NG, or TV. In all 33 of these cases where patients were syndromically treated, the Visby STI Panel correctly identified the overtreated patients as uninfected at the time of the visit. The overtreated patients were most commonly prescribed azithromycin for CT, azithromycin + ceftriaxone for NG, and metronidazole for TV.
To assess the ease of integrating the Visby STI Panel into clinical practice, we measured the impact on staff time (Table 3 ). Staff spent slightly less time on the Visby STI Panel than on SOC procedures (12.3 ± 3.0 vs. 15 ± 7.5 minutes). Of this, an average of 4.6 minutes was required to explain self-collection procedures for the Visby STI Panel to patients. The remainder of the time was spent on informed-consent process and would not be a component of a typical visit. Patients queried about their experience using the Visby collection kit strongly agreed that it was easy for them to use, but 9% (5 of 55) felt that they needed the support of clinical staff (Fig. 1 ). If needed, clinics may need to support patients by explaining how to self-collect vaginal samples.13,14
TABLE 3 -
Time to Complete Clinical Procedures
Procedure/Event
n
Mean (Min)
SD
Median (Min)
Range (Min)
Visit duration (time from registration to end of care)
54
65.4
21.1
63.5
30–115
Staff time spent on SOC
31
15.0
7.5
15.0
2–45
Total staff time spent on Visby STI Panel
31
12.3
3.0
12.0
8–20
Staff time spent explaining the Visby STI Panel self-collection kit
31
4.6
2.4
5.0
1–11
Patient time spent collecting and returning vaginal swab
55
6.3
3.4
5.0
1–18
Time from swab returned until valid Visby STI Panel result
48
39.7
12.0
36.5
28–75
Time from swab returned until valid Visby STI Panel result if a retest is required
6
75.7
18.1
75
52–97
SOC indicates standard of care; STI, sexually transmitted infection.
Figure 1: Ease of use of sexual health test vaginal swab self-collection Kit. Surveys were administered to participants after use of the Visby STI Panel vaginal swab self-collection kit. A, Survey questions. B, Survey responses. n = 54 for Q6 and Q8; n = 55 for all other questions. Error bars indicate SEM.
Notably, explaining the Visby collection kit involved fewer staff (1.06, maximum of 2) than SOC (2.65, maximum of 4). The most common types of staff required to explain the Visby process were Licensed Practical Nurse or technicians. Very few physician assistants (3 of 31) were required to explain the Visby process as compared with the SOC process (22 of 31).
We also measured the total time required to complete the Visby STI Panel. On average, it took patients 6.3 ± 3.4 minutes to self-collect and return a vaginal swab, and the results of the initial Visby STI Panel were ready 39.7 ± 12.0 minutes later (Table 3 ). For the 6 of 55 tests that had to be repeated because of control failure (4) or device error (2), the mean time to a valid test was 75.7 ± 18.1 minutes.
Because the Visby STI Panel could dramatically reduce syndromic treatment errors (Table 2 ), we investigated how quickly treatment could be initiated with a Visby test result. In contrast to the mean of <45 minutes to obtain Visby STI Panel results, it took an average of 6 days (6.0 ± 3.4 days; range of 1–18 days) for the clinic to receive SOC results. It then typically required 2 (1.9 ± 1.0) contact attempts to notify patients who received test results 7 (7.1 ± 3.5) days after their clinic visit (Table 4 ). In this setting, all patients were successfully contacted, and the correct prescription sent to the pharmacy for pickup or a follow-up appointment was made if the patient required an injection.
TABLE 4 -
Time Related to SOC Tests
Procedure/Event
n
Mean
SD
Median
Range
Time to receive SOC test results at clinic, d
54
6.0
3.4
5.0
1–18
Number of Contact attempts by clinic staff, count
55
1.9
1.0
2.0
1–5
Average total time to communicate SOC test results, min
55
4.7
2.7
4.0
2–17
Average time from clinic visit to final patient notification of SOC test results, d
55
7.1
3.5
6.0
2–19
SOC indicates standard of care.
Patients strongly agreed that they preferred receiving laboratory results and treatment at the initial visit rather than waiting for several days (Fig. 2 ). Also, 93% (51 of 55) patients agreed or strongly agreed that they would be fine waiting an extra 30 minutes for same-day results, and 61.8% (34 of 55) were willing to wait an extra 60 minutes (Fig. 2 ). Most (81% [22 of 27]) patients also agreed or strongly agreed that they would prefer not to take unnecessary antibiotics.
Figure 2: Value of same-day test results to patient. Surveys were administered to participants after use of the Visby vaginal swab self-collection kit. A, Survey questions. B, Survey responses. n = 27 for Q6 and Q7; n = 55 for all other questions. Error bars indicate SEM.
We surveyed 3 operators about their experience (Fig. 3 ) to understand whether the clinical acceptance of the Visby STI Panel could be improved. Overall, operators reported positive experiences, all strongly agreeing that they would use the test in clinical practice. All operators strongly agreed that the device was easy to use with the provided instructions and that most people would quickly learn to use the device.
Figure 3: Operator comfort performing the Visby STI Panel. Clinical staff were given the Visby STI Panel test and the included instructions. No additional training or resources were provided. Operators were surveyed on ease of use of the device upon completion of the study. A, Survey questions. B, Survey responses. n = 3. Error bars indicate SEM.
Finally, we evaluated the potential value of same-day test-and-treat to providers (Fig. 4 ). Five clinicians completed the prestudy, and 3 completed the poststudy survey. Attitudes for individual clinicians changed little, although after the study, 1 of the 3 was more concerned about the willingness of patients to wait 30 to 60 minutes for same-day test results. One of the 3 remained less worried about syndromically treating patients, even though syndromic treatment did not correlate with SOC laboratory (or Visby STI Panel) results for 39% of CT, 24% of NG, and 26% of TV patients.
Figure 4: Provider attitudes related to same-day PCR testing. Providers who were responsible for treatment decisions were surveyed before beginning (n = 5) and after completing (n = 3) the Visby STI Panel study. A, Survey questions. B, Survey responses. Error bars indicate SEM.
DISCUSSION
Undertreatment and overtreatment have real-world consequences. Administering the appropriate treatment early in the course of disease can shorten the duration of infection for the patient. Untreated STIs lead to serious health consequences including pelvic inflammatory disease, chronic abdominal pain, infertility, ectopic pregnancy, neonatal death, and congenital abnormalities.2 15,16 17 18
Consistent with prior studies, our data show a disturbingly high rate of both overtreatment and undertreatment of STIs in syndromically treated patients.19 Table 2 ) are in line with 85% to 86% overtreatment rates reported by others.20,21
Providing clinicians with the test results at the time that they need to make a clinical decision could inform treatment decisions. Findings here suggest that the Visby STI Panel may make that possible. As previously described, the Visby STI Panel performs with high sensitivity, specificity, and accuracy compared with the SOC. Had the clinicians based their treatment on the Visby STI Panel results, all 33 of the overtreatments would have been prevented and 87% (13 of 15) of the undertreatments avoided. There would have been only 2 cases of overtreatment of CT and 1 case of undertreatment each for NG and TV. In low-prevalence settings, where pretest probability is low and positive test results are not supported by sexual history data or clinical findings, health care providers may wish to retest with the same or different NAAT.22
Although patients generally found the vaginal swab self-collection kit easy to use, as in other studies,23
All operators felt the device was easy to use, were confident using it, and were able to run the test with only the provided instructional material. All of them strongly agreed that they would use the Visby STI Panel in their current practice. The initial invalid test rate was 11% in this study (higher than the 6.5% rate in an earlier large trial12
In the future, clinics will need to establish protocols to integrate testing into workflows to minimize its impact on visit duration and staff time. Patients are willing to wait 30 minutes for same-day test results that facilitate earlier and more appropriate treatment, but less willing to wait longer.23,24 25
Impact on staff time is also a concern. The time staff spent following up with patients on laboratory results was, on average, 4.7 minutes and 1.9 contact attempts (Table 4 ). If the patient left the clinic with the right diagnosis, staff will not need to follow up with the patients, making up for the time needed to run the Visby test in the clinic.
Irrespective of World Health Organization and CDC recommendations for POC STI testing, providers must recognize its benefits to their own patients to drive adoption into clinical practice. Our participating clinicians and physician assistants valued immediate test results, would consider using single-visit test results to customize treatment, were concerned about treating patients syndromically, and considered themselves advocates for antibiotic stewardship. They were also concerned about losing contact with patients once they leave the clinic. In this study, however, the provider was able to successfully contact patients that required a follow-up visit, and this may not be the case in many settings where patients are lost to care.
Our study is not without limitations. First, because the test was not Food and Drug Administration cleared at the time of the study, only a theoretical outcome study was possible and important parameters like “how much time does the Visby test add to the patient visit” were not studied. Second, because of the COVID-19 pandemic, the number of tests completed and clinicians interviewed were far fewer than the original intent of the study. The data are not powered to meet 95% with a lower confidence bound of 90% as is typical. Sensitivity and specificity calculations were made simply for illustration. Third, patient samples with discrepant results, that is, Visby result did not agree with SOC results, were not available to retest on a tiebreaker assay to determine the patient infected status. Fourth, the average time to return SOC results was 6.0 ± 3.4 days (range of 1–18 days). Part of this study took place during the pandemic where laboratories were experiencing severe supply chain and personnel shortages. This could account for the lengthy lead times to receive SOC test results. Lastly, the data reported in this article are derived from a single urgent care center and reflect the individuals in the surrounding community that use this urgent care. This patient demographic may be different between urgent cares and geographic regions, and such data may not be generalizable across all urgent cares.
Overall, our findings demonstrated that clinical adoption of the Visby STI Panel could dramatically reduce overtreatment and undertreatment of STIs providing benefit to the patient, while breaking the transmission cycle otherwise perpetuated by delayed or wrong treatment decisions. Patients seemed to have little difficulty providing specimens and operators in using the device. Both patients and clinicians appreciated same-day laboratory results, and if integrated efficiently into the clinical workflow, the Visby STI Panel should have minimal impact on overall staff time and clinic turnaround time for a patient visit, and be within acceptable limits of waiting time for patients.
REFERENCES
1. US Centers for Disease Control and Prevention. Sexually Transmitted Infections Prevalence, Incidence, and Cost Estimates in the United States [Internet]. Available at:
https://www.cdc.gov/std/statistics/prevalence-2020-at-a-glance.htm . Accessed September 8, 2021.
2. US Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance 2019 [Internet]. Available at:
https://www.cdc.gov/std/statistics/2019/overview.htm . Accessed September 8, 2021.
3. Workowski KA, Bachmann LH, Chan PA, et al. Sexually transmitted infections treatment guidelines, 2021. MMWR Recomm Rep 2021; 70:1–187.
4. Unemo M, Bradshaw CS, Hocking JS, et al. Sexually transmitted infections: Challenges ahead. Lancet Infect Dis 2017; 17:e235–e279.
5. Tien V, Punjabi C, Holubar MK. Antimicrobial resistance in sexually transmitted infections. J Travel Med 2020; 27:taz101.
6. Filice GA, Drekonja DM, Thurn JR, et al. Diagnostic errors that lead to inappropriate antimicrobial use. Infect Control Hosp Epidemiol 2015; 36:949–956.
7. Huppert JS, Reed JL, Munafo JK, et al. Improving notification of sexually transmitted infections: A quality improvement project and planned experiment. Pediatrics 2012; 130:e415–e422.
8. Eisinger RW, Erbelding E, Fauci AS. Refocusing research on sexually transmitted infections. J Infect Dis 2020; 222:1432–1434.
9. Cristillo AD, Bristow CC, Peeling R, et al. Point-of-care sexually transmitted infection diagnostics: Proceedings of the STAR Sexually Transmitted Infection–Clinical Trial Group programmatic meeting. Sex Transm Dis 2017; 44:211–218.
10. Toskin I, Peeling RW, Mabey D, et al. Point-of-care tests for STIs: The way forward. Sex Transm Infect 2017; 93(S4):S1–S2.
11. National Academies of Sciences, Engineering and Medicine. Sexually Transmitted Infections: Adopting a Sexual Health Paradigm [Internet]. Vol. 12, Kartografija i geoinformacije (Cartography and Geoinformation). Washington, DC: National Academies Press, 2021. 750 p. Available at:
https://doi.org/10.17226/25955 .
12. Morris SR, Bristow CC, Wierzbicki MR, et al. Performance of a single-use, rapid, point-of-care PCR device for the detection of
Neisseria gonorrhoeae ,
Chlamydia trachomatis , and
Trichomonas Vaginalis : A cross-sectional study. Lancet Infect Dis 2021; 21:668–676.
13. Schachter J, Chernesky MA, Willis DE, et al. Vaginal swabs are the specimens of choice when screening for
Chlamydia trachomatis and
Neisseria gonorrhoeae : Results from a multicenter evaluation of the APTIMA assays for both infections. Sex Transm Dis 2005; 32:725–728.
14. Gaydos CA, Barnes M, Holden J, et al. Acceptability and feasibility of recruiting women to collect a self-administered vaginal swab at a pharmacy clinic for sexually transmissible infection screening. Sex Health 2020; 17:392–394.
15. Chesson HW, Blandford JM, Pinkerton SD. Estimates of the annual number and cost of new HIV infections among women attributable to trichomoniasis in the United States. Sex Transm Dis 2004; 31:547–551.
16. Sexton J, Garnett G, Røttingen J-A. Metaanalysis and metaregression in interpreting study variability in the impact of sexually transmitted diseases on susceptibility to HIV infection. Sex Transm Dis 2005; 32:351–357.
17. Rönn MM, Menzies NA, Gift TL, et al. Potential for point-of-care tests to reduce chlamydia-associated burden in the United States: A mathematical modeling analysis. Clin Infect Dis 2020; 70:1816–1823.
18. Morehead MS, Scarbrough C. Emergence of global antibiotic resistance. Prim Care 2018; 45:467–484.
19. Gaydos CA, Ako M-C, Lewis M, et al. Use of a rapid diagnostic for
Chlamydi a
trachomatis and
Neisseria gonorrhoeae for women in the emergency department can improve clinical management: Report of a randomized clinical trial. Ann Emerg Med 2019; 74:36–44.
20. Holley CE, Van Pham T, Mezzadra HM, et al. Overtreatment of gonorrhea and chlamydial infections in 2 inner-city emergency departments. Am J Emerg Med 2015; 33:1265–1268.
21. Sonkar SC, Wasnik K, Kumar A, et al. Comparative analysis of syndromic and PCR-based diagnostic assay reveals misdiagnosis/overtreatment for trichomoniasis based on subjective judgment in symptomatic patients. Infect Dis Poverty 2016; 5:42.
22. Katz AR, Effler PV, Ohye RG, et al. False-positive gonorrhea test results with a nucleic acid amplification test: The impact of low prevalence on positive predictive value. Clin Infect Dis 2004; 38:814–819.
23. Widdice LE, Hsieh YH, Silver B, et al. Performance of the Atlas genetics rapid test for
Chlamydia trachomatis and women's attitudes toward point-of-care testing. Sex Transm Dis 2018; 45:723–727.
24. Gettinger J, Van Wagoner N, Daniels B, et al. Patients are willing to wait for rapid sexually transmitted infection results in a university student health clinic. Sex Transm Dis 2020; 47:67–69.
25. Gaydos CA, Van Der Pol B, Jett-Goheen M, et al. Performance of the Cepheid CT/NG Xpert rapid PCR test for detection of
Chlamydia trachomatis and
Neisseria gonorrhoeae . J Clin Microbiol 2013; 51:1666–1672.
