Validation of a Clinical Prediction Rule to Predict Asymptomatic Chlamydia and Gonorrhea Infections Among Internet-Based Testers : Sexually Transmitted Diseases

Secondary Logo

Journal Logo

Original Studies

Validation of a Clinical Prediction Rule to Predict Asymptomatic Chlamydia and Gonorrhea Infections Among Internet-Based Testers

Ablona, Aidan MPH; Falasinnu, Titilola PhD; Irvine, Michael PhD; Estcourt, Claudia MD§; Flowers, Paul PhD; Murti, Michelle MD; Gómez-Ramírez, Oralia PhD∗,∗∗; Fairley, Christopher K. MD, PhD††; Mishra, Sharmistha MD, PhD‡‡; Burchell, Ann PhD‡‡; Grennan, Troy MD, MSc; Gilbert, Mark MD, MHSc∗,∗∗

Author Information
Sexually Transmitted Diseases 48(7):p 481-487, July 2021. | DOI: 10.1097/OLQ.0000000000001340

Testing is an essential response to the rising rates of sexually transmitted infections (STIs) globally, facilitating diagnosis, treatment, and prevention of further complications.1,2 With declining investments in publicly funded sexual health systems, there is a need to optimize the allocation of resources and spending related to STI testing.3 To reduce testing volumes in universal screening programs, strategies to restrict screening to individuals at a higher infection risk may be considered, including the use of population-based screening guidelines or selective screening methods such as clinical prediction rules (CPRs).4–6 Clinical prediction rules estimate an individual's probability of having an infection based on a predetermined set of variables.7 Moreover, in low-prevalence settings, selective screening is more cost-effective than universal screening.8

With this in mind, we previously developed a CPR to detect asymptomatic chlamydia and/or gonorrhea (CT/NG) infections among predominantly heterosexual clients (women and men) attending in-person STI clinic services, in British Columbia (BC), Canada.9 The CPR included 5 predictor variables including age, race/ethnicity, number of recent sex partners (past 6 months), previous chlamydia diagnosis (ever), and previous gonorrhea diagnosis (ever). Diagnosing asymptomatic infections in this population can prevent further complications such as pelvic inflammatory disease in women and epididymitis in men.10 Screening for CT/NG was recommended as a composite outcome, as clinically both infections are typically tested for in the same sample. Recommending screening only for clients with a minimum CPR-predicted risk score of 6 maintained a 91% sensitivity in detecting infections (168/184 infections detected) while reducing potentially unnecessary tests by 32% (3361/10,437 tests not needed).

Internet-based STI testing services have expanded in recent years.11,12 Notably, the COVID-19 pandemic has led to significant closures of publicly funded sexual health clinics, leading to the necessary expansion of Internet-based services in response to restricted access to in-person care.13,14 The application of tools like CPRs to these settings is an emerging area of interest.15,16 Existing CPRs that have been developed within in-person clinic settings may not be directly applicable to Internet-based testing services because of population differences among those accessing either setting.17,18 Thus, validation studies applying existing CPRs to Internet-based testing environments are necessary before their implementation. GetCheckedOnline, an Internet-based testing service for STIs and blood-borne infections in BC, Canada, currently uses a universal approach for CT/NG screening, wherein all individuals are routinely recommended CT/NG nucleic acid amplification test (NAAT) urine screening, regardless of reported risk.19 The primary objective of this study was to assess the validity, sensitivity, and overall performance of our existing clinic-based CPR for recommending CT/NG screening within the GetCheckedOnline testing environment.


The TRIPOD checklist for reporting the validation of a multivariable prediction model for individual prognosis or diagnosis was used to inform the description of our methods.20

Data Source

GetCheckedOnline is available in 6 cities across the province of BC, Canada, with approximately 1000 STI tests completed monthly and an overall test positivity rate of 6% (oral communication with Heather Pedersen, Online Sexual Health Services Manager, BC Centre for Disease Control, December 2019). Use of GetCheckedOnline is free of charge and confidential, with formal enrollment in a publicly funded health program not required. The model for testing is described elsewhere.19 In brief, clients must first create an account online and provide demographic information, including mandatory provision of gender and date of birth (used to derive age), with optional provision of race/ethnicity. GetCheckedOnline assesses gender using sex and gender terms interchangeably (e.g., “man/male” and “woman/female”); for clarity, “men” and “women” are used herein. To initiate a test episode, clients complete a 14-question risk assessment questionnaire. This includes mandatory questions about the gender of sex partners, the presence of STI-related symptoms, and having a sex partner(s) with a diagnosed STI, in addition to optional questions about types of sex (past 3 months), number of sex partners (past 3 months), previous chlamydia diagnosis (past year), and previous gonorrhea diagnosis (past year). Recommendations for any urine, oropharyngeal, and rectal NAAT for chlamydia and gonorrhea, as well as serology for syphilis, HIV, and hepatitis C, are tailored based on risk assessment questionnaire responses. All women and heterosexual men, categorized at a test episode level by sexual behavior, are automatically recommended chlamydia and gonorrhea urine NAAT, with additional rectal swabs recommended for clients who report being a receptive anal sex partner. GetCheckedOnline clients download their laboratory requisition forms to bring directly to participating laboratory locations for specimen collection. All STI testing is conducted centrally by the provincial Public Health Laboratory, and results are recorded in each client's electronic medical record; results are provided online (if negative) or by telephone (if positive or indeterminate). Compared with clients of in-person sexual health services, GetCheckedOnline clients are more likely to be older, identify as a sexual minority, and have had previous experience accessing STI testing services.18

CPR for CT/NG Screening

The existing clinic-based CPR used in this study predicts CT/NG infection among asymptomatic women and heterosexual men (excluding transgender individuals because of low sample sizes).9 Clinician-recorded predictor variables were extracted from electronic medical records of STI clinic attendees in Vancouver, BC, Canada, from 2000 to 2006. Multivariable logistic regression modeling was used to quantify relationships between predictors and CT/NG infection. The CPR assigns numeric scores to each test episode, with total risk scores ranging from −2 (negligible risk) to 27 (very high risk; Table 1). CT/NG screening was recommended by authors for clinic attendees with a minimum total risk score of 6, as this detected CT/NG infection with 91% sensitivity while reducing the fraction screened by 32%, close to the recommended performance benchmarks of 90% sensitivity and 40% reduction in fraction screened.21 The area under the receiver operating characteristic curve (AUC) was 0.74 (95% confidence interval [CI], 0.70–0.77).9

TABLE 1 - Clinical Prediction Rule for Estimating Risk of Chlamydia and/or Gonorrhea Infection Among STI Clinic Clients in Vancouver, British Columbia9
Variable Scoring Points
Age, y
 14–19 8
 20–24 3
 25–29 1
 30–39 −2
 ≥40 0
 White 0
 Non-White 5
No. sexual partners in previous 6 mo
 0 0
 1–2 5
 ≥3 6
Previous chlamydia diagnosis ever
 Yes 7
 No 0
Previous gonorrhea diagnosis ever
 Yes 1
 No 0

Study Population, Data Collection, and Preparation

We extracted clinical data from the GetCheckedOnline program database and laboratory data from electronic medical records between October 2015 and June 2019. Client demographics, risk assessment questionnaire responses, and testing results were linked at a test episode level using a unique identifier. We included all CT/NG test episodes (urine and/or swabs) completed within the study period among clients who were at least 14 years old. We excluded test episodes completed by nonheterosexual men or men who did not report the gender of their sexual partners, and by clients who identified as transgender. Test episodes where clients reported experiencing STI-related symptoms or contact with a partner with a diagnosed STI were excluded because testing would be clinically indicated, obviating the need to apply a CPR.

Our primary outcome of interest was CT/NG infection diagnosed within a GetCheckedOnline test episode. Risk of infection was predicted by the clinic-based CPR. Responses for the 5 CPR predictor variables, captured in client demographics and risk assessment questionnaires, were categorized to match those in the CPR. Variables were considered to have missing data if no response was provided or if clients selected “Prefer not to answer,” “Do not know,” or “Not applicable.” A complete case analysis was conducted, excluding any testing episodes with missing data for any of the 5 CPR predictor variables, with the justification that, in implementation, any testing episode with missing data would be recommended automatically for CT/NG testing within GetCheckedOnline.

Statistical Analysis

We conducted bivariate analyses to compare CT/NG infection prevalence and the distribution of CPR predictor variables between the STI clinic derivation population (data previously published) and the GetCheckedOnline validation population using χ2 tests, significant at α < 0.01. We described CT/NG infection prevalence for each predictor variable and calculated univariate odds ratios (ORs; with 95% CI) to assess the direction and magnitude of association between predictor variables and CT/NG infection. Predicted probabilities of CT/NG infection were calculated per test episode by assigning a score for each predictor variable according to the CPR and calculating a total risk score (see CPR scoring in Table 1). We assessed calibration by conducting the Hosmer-Lemeshow goodness-of-fit test using the score as its discretization, with P < 0.05 indicating poor model fit, and by describing CT/NG prevalence within total risk score categories. The model's ability to discriminate between CT/NG-positive and CT/NG-negative episodes was assessed by calculating the AUC, with 95% CI estimated using bootstrap samples on 1000 resamples of the data. Sensitivity, specificity, the fraction of clients screened, and the positive predictive value were quantified. Cutoff scores meeting the same performance benchmarks as used in the CPR derivation (≥90% sensitivity) were determined.

We conducted sensitivity analyses to explore the impact of excluding missing data from the 5 CPR predictor variables. Missing data were first described per CPR predictor variable and then assessed for associations with CT/NG infection. We then imputed missing data using Multiple Imputation with Chained Equations using the mice package in R.22 Furthermore, CPRs may be modified or updated with the addition or removal of predictor variables to improve performance in novel settings.23 We updated the CPR by removing any predictor variables having different associations with the outcome within the GetCheckedOnline and STI clinic populations, keeping all other variables constant, and assessed the updated CPR's sensitivity, fraction screened, and AUC within the complete case data set. Absolute total risk scores per testing episode would change; however, relative scores were used to calculate the AUC. Analyses were performed using R version 3.5.2 (2018-12-20).24 Ethics approval was granted by the University of British Columbia's Research Ethics Board (certificate no. H18-00437).


From October 2015 to June 2019, 14,967 CT/NG test episodes were completed using GetCheckedOnline. We excluded 4815 test episodes among behaviorally nonheterosexual men and transgender-identified individuals and further excluded 2603 test episodes where individuals reported symptoms or a partner with an STI. Of the resulting 7549 episodes among asymptomatic women and heterosexual men, 2071 episodes (27.4%) were excluded from the analysis because of missingness for any CPR predictor variable. A total of n = 5478 CT/NG testing episodes remained for our complete case analysis, representing 36.6% of all completed CT/NG testing episodes during the study period (Fig. 1).

Figure 1:
Study sample selection.

The overall prevalence of CT/NG infection was slightly higher in the GetCheckedOnline validation population (n = 5478; 2.4% prevalence) than in the STI clinic derivation population (n = 10,437; 1.8% prevalence), and this was significant (P = 0.007). The GetCheckedOnline testers included greater proportions of women, those aged 30 to 39 years, those who reported more than 3 recent sexual partners, and those without previous histories of either CT or NG diagnosis, but had a similar distribution by race/ethnicity (Table 2). CT/NG infection prevalence within levels of each CPR predictor variable also differed between populations (Table 3). The direction of association between most predictors and CT/NG infection was the same in the GetCheckedOnline and STI clinic populations; however, the magnitude of these relationships differed. Notably, in the GetCheckedOnline population, there was no association between race/ethnicity and CT/NG infection (OR, 1.14; 95% CI, 0.76–1.67).

TABLE 2 - Population Comparisons Between Sexually Transmitted Infection (STI) clinic9 and GetCheckedOnline Clients
Variable STI Clinic, 2000–2006
Derivation Population, n (%)
GetCheckedOnline, 2015–2019
Validation Population, n (%)
χ2, P Value
Chlamydia or gonorrhea case 0.007
 Yes 184 (1.8) 132 (2.4)
 No 10,253 (98.2) 5346 (97.6)
Gender* <0.001
 Women 3496 (33.5) 2655 (48.5)
 Men 6941 (66.5) 2823 (51.5)
Age, y <0.001
 14–19 257 (2.5) 168 (3.1)
 20–24 1962 (18.8) 1071 (19.5)
 25–29 2651 (25.4) 1304 (23.8)
 30–39 3181 (30.5) 1780 (32.5)
 ≥40 2386 (22.9) 1155 (21.1)
Race/ethnicity 0.013
 White 7732 (74.1) 4158 (75.9)
 Non-White 2705 (25.9) 1320 (24.1)
No. recent sexual partners <0.001
 0 644 (6.2) 222 (4.1)
 1–2 6857 (65.7) 2926 (53.4)
 ≥3 2936 (28.1) 2330 (42.5)
Previous chlamydia diagnosis <0.001
 Yes 1518 (14.5) 388 (7.1)
 No 8919 (85.5) 5090 (92.9)
Previous gonorrhea diagnosis <0.001
 Yes 619 (5.9) 38 (0.7)
 No 9819 (94.1) 5440 (99.3)
 Total 10,437 (100) 5478 (100)
Significant difference between populations is set in bold (P < 0.01).
*In derivation population, gender categorized as “female” and “male.”
STI clinic: past 6 months; GetCheckedOnline: past 3 months.
STI clinic: ever; GetCheckedOnline: past year.

TABLE 3 - Prevalence and Unadjusted Odds Ratios (ORs) of Chlamydia/Gonorrhea (CT/NG) Infection in STI clinic9 and GetCheckedOnline Populations
STI Clinic, 2000–2006, n = 10,437 GetCheckedOnline, 2015–2019, n = 5478
Variable % CT/NG Positive OR (95% CI) % CT/NG Positive OR (95% CI)
 Women 2.1 1.31 (0.97–1.77) 2.8 1.41 (1.00–2.01)
 Men 1.6 Reference 2.0 Reference
Age, y
 14–19 7.4 6.49 (3.58–11.75) 4.2 2.91 (1.11–6.86)
 20–24 2.8 2.30 (1.46–3.63) 4.7 3.28 (1.92–5.88)
 25–29 1.8 1.50 (0.94–2.38) 1.9 1.31 (0.71–2.48)
 30–39 1.1 0.88 (0.53–1.45) 1.9 1.26 (0.71–2.33)
 ≥40 1.2 Reference 1.5 Reference
 White 1.2 Reference 2.3 Reference
 Non-White 3.4 2.90 (2.16–3.89) 2.7 1.14 (0.76–1.67)
No. sexual partners in past 6 (clinic) or 3 (GetCheckedOnline) mo
 0 0.5 Reference 0.9 Reference
 1–2 1.8 3.43 (1.10–10.73) 1.7 1.91 (0.59–11.74)
 ≥3 2.0 3.92 (1.23–12.45) 3.4 3.91 (1.22–23.87)
Previous chlamydia diagnosis (clinic: ever; GetCheckedOnline: past year)
 Yes 5.1 4.40 (3.27–5.93) 5.2 2.42 (1.44–3.85)
 No 1.2 Reference 2.2 Reference
Previous gonorrhea diagnosis (clinic: ever; GetCheckedOnline: past year)
 Yes 2.1 1.21 (0.69–2.14) 5.3 2.27 (0.37–7.53)
 No 1.7 Reference 2.4 Reference
*In derivation population, gender categorized as “female” and “male.”
Bold signifies that the 95% confidence interval excludes 1.

Total CPR-predicted risk scores for the GetCheckedOnline population ranged from −2 to 25. The CPR model fit the GetCheckedOnline data well (Hosmer-Lemeshow statistic = 3.43, P > 0.90), and similar to trends within the STI clinic population, CT/NG prevalence within the GetCheckedOnline population increased with increasing total risk score categories (Supplementary Fig. 1, The CPR had a reasonable discriminative ability between those with and without CT/NG infection (AUC, 0.64; 95% CI, 0.60–0.69; Supplementary Fig. 2, Applying the minimum cutoff score of 6 (recommended in the derivation study) to the GetCheckedOnline population resulted in a sensitivity of 82% while screening 66% of the population (Table 4). However, because a sensitivity of at least 90% is recommended, the screening threshold would need to be lowered to a minimum total risk score of 4 or greater, which would yield a sensitivity of 97% and a greater proportion of clients screened (86%). The prevalence of CT/NG infection among those eligible for screening at a risk score cutoff of 4 was 2.7% compared with 0.5% among those not recommended for screening.

TABLE 4 - Sensitivity and Specificity of Cutoff Scores in STI Clinic9 and GetCheckedOnline Populations
Score Cutoff STI Clinic
Derivation Population, %
Validation Population, %
Sensitivity Specificity Screened PPV* Sensitivity Specificity Screened PPV
≥−2 100.0 0.0 100.0 1.8 100.0 0.0 100.0 2.4
≥−1 100.0 1.2 98.8 1.8 100.0 0.9 99.1 2.4
≥0 100.0 1.3 98.7 1.8 100.0 0.9 99.1 2.4
≥1 100.0 2.6 97.4 1.8 100.0 2.3 97.8 2.5
≥2 99.5 3.7 96.4 1.8 100.0 2.3 97.8 2.5
≥3 99.5 3.7 96.4 1.8 100.0 2.3 97.8 2.5
≥4 96.7 16.7 83.5 2.0 97.0 14.6 85.7 2.7
≥5 95.8 22.2 78.1 2.2 87.1 24.9 75.4 2.8
≥6 91.2 32.7 67.8 2.4 81.8 34.2 66.2 3.0
≥7 84.9 47.7 52.8 2.8 72.0 50.6 50.0 3.5
≥8 82.2 53.0 47.7 3.0 64.4 56.5 44.0 3.5
≥9 72.2 65.2 35.4 3.6 53.0 69.0 31.5 4.1
≥10 67.3 70.6 30.1 3.9 33.3 77.4 22.9 3.5
≥11 62.5 74.8 25.8 4.3 31.1 79.8 20.5 3.7
≥12 52.6 81.4 19.2 4.8 27.3 85.3 15.0 4.4
≥13 47.7 84.5 16.1 5.2 26.5 88.2 12.1 5.3
≥14 35.1 91.1 9.3 6.6 17.4 92.8 7.4 5.7
≥15 32.4 94.2 6.3 9.1 8.3 96.0 4.1 4.9
≥16 25.9 95.9 4.5 10.1 7.6 97.1 3.0 6.0
≥17 21.2 96.9 3.4 10.9 3.0 98.5 1.6 4.7
≥18 19.9 97.2 3.1 11.4 3.0 98.6 1.5 5.0
≥19 14.3 98.5 1.8 14.4 2.3 99.3 0.8 7.0
≥20 11.6 99.0 1.2 16.9 1.5 99.4 0.6 5.7
≥21 7.3 99.5 0.6 20.9 1.5 99.7 0.3 11.1
≥22 2.8 99.9 0.2 28.0 0.0 99.9 0.1 0.0
≥23 2.8 99.9 0.1 33.8 0.0 99.9 0.1 0.0
≥24 2.8 99.9 0.1 33.8 0.0 99.9 0.1 0.0
≥25 2.8 99.9 0.1 33.8 0.0 99.9 0.1 0.0
≥26 0.7 100.0 0.0 30.0 N/A N/A N/A N/A
≥27 N/A N/A N/A N/A N/A N/A N/A N/A
PPV indicates positive predictive value.

Proportions of missing data in GetCheckedOnline varied by predictor variable, with ethnicity having the highest proportion of missing data (23.0%), followed by number of sexual partners in the past 3 months (3.9%), past-year chlamydia diagnosis (3.5%), and past-year gonorrhea diagnosis (3.5%). Prevalence of CT/NG infection among observations with missing data for any predictor variable was 2.2%. Missing data for any predictor variable were not associated with positivity (OR, 0.90; 95% CI, 0.63–1.25). In applying the CPR to a multiply imputed data set (n = 7549), the CPR had a similar ability to discriminate between CT/NG-positive and CT/NG-negative GetCheckedOnline testers as in the complete case analysis (AUC, 0.65; 95% CI, 0.60–0.69). At a cutoff risk score of 4, 93% of infections would be detected by screening only 81% of all clients (Supplementary Table 1,

Given that there was no association between race/ethnicity and CT/NG infection in the GetCheckedOnline population (Table 3), an updated CPR excluding race/ethnicity as a predictor variable was also applied to the complete case data set. Compared with the performance of the original CPR, this modified CPR had a similar overall discrimination (AUC, 0.66; 95% CI, 0.62–0.71) and a similar sensitivity (94%) and fraction screened (83%) at a minimum cutoff score of 4 (Supplementary Table 1,


The application of an existing CPR to detect asymptomatic CT/NG infection is valid within an Internet-based STI testing environment. The distribution of predictor variables, overall CT/NG infection prevalence, and unadjusted associations between predictor variables and CT/NG infection differed between the GetCheckedOnline and STI clinic populations. The CPR had a lower discriminative ability when applied to the GetCheckedOnline validation population (AUC, 0.64) compared with the STI clinic derivation population (AUC, 0.74). To maintain a minimum sensitivity of ≥90%, testing would be reduced by 14%. Sensitivity analyses demonstrated little difference in the CPR's performance by proportions of missingness in data collected online.

Population differences in in-person, clinic-based, and Internet-based testers are important considerations comparing this CPR's performance across settings and time. In considering STI services specifically, population dynamics (e.g., shifts in incidence within specific sexual networks or population subgroups) may influence risk over time beyond individual-level factors used in a prediction tool created at one time point. Temporal differences in transmission and prevalence between the period of CPR derivation (2000–2006) and validation (2015–2019) may explain performance differences; similar to our findings, a lower discrimination (AUC, 0.64) was observed when the CPR was validated in a clinic-based population at a later period.9 Also similar to our findings, external validation of this CPR to STI clinic settings beyond Vancouver (the site of the CPR's derivation) demonstrated lower discriminative power (AUC, 0.69) within a higher CT/NG prevalence (5.3%) sample.25 Regional differences in underlying CT/NG prevalence, as can be observed within GetCheckedOnline's provincial scope, may have impacted the CPR's performance.26 Although we observed differences in the distribution of gender between the STI clinic and GetCheckedOnline populations, discrimination on GetCheckedOnline was similar between women (AUC, 0.63; 95% CI, 0.54–0.69) and men (AUC, 0.64; 95% CI, 0.57–0.71; data not shown). The lower AUC may be explained by the nonassociation between non-White race/ethnicity and CT/NG infection, and CPR performance was similar when race/ethnicity was removed as a predictor variable. Given that GetCheckedOnline addresses physical, psychosocial, and sociocultural barriers in accessing in-person STI testing services,18,27 our findings support the notion that the population of Internet-based testers may be distinct from those testing in-person.

To our knowledge, this is the first study validating the use of a clinic-based tool online, with several important strengths and limitations. Our validation study uses a large sample size of Internet-based testers to identify population differences compared with clinic-based testers. Similar methods could be used to develop or validate existing tools specific to other priority populations, such as men who have sex with men, and for other STIs, such as syphilis. One limitation of our analyses may be outcome ascertainment bias and underestimating undiagnosed infections, particularly among women, as GetCheckedOnline did not offer oropharyngeal CT/NG swab testing for women during the study period. The complete case analysis may have introduced selection bias into our analytic sample, particularly when data are not missing completely at random.28 However, predictor missingness being unassociated with CT/NG positivity and multiple imputation analyses demonstrated that missing responses may have been missing at random. Requiring mandatory responses to avoid missingness in GetCheckedOnline data may be unnecessary and may infringe upon service users' personal autonomy and right to skip or refuse answering questions. That being said, misclassification of sexual orientation by behavior was possible if men did not report the gender of their sexual partners, in which case they were excluded from this analysis and the number of behaviorally heterosexual men using GetCheckedOnline may be underestimated. We also recognize the limitation of using a combined CT/NG outcome, and this may limit generalizability; in future research, we will be assessing the utility of separate CPRs for these infections.

Clinical prediction rules can be used as tools in Internet-based STI testing platforms for streamlining resource allocation through restricting testing to those at highest risk, reducing both client- and systems-level testing-related burdens. These tools could also be used as self-assessment educational tools to raise awareness of STI infection risk, or to direct clients to different types of STI supports, such as broader psychosocial and preventative interventions, based on predicted infection risk. Deriving a novel CPR using data from clients of Internet-based testing services may address performance differences and improve yields in reducing testing volumes, as variables not included in this CPR (e.g., condomless sex) may be relevant in predicting infections online.15 However, missing data characteristic of online environments may limit the yield of applying CPRs because of the need for complete data at the time tests are recommended.

Further research is needed before determining whether or how to fully integrate CPRs into Internet-based testing services. For example, evaluating the clinical impact (such as earlier diagnosis) and cost-effectiveness of using this CPR in an online setting, compared with not applying the CPR, would help inform the use of these tools.23 From the service user perspective, it will be important to study the acceptability of using CPRs to guide STI test recommendations, particularly for clients of Internet-based testing services, who may or may not be recommended testing that matches their own estimate of sexual risk or situation. This is especially pertinent given the absence of a direct clinical consultation in Internet-based testing and could lead to some clients answering assessment questions in ways that allow them to access tests they would otherwise be denied. Finally, we recognize the need for further research among individuals holding socially marginalized identities who may be impacted by these tools, including co-developing effective and appropriate messaging as to why testing may or may not be recommended. Before implementation, meaningful consideration is also necessary regarding racialized and transgender communities who are or are not represented in these data and broader health services.29,30

As Internet-based STI testing services become more integrated in publicly funded healthcare systems, it is critical to consider the translation of existing selective screening tools to these contexts. Our findings provide strong evidence of the validity of using a clinic-based CPR in optimizing Internet-based testing services such as GetCheckedOnline. However, further work is needed both in considering the practicalities of implementing tools like CPRs and in mitigating unintended consequences of their use within these services.


1. Rowley J, Vander Hoorn S, Korenromp E, et al. Chlamydia, gonorrhoea, trichomoniasis and syphilis: Global prevalence and incidence estimates, 2016. Bull World Health Organ 2019; 97:548–562p.
2. Rietmeijer CA. Improving care for sexually transmitted infections. J Int AIDS Soc 2019; 22(Suppl 6):e25349.
3. Leichliter JS, Heyer K, Peterman TA, et al. US public sexually transmitted disease clinical services in an era of declining public health funding: 2013–14. Sex Transm Dis 2017; 44:505–509.
4. Falasinnu T, Gustafson P, Hottes TS, et al. A critical appraisal of risk models for predicting sexually transmitted infections. Sex Transm Dis 2014; 41:321–330.
5. Edelman NL, Cassell JA, Mercer CH, et al. Deriving a clinical prediction rule to target sexual healthcare to women attending British General Practices. Prev Med 2018; 112:185–192.
6. van Klaveren D, Götz HM, Op de Coul EL, et al. Prediction of Chlamydia trachomatis infection to facilitate selective screening on population and individual level: A cross-sectional study of a population-based screening programme. Sex Transm Infect 2016; 92:433–440.
7. Adams ST, Leveson SH. Clinical prediction rules. BMJ 2012; 344:d8312.
8. Marrazzo JM, Celum CL, Hillis SD, et al. Performance and cost-effectiveness of selective screening criteria for Chlamydia trachomatis infection in women. Implications for a national chlamydia control strategy. Sex Transm Dis 1997; 24:131–141.
9. Falasinnu T, Gilbert M, Gustafson P, et al. Deriving and validating a risk estimation tool for screening asymptomatic chlamydia and gonorrhea. Sex Transm Dis 2014; 41:706–712.
10. Kalwij S, Macintosh M, Baraitser P. Screening and treatment of Chlamydia trachomatis infections. BMJ 2010; 340:c1915.
11. Estcourt CS, Gibbs J, Sutcliffe LJ, et al. The eSexual Health Clinic system for management, prevention, and control of sexually transmitted infections: Exploratory studies in people testing for Chlamydia trachomatis. Lancet Public Health 2017; 2:e182–e190.
12. Söderqvist J, Gullsby K, Stark L, et al. Internet-based self-sampling for Chlamydia trachomatis testing: A national evaluation in Sweden. Sex Transm Infect 2020; 96:160–165.
13. Nagendra G, Carnevale C, Neu N, et al. The potential impact and availability of sexual health services during the COVID-19 pandemic. Sex Transm Dis 2020; 47:434–436.
14. Joint submission by the British Association for Sexual Health and HIV (BASHH) and the British HIV Association (BHIVA). Health and Social Care Inquiry on delivering core NHS and care services during the pandemic and beyond. 2020; Available at: Accessed July 10, 2020.
15. Patel AV, Gaydos CA, Jett-Goheen M, et al. Assessing association between IWantTheKit risk quiz tool and sexually transmitted infection positivity in male users for sexually transmitted infection screening. Int J STD AIDS 2018; 29:122–127.
16. Allan-Blitz LT, Konda KA, Vargas SK, et al. The development of an online risk calculator for the prediction of future syphilis among a high-risk cohort of men who have sex with men and transgender women in Lima, Peru. Sex Health 2018; 15:261–268.
17. Gilbert M, Salway T, Haag D, et al. A cohort study comparing rate of repeat testing for sexually transmitted and blood-borne infections between clients of an Internet-based testing programme and of sexually transmitted infection clinics in Vancouver, Canada. Sex Transm Infect 2019; 95:540–546.
18. Gilbert M, Thomson K, Salway T, et al. Differences in experiences of barriers to STI testing between clients of the Internet-based diagnostic testing service and an STI clinic in Vancouver, Canada. Sex Transm Infect 2019; 95:151–156.
19. Gilbert M, Haag D, Hottes TS, et al. Get checked... where? The development of a comprehensive, integrated Internet-based testing program for sexually transmitted and blood-borne infections in British Columbia, Canada. JMIR Res Protoc 2016; 5:e186.
20. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMJ 2015; 350:g7594.
21. La Montagne DS, Patrick LE, Fine DN, et al., Region X Infertility Prevention Project. Re-evaluating selective screening criteria for chlamydial infection among women in the U S Pacific Northwest. Sex Transm Dis 2004; 31:283–289.
22. van Buuren S, Groothuis-Oudshoorn K. Mice: Multivariate imputation by chained equations in R. J Stat Softw 2011; 45:67.
23. Moons KG, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012; 98:691–698.
24. R: A Language and Environment for Statistical Computing [computer program]. Vienna, Austria: R Foundation for Statistical Computing, 2018.
25. Falasinnu T, Gilbert M, Gustafson P, et al. A validation study of a clinical prediction rule for screening asymptomatic chlamydia and gonorrhoea infections among heterosexuals in British Columbia. Sex Transm Infect 2016; 92:12–18.
26. Götz HM, van Oeffelen LA, Hoebe CJPA, et al. Regional differences in chlamydia and gonorrhoeae positivity rate among heterosexual STI clinic visitors in the Netherlands: Contribution of client and regional characteristics as assessed by cross-sectional surveillance data. BMJ Open 2019; 9:e022793.
27. Knight RE, Chabot C, Carson A, et al. Qualitative analysis of the experiences of gay, bisexual and other men who have sex with men who use A comprehensive Internet-based diagnostic service for HIV and other STIs. Sex Transm Infect 2019; 95:145–150.
28. Perkins NJ, Cole SR, Harel O, et al. Principled approaches to missing data in epidemiologic studies. Am J Epidemiol 2018; 187:568–575.
29. Bauer GR, Hammond R, Travers R, et al. “I don't think this is theoretical; this is our lives”: How erasure impacts health care for transgender people. J Assoc Nurses AIDS Care 2009; 20:348–361.
30. Obermeyer Z, Powers B, Vogeli C, et al. Dissecting racial bias in an algorithm used to manage the health of populations. Science 2019; 366:447–453.

Supplemental Digital Content

Copyright © 2020 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the American Sexually Transmitted Diseases Association.