Sexually transmitted disease clinics are a key component of the national infrastructure for the control of bacterial sexually transmitted diseases (STDs) and human immunodeficiency virus (HIV).1 Due to dwindling resources2 and the inability of some STD clinics to meet the current demand for services, many clinics have incorporated “express visits,” testing-only visits without a clinician evaluation, to streamline clinic flow and provide rapid service for patients seeking STD screening. There is increasing evidence that express care can reduce utilization of services that provide minimal personal or public health benefit, such as physical examinations for asymptomatic patients. This can reduce long-term costs,3,4 but the impact on disease detection and quality of care remains uncertain.
Although the express care approach has many potential advantages, it may miss infections of personal or public health importance that could be detected through clinician evaluation in a standard visit. Erroneously triaging patients with disease to express visits would delay diagnosis and treatment, which could lead to health consequences and ongoing transmission. The converse, triaging patients without disease to standard visits, may unnecessarily use limited clinic resources.5 Thus, evaluating the ability of STD clinic express care models to appropriately identify clinically relevant cases is crucial for determining which models maximize population-level impact on disease control, quality of care, and clinic efficiency.6
The Public Health—Seattle and King County (PHSKC) STD Clinic in Seattle, Wash, uses a comprehensive computer-assisted self-interview (CASI) to collect patient history and an automated triage system based on the CASI to identify patients for express care. This STD clinic implemented systematic assessments for triage to express care in 20107; however, due to the staffing model in place, almost all patients continued to receive a clinician evaluation irrespective of triage status. Therefore, the PHSKC STD Clinic database contains both CASI responses and outcomes of a clinician evaluation for each patient, providing an opportunity to evaluate the potential impact of express care on disease detection over a 5-year period. The objective of this study was to evaluate the test characteristics of the PHSKC STD Clinic triage algorithm compared to a disease-focused outcome, to characterize patients who were inappropriately triaged to express care based on that outcome, and to identify the most common reasons patients who did not meet the disease-focused outcome were triaged to standard visits. The disease-focused outcome that we examined was designed to prioritize the same-day detection and treatment of health conditions of personal or public health importance (ie, symptomatic or transmissible and associated with significant morbidity).
MATERIALS AND METHODS
Study Design and Sample
This cross-sectional study included all patients at the PHSKC STD Clinic who completed a CASI at a new problem visit between October 1, 2010 and June 30, 2015.
We analyzed electronic medical record data and CASI responses. Patients who present for a new problem visit complete the CASI in a private cubicle in the clinic waiting room after check-in at the front desk. Once a patient submits their CASI, an automated computer algorithm determines the patient’s triage status. A printed report that includes the triage status and a summary of the CASI responses is provided to the clinician who sees the patient. Two versions of the CASI were used during the analysis period: CASI 1 from October 2010 to January 2012 and CASI 2 from May 2012 to June 2015. The CASIs requested almost identical information. Of note, CASI 1 did not and CASI 2 did include an option for women to request emergency contraception. For this analysis, we assumed that women who completed CASI 1 were not seeking emergency contraception. There were no other important differences between the CASIs for this analysis.
Some patients (36%) did not complete the CASI and were excluded from this analysis. Patients did not complete the CASI if it was down for repair or revisions; due to English language, reading, or computer proficiency; or in some instances when clinicians saw patients without the CASI to facilitate clinic flow. Patients who did and did not complete the CASI were similar with respect to age group, gender, and race; however, Hispanic patients were less likely to complete the CASI.
Measures and Definitions
Using CASI responses, we reconstructed a triage status according to the PHSKC STD Clinic triage algorithm as of October 1, 2015, to evaluate the current algorithm for all visits during the analysis period. The criteria for standard visits (ie, exclusion from express care), listed in Table 1, include both “disease-focused” and “health service-focused” priorities. Of note, patients who identified as transgender or reported transgender sex partners were triaged to standard visits. This was because, at the time the CASI was first implemented, the perspective of clinic leadership was that the patient history in these situations could be better collected in person.
A triage status was assigned to each patient at the time of the visit, according to the automated algorithm based on CASI responses. For most of the analysis period, the clinic did not have support staff available to see patients triaged to express care, and clinicians evaluated all patients, regardless of triage status. From the end of 2012 through mid-2013, a medical assistant (MA) conducted some express visits; however, the MA primarily assisted with before care and aftercare, with the patient still receiving a clinician evaluation. Although clinicians were encouraged to expedite visits for patients triaged to express care by the automated system, many clinicians voiced reluctance to do so, and the average clinician visit time did not change after implementation of the triage algorithm.7 Similarly, the clinicians frequently collected screening swabs rather than offer the option for self-obtained swabs.8 Therefore, based on our experience in the clinic and supporting data, we assumed for this analysis that all patients were evaluated by clinicians, irrespective of triage status, and used this clinician assessment to construct the study outcome, “need for a standard visit.” Importantly, the screening tests performed in express care are the same as those performed in standard visits.
The need for a standard visit was based on electronic medical record review. We coded each visit to indicate whether the patient needed a standard visit or could have received an express visit, based on disease-focused criteria intended to identify patients with an infection of public health importance that could be diagnosed and treated at the same visit. We classified patients as needing a standard visit if they reported “key symptoms” (genital symptoms, abdominal pain (for women), or body rash), received empiric treatment, or were diagnosed with and treated for a key infection or syndrome at the same visit. Such infections were (1) possible to diagnose with clinical examination and point-of-care testing alone and (2) of public health importance due to the potential for ongoing transmission or health consequences of delayed treatment. These included primary or secondary syphilis; urethral, cervical, or rectal Neisseria gonorrhoeae detected on Gram stain; nongonococcal urethritis; epididymitis; proctitis; mucopurulent cervicitis; pelvic inflammatory disease; bacterial vaginosis; vaginal candidiasis; trichomoniasis; urinary tract infection; genital ulcer of unknown etiology; skin and soft tissue infection (ie, cellulitis); and herpes simplex virus. Patients who did not meet any of these conditions were classified as eligible to receive express care. Of note, HIV diagnosed by point-of-care test was not a criterion for a standard visit because a patient with a positive HIV test during an express visit can be routed to see a clinician at that time.
For the purposes of this analysis, we classified patients who were mistriaged by the algorithm into two categories. We defined patients who needed a standard visit according to the disease-focused outcome but were triaged to express care by the algorithm as “underserved” and patients who were eligible for express care but triaged to standard visits by the algorithm as “overserved.” This terminology reflects the criteria we used for the outcome of this analysis, which is disease-focused, rather than the criteria in use at the PHSKC STD Clinic, which include both disease-focused and health service-focused criteria. Furthermore, the terms are oversimplified for clarity. Because our analysis evaluates a counterfactual scenario, the “underserved” patients were not underserved in actuality.
We estimated the sensitivity of the algorithm for identifying patients needing standard visits and the percent of patients who did not need standard visits that was triaged to express visits, stratified by gender. We also estimated the area under the receiver operating curve (AUC) of the algorithm to appropriately triage patients based on our disease-focused outcome. For underserved patients, we examined the clinical outcomes that indicated need for a standard visit. We used χ2 tests to compare characteristics of underserved patients to all other patients. For overserved patients, we examined which CASI triage criteria led them to be triaged to standard visits. Patient-reported reasons for seeking care, reasons underserved, and reasons overserved are presented in mutually exclusive and hierarchical categories based on our STD clinic disease control and patient care priorities. We created mutually exclusive and hierarchical categories to facilitate interpretation of the results; although, in practice, more than one category could pertain to a patient.
This project was approved by the University of Washington Human Subjects Division. Analyses were conducted using Stata 13 (StataCorp, College Station, Texas).
Between October 1, 2010, and June 30, 2015, 18,660 patients completed a CASI at 32,113 new problem visits, including 24,474 (76%) visits among men and 7639 (24%) visits among women. Characteristics of the study sample are presented in Table 2. Male patients were older than female patients and more likely to be white and Hispanic. For just over half of visits among men (55%), the patient reported sex with men in the past year (men who have sex with men [MSM]). At 1342 (4%) visits, the patient reported having a positive STD test result from another provider for which they were seeking treatment. Of visits for patients not reporting a positive STD result, 17,349 (54%) were among patients with key symptoms. Of the remaining visits, 3104 (10%) were among patients who reported contact to STD. Of 19,059 visits where the patient reported key symptoms to either the CASI or the clinician, these symptoms were reported to the CASI at 18,148 visits (95%). Of the 18,148 patients who reported key symptoms to the CASI, 2320 (13%) had “no symptoms” specified by the clinician.
Patients were triaged to express care at 7366 (23%) visits. Patients at 9857 (31%) visits did not need a standard visit based on our study outcome. The automated algorithm appropriately triaged 21,337 (87%) men and 6259 (82%) women (Table 3). The algorithm had 95% and 98% sensitivity for identifying men and women, respectively, needing standard visits. The algorithm triaged 72% of men and 33% of women who did not need a standard visit to express visits. The algorithm had an AUC of 0.83 (95% confidence interval, 0.83–0.84) for men and AUC of 0.65 (95% confidence interval, 0.64–0.67) for women. The automated algorithm led 893 (4%) men and 120 (2%) women to be underserved and 244 (9%) men and 1260 (16%) women to be overserved.
The reasons patients were underserved are presented in Table 4. Underserved men most frequently needed standard visits due to patient report of key symptoms to the clinician but not the CASI (52%). The symptoms these men most frequently reported to the clinician but not the CASI were genital lesions (50%), “other” symptoms (23%), and nongenital rash (9%). Receipt of empiric treatment for contact to STD (18%) or otherwise (24%) when the patient did not disclose key symptoms or contact to STD to the CASI also accounted for a large percentage of underserved men. Underserved women most frequently needed standard visits due to diagnosis with a key infection or syndrome without reporting symptoms to the CASI or clinician (46%). Asymptomatic bacterial vaginosis was the most frequent diagnosis among these underserved women (n = 40, 73%), potentially identified during research study visits. Report of key symptoms to the clinician but not the CASI also accounted for a large percentage of underserved women (32%). The symptoms these women most frequently reported to the clinician but not the CASI were “other” symptoms (29%), vaginal discharge (24%), and vaginal odor (18%). Hispanic patients (P = 0.004), men (both MSM and men who have sex with women only; P < 0.001 v. women), and patients who completed CASI 1 (P < 0.001) were significantly more likely to be underserved (Table 5).
The reasons patients were overserved are presented in Table 6. The greatest proportion of overserved men (44%) was triaged to standard visits because they reported contact to STD to the CASI but did not subsequently receive treatment during their visit. Reporting a sore throat to the CASI, which was not considered a key symptom for this analysis due to lack of specificity in practice, also led a number of these men to be overserved (18%). None of these men were diagnosed with an infection/syndrome or received empiric treatment. The greatest proportion of overserved women (51%) was triaged to standard visits because they reported not having had a Pap test in the past year to the CASI, followed by reporting that they wanted contraception to the CASI (15%) and reporting contact to STD to the CASI but not subsequently receiving treatment during their visit (15%). Of the overserved men and women who reported contact to STD but did not subsequently receive treatment, 40% reported contact to human papillomavirus (HPV)/warts and 6% were not sure which STD.
Using a CASI administered to most patients seen for new problem visits in our STD clinic, we found that 23% of patients were triaged to express care and that our CASI-based algorithm performed well for identifying patients who need standard clinician visits. It had high sensitivity for men (95%) and women (98%), although many patients who did not need a standard visit based on our disease-focused outcome were triaged to standard visits, particularly women (67%). Thus, almost all patients who needed a standard visit for disease-focused reasons were triaged to standard visits; however, many patients who were eligible for express care were also triaged to standard visits, often for health service-focused reasons. These findings highlight the safety and potential utility of automated triage as a means to improve clinic efficiency and potentially quality of care.
Our findings complement prior evaluations of STD clinic express care models that have primarily focused on the impact on clinic efficiency3,4,9,10 or have compared STD diagnosis rates between express and standard visits3,4,10–12 or before and after introducing express visits.4,13 In other settings, the percentage of patients receiving express visits generally varied from 20%3 to 36%,11 in-line with the 23% triaged to express care in our clinic. Prior studies have consistently found that Chlamydia trachomatis, Neisseria gonorrhoeae, syphilis, and HIV test-positivity is lower among patients who receive express visits compared to standard visits,3,4,10–12 suggesting that triage criteria can effectively identify lower risk patients. Our study builds on these findings by evaluating an automated algorithm and demonstrating that only 5% of men and 2% of women who needed a clinician evaluation for disease-focused reasons were triaged to express visits. This provides evidence that diagnosis and treatment of disease is delayed for very few patients in an express care model. This finding is consistent with a prior study of diagnoses made among low-risk, asymptomatic heterosexual patients who completed a CASI, which found that just over 2% received a diagnosis that would not have been made in an express visit.14 Some prior studies suggest that a substantial number of asymptomatic urethritis cases may be missed with express care, if physical examinations and Gram stains are routine for asymptomatic men,10,15 which are not in our clinic nor is this practice recommended by treatment guidelines.
Despite the very high sensitivity of the algorithm, we found that it disproportionately underserved Hispanic patients. Although the absolute difference in the proportion of underserved and not-underserved patients reporting Hispanic ethnicity was small, this could be a signal of a language barrier for some Hispanic patients when completing the CASI. Overall, the most common reason that the algorithm led to underserving patients was that some patients (5%) did not report having symptoms to the CASI but subsequently reported symptoms to the clinician, which limits the ability of any automated triage algorithm to identify health conditions of personal or public health importance. Although the reasons for misreporting symptoms are unknown, they could include discomfort with disclosing symptoms to the CASI or a shortcoming of the CASI questionnaire. Alternatively, this may reflect a priming effect of the CASI that increases the likelihood of reporting symptoms to the clinician. Other studies in STD clinics have similarly found that clinicians elicit a greater report of STD symptoms compared with a CASI.16,17 However, multiple studies in STD clinics have also identified increased reporting of sensitive risk behaviors to a CASI compared with clinician interview.16–20 Thus, the benefits of more accurately obtaining sensitive risk information must be considered when weighing the potential for misreported symptoms with a CASI. To yield the benefits of the CASI for collecting sensitive information while minimizing misreport of STD symptoms, CASI should be combined with a brief in-person symptom screening for patients who do not disclose symptoms to the CASI. This could be a simple question, such as “Are you having any symptoms that you would like to discuss with the clinician today?” Alternatively, it may be that some patients did not fully understand questions related to symptoms and that modifications to our CASI could improve its performance. For example, the largest number of patients underreported genital lesions to the CASI. Perhaps rephrasing the question, currently “Are you having any of these symptoms today (check all that apply): […] Sores, bumps or rashes on the genitals”, could elicit a greater report of this symptom, particularly from men. Cognitive interviews with patients could help us better understand this issue.
Our finding that many patients (44%) who did not need standard visits for disease-focused reasons were triaged to standard visits due to health service-focused criteria raises the important question of which health services to provide routinely in STD clinics. The answer is not straightforward or uniform across sites. The decision about whether to include health service indications in triage considerations depends on local priorities and resources. In our clinic, over half of overserved women were triaged to standard visits because they were aged 21 years or older and indicated that they had not had a Pap test in the past year. However, of these women, only a small minority (12%) actually received a Pap test at that visit. This likely reflects the complexity of Pap guidelines, which do not depend solely on time since the last Pap test, but also on factors difficult to capture in a CASI, such as history of dysplasia, degree of dysplasia and subsequent evaluation, and whether HPV cotesting was performed. Although there is a compelling rationale for providing Pap tests in STD clinics, STD clinics could also reasonably decide that Pap testing is not a high priority relative to other HIV/STD public health priorities in the context of limited resources. In some locations, preventive services, such as Pap tests, are provided at no cost in the primary care setting. However, in areas where some women are uninsured, women who seek care in STD clinics may not have access to primary care, so STD clinic visits may provide a critical opportunity for preventive sexual health services. Moreover, as we look to the future, there are other key health services that must be considered. For example, MSM who meet the criteria for antiretroviral pre-exposure prophylaxis would ideally be triaged to standard visits to discuss pre-exposure prophylaxis with a clinician. Another reason that some patients were triaged to standard visits but did not require them was report of contact to STD to the CASI among patients did not subsequently receive treatment during their visit. The reasons for not providing treatment are not evaluable in our data, but from our experience in the clinic primarily include contact to an STD for which presumptive therapy is not indicated (eg, herpes simplex virus or HPV), report of a contact for which the patient had already received treatment, or remote (>1 year) contact. This could also represent inappropriate care.
Our analysis was subject to important limitations. First, there was likely some misclassification of our triage and outcome groups. Some women had missing information on a triage criterion (requesting emergency contraception), some individuals may have had incomplete outcome information if they truly received an express visit, and some patients who we considered overserved may have been appropriately triaged but received inappropriate care. Second, the representativeness of our study sample is unknown. Our study was limited to visits with a CASI, and we used data from a single STD clinic that serves predominantly MSM in the Pacific Northwest. These results should be explored in other populations. Finally, our disease-focused outcome did not consider patient preferences or other factors that are part of quality STD services. However, a key strength of our study is that we used real-world, programmatic data to evaluate the performance of an express care triage model.
In conclusion, we have shown that an express care triage algorithm can achieve very high sensitivity for screening purposes in an STD clinic. Moreover, we have identified primary reasons patients were underserved and overserved by the algorithm, which can guide efforts to improve disease control, patient care, and clinic efficiency. Our clinic has not yet implemented express care as it was initially envisioned when we created the CASI-based triage algorithm. To date, the primary barrier to this has been the lack of a staffing model to facilitate express care. During the time of this analysis, mid-level medical providers completed all tasks in the clinic from the time that the patient was called from the waiting room to administration of treatment. We are currently undergoing a staff reorganization that will add MAs to the staff who can conduct express care, thereby allowing the clinicians to see symptomatic and otherwise more complex patients. In addition to the current study, we are working to define the ideal triage algorithm, which optimizes disease detection and clinic efficiency.
1. Golden MR, Kerndt PR. Improving clinical operations: Can we and should we save our STD clinics? Sex Transm Dis 2010; 37:264–265.
2. Wong W. Fact Sheet: STD program capacity and preparedness in the United States: Results of a national survey. Washington, DC: National Coalition of STD Directors, 2009.
3. Rukh S, Khurana R, Mickey T, et al. Chlamydia and gonorrhea diagnosis, treatment, personnel cost savings, and service delivery improvements after the implementation of express sexually transmitted disease testing in Maricopa County, Arizona. Sex Transm Dis 2014; 41:74–78.
4. Gratrix J, Bergman J, Brandley J, et al. Impact of introducing triage criteria for express testing at a Canadian sexually transmitted infection clinic. Sex Transm Dis 2015; 42:660–663.
5. Rietmeijer CA. The pros and cons of the express visit option. Sex Transm Dis 2013; 40:62–63.
6. Dombrowski JC, Golden MR. Modernizing operations to improve efficiency and refine the role and mission of sexually transmitted infection clinics. Sex Transm Dis 2013; 40:81–84.
7. Dombrowski JC, Kerani RP, Golden MR. STD Clinic Triage Based on Computer-Assisted Self Interview: The King County Experience. Quebec: Presented at: 19th meeting of the International Society for Sexually Transmitted Diseases Research [O5-S1.06], 2011.
8. Barbee LA, Dombrowski JC, Kerani R, et al. Effect of nucleic acid amplification testing on detection of extragenital gonorrhea and chlamydial infections in men who have sex with men sexually transmitted disease clinic patients. Sex Transm Dis 2014; 41:168–172.
9. Knight VC, McNulty A. Triage in a public outpatient sexual health clinic. Sex Health 2006; 3:87–90.
10. Shamos SJ, Mettenbrink CJ, Subiadur JA, et al. Evaluation of a testing-only “Express” visit option to enhance efficiency in a busy STI clinic. Sex Transm Dis 2008; 35:336–340.
11. Heijman TL, Van der Bij AK, De Vries HJ, et al. Effectiveness of a risk-based visitor-prioritizing system at a sexually transmitted infection outpatient clinic. Sex Transm Dis 2007; 34:508–512.
12. Wong W, Broad J, Rutledge T, et al. Impact of a Pilot Fast-Track Screening Program for a High-Volume STD Clinic—Chicago, Illinois. Atlanta: Presented at: 2006 National STD Prevention Conference, 2006.
13. Borrelli JM, Paneth-Pollak R, Wright S, et al. The Impact of Introducing “Express Visits” for Asymptomatic Persons Seeking STD Services in a Busy Urban STD Clinic System, New York City, 2005–2006. Atlanta: Presented at: 2008 National STD Prevention Conference [C2d], 2008.
14. Yeung A, Bush M, Cummings R, et al. Use of computerized medical records to determine the feasibility of testing for chlamydia without patients seeing a practitioner. Int J STD AIDS 2010; 21:755–757.
15. Xu F, Stoner BP, Taylor SN, et al. “Testing-only” visits: An assessment of missed diagnoses in clients attending sexually transmitted disease clinics. Sex Transm Dis 2013; 40:64–69.
16. Kurth AE, Martin DP, Golden MR, et al. A comparison between audio computer-assisted self-interviews and clinician interviews for obtaining the sexual history. Sex Transm Dis 2004; 31:719–726.
17. Rogers SM, Willis G, Al-Tayyib A, et al. Audio computer assisted interviewing to measure HIV risk behaviours in a clinic population. Sex Transm Infect 2005; 81:501–507.
18. Ghanem KG, Hutton HE, Zenilman JM, et al. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect 2005; 81:421–425.
19. Richens J, Copas A, Sadiq ST, et al. A randomised controlled trial of computer-assisted interviewing in sexual health clinics. Sex Transm Infect 2010; 86:310–314.
© Copyright 2017 American Sexually Transmitted Diseases Association
20. Fairley CK, Sze JK, Vodstrcil LA, et al. Computer-assisted self-interviewing in sexual health clinics. Sex Transm Dis 2010; 37:665–668.