Share this article on:

Evaluating a Web-Based Test Results System at an Urban STI Clinic

Ling, Sarah B. MPH*†; Richardson, Douglas B. MAS*; Mettenbrink, Christie J. MSPH, MT(ASCP)*; Westergaard, Benton C. BA*; Sapp-Jones, Terri D.*; Crane, Lori A. PHD, MPH†; Nyquist, Ann-Christine MD, MSPH†‡; McFarlane, Mary PHD§; Kachur, Rachel MPH§; Rietmeijer, Cornelis A. MD, PHD, MSPH*†

Sexually Transmitted Diseases: April 2010 - Volume 37 - Issue 4 - pp 259-263
doi: 10.1097/OLQ.0b013e3181d3d037
Original Study

Background: Notifying patients of gonorrhea and chlamydia test results using online services may improve clinic efficiency and increase receipt of test results. This study evaluated the implementation of an online results system in an urban sexually transmitted infections clinic.

Methods: Using the clinic's electronic medical records system to assess if and how gonorrhea and chlamydia test results were obtained, 3 time periods were examined between December 2007 and April 2009: Period 1, six months before initiation of the online results system; Period 2, six months when patients could opt in for online results by creating their own access codes; and Period 3, four months when access codes were assigned. In addition, a survey was conducted to assess reasons for accepting or declining the online results system.

Results: A total of 9056 new patient visits were evaluated. During Periods 1, 2, and 3, respectively 67%, 67%, and 70% patients received results either online or by telephone (NS). The proportion of patients calling the clinic for results decreased from 67% in Period 1, to 51% in Period 2, and 36% in Period 3 (P < 0.0001). Survey results indicated that patients accepted online results primarily because of the ability to check results anytime of day. Reasons for not accepting results online included lack of Internet access or a preference to receive results via the telephone.

Conclusions: The online results system decreased the number of phone calls to the clinic pertaining to STI test results, but had no effect on the overall proportion of patients receiving results.

Evaluation of a Web-based test results system at an urban sexually transmitted infection clinic demonstrated that offering online results decreased the proportion of patients calling the clinic for chlamydia and/or gonorrhea results but neither increased nor decreased the proportion of patients actually receiving results.

From the *Denver Public Health Department, Denver, CO; †Department of Community and Behavioral Health, Colorado School of Public Health, University of Colorado Denver, Aurora, CO; ‡Section of Epidemiology, The Children's Hospital, Aurora, CO; and §Division of STD Prevention, Centers for Disease Control and Prevention, Atlanta, GA

This publication was made possible through the Centers for Disease Control and Prevention (CDC) and the Association for Prevention Teaching and Research (APTR) Cooperative Agreement No. 5U50CD300860–21, Project TS-1400.

The findings and conclusions in this manuscript are those of the author(s) and do not necessarily represent the views of the Centers for Disease Control and Prevention or the Association for Prevention Teaching and Research.

Correspondence: Cornelis A. Rietmeijer, MD, PhD, MSPH, Denver Public Health Department, Pavilion H, Unit 03, Public Health, 605 Bannock St, Denver, CO 80204. E-mail:

Received for publication July 15, 2009, and accepted September 27, 2009.

Prompt treatment of sexually transmitted infections (STIs) is important in improving disease prognosis and prevention of ongoing transmission.1 The Internet is increasingly being used by health care professionals to efficiently deliver health information and test results to patients. In general, patients report that viewing personal health records enhances their sense of control over their health condition.2 Technology used for communicating test results, including text messaging, can reduce time to treatment.3 Web-based messaging reduces staff time spent answering patients' telephone calls4 as well as unnecessary patient visits.5 The Internet has also been utilized by STI clinics in key prevention strategies, such as partner notification.6,7 Offering automated test results is one use of online communication that patients prefer8,9 and provides efficient access to disease status information. An online results system offers 2 potential benefits. First, it allows patients access to their test results 24 hours a day, 7 days a week, potentially increasing patient result receipt. Second, online results may improve the efficiency of clinic services by decreasing staff time spent answering phone calls, which currently is the predominant mode of providing results in STI clinics. However, many providers are unprepared for the technology shift,10 and patients admit to carefully weighing benefits and drawbacks when considering the Internet for personal use in keeping track of medical records.11 Challenges in using the Internet in patient communication include the financial burden to the institution offering the service, logistical problems in utilizing technology, as well as the need for additional investments in data, communication, and operations management.12 Considering the potential advantages and disadvantages of introducing technology into the clinical setting, services providing test results online should be evaluated as a tool for improving both patient care and clinic operations.

The San Francisco Department of Public Health's STI Clinic started offering STI results online in 2004 in an attempt to improve patient receipt of results and increase treatment rates. Forty percent of patients initially accepted the service in 2004. The clinic demonstrated that offering online results was associated with an increase in the number of patients that received STI results and a decrease in the number of patients who requested results over the phone.13 Similarly, the Amsterdam Public Health STI clinic began offering online results in September of 2007 with more than 70% of patients receiving results online. Another 23% received results over the phone and 5% did not receive results at all. Online usage was the same by gender and sexual orientation groups, but younger populations and those with a higher frequency of negative STI results utilized online results more often.14 However, the benefit of online STI results has yet to be demonstrated in clinics other than the San Francisco and Amsterdam Public Health STI clinics. Moreover, these clinics serve large numbers of men who have sex with men, resulting in a demographic composition that may be different from populations in the average STI clinic in the United States, especially with regards to their familiarity navigating the Internet. It is therefore important to assess the utility of offering STI results online in clinics that are more representative of STI clinics in the United States.

Because of its potential benefits, an online results system was implemented at the Denver Metro Health (STI) Clinic (DMHC) on June 11, 2008, allowing patients to opt for online results by creating their own access code. On the basis of the findings from the ongoing evaluation reported in this manuscript, on December 29, 2008, the clinic began assigning every patient an access code, making the online system the standard method for receiving results. The main objectives of this study were to determine if offering online STI results, either as an option or by assignment was associated with (1) changes in the proportion of all patients receiving chlamydia and/or gonorrhea test results and (2) a decrease in the proportion of patients calling the clinic for test results. We also ascertained reasons among clinic patients for accepting or declining the option to obtain test results online.

Back to Top | Article Outline


The DMHC is the largest STI clinic in the Rocky Mountain Region, logging over 15,000 patient visits per year. In 2008, the clinic diagnosed 2089 cases of chlamydia and 744 cases of gonorrhea infections (unpublished observations; DMHC). Since chlamydia and gonorrhea infections are the most common treatable infections diagnosed in the DMHC and account for the largest burden of STI morbidity in the Denver area, we chose to use these infections to evaluate the utility of the online results system. Two datasets were used for the evaluation: the clinic's electronic medical record (EMR) system and a survey that is conducted on a regular basis among patients of the DMHC. The study was reviewed and determined exempt by the Colorado Multiple Institutional Review Board.

Back to Top | Article Outline

Evaluation Using the Clinic Electronic Medical Record System

A 17-month time span (December 1, 2007 to April 30, 2009) was selected to evaluate 3 time periods: Period 1, before implementation of optional online results (December 1, 2007 to June 10, 2008), Period 2, during the optional online results timeframe (June 11, 2008 to December 28, 2008) and Period 3, during automatic online results assignment (December 29, 2008 to April 30, 2009). Initially, to access optional online results, patients voluntarily created an access code that was entered into the clinic EMR system with their visit information at registration. Patients were then instructed to access online results 1 week from their visit date by using their access code, a unique patient log number, and their month and year of birth. During Period 3, patients were automatically given a card at registration with an assigned access code and login information that they used as described earlier. Patients who had limited Internet access or did not speak English were instructed to call the clinic for results. Regardless of opting for online results, access to telephone results was available to all patients during all 3 periods.

The DMHC electronic medical record system was used to determine whether patients elected or were instructed to receive chlamydia and gonorrhea results (online vs. via the telephone) and whether they successfully received their results. Patients were excluded if they were (presumptively) treated for chlamydia and/or gonorrhea on the day of initial visit, for example, if they had been exposed to individuals with chlamydia or gonorrhea infections, or if they were diagnosed with symptoms compatible with either infection. These patients would have received adequate treatment on the day of the visit and may have had less incentive for obtaining test results. Patients were also excluded if they were non-English speaking, as there is currently only an English version of the online results system.

Subject demographic information was provided by the clinic's EMR. Chi square tests were used to determine population differences between subjects enrolled in Period 1, Period 2, and Period 3. Logistic regression determined whether there were differences in the proportion of patients receiving results between periods. Similarly, differences in proportions of patients calling the clinic between the 3 periods were assessed.

A series of multivariate analyses were performed to assess the association between receiving results and demographic characteristics including gender identification (male, female, men who have sex with men), race/ethnicity (white, black, Hispanic, other), age group (under 20, age 20 and older), as well as chlamydia and/or gonorrhea test result status (positive, negative) and online results utilization. Additional frequency calculations and χ2 tests were executed to explore the relationship between test result status and receiving results. A significance level of 0.05 was used in analyses. SAS software, version 8 for Windows, was used to perform all logistic regression and χ2 analyses (SAS Institute Inc, Cary, NC).

As it is arguably more important for those who are infected with an STI to receive results, we conducted a separate analysis to investigate the outcomes of patients who had a positive chlamydia and/or gonorrhea diagnosis. For this analysis only, we also included those patients who were presumptively treated for chlamydia and/or gonorrhea on the day of their initial visit.

Back to Top | Article Outline

STI Clinic Survey

As part of ongoing clinic evaluation and quality assurance, the DMHC conducts periodic surveys among its patients. Surveys are programmed for computer self-administration using Questionnaire Development Systems software (Nova Research Company, Bethesda, MD). Using a systematic sampling scheme, approximately 500 patients are consented and enrolled in each survey administration. Survey data are linked to patients' medical record using a unique identifier; however, the resulting database does not contain any private health information. To date, 3 surveys have been completed. The data used in the present analysis were from the third survey conducted during Period 2 between August 21, 2008 and December 28, 2008. Every third patient visiting for a diagnosis problem who declared English as a primary language was asked to participate.

The survey included 48 questions and generally took less than 5 minutes to complete. Four questions pertained to the online results system including whether a person had created a password to access results online, and if not, why he/she chose not to receive STI results online. In addition, the most important reasons for either declining or accepting online results were elicited.

We calculated frequencies of responses to survey questions to quantify patient opinions. Demographic data of survey participants were compared to demographic data from the overall STI clinic population using the clinic's EMR to determine if a representative sample was obtained. Finally, logistic regression was used to determine the association between reported acceptance of online results and demographic characteristics.

Back to Top | Article Outline


Electronic Medical Record

Between December 1, 2007 and April 30, 2009, there were 12,494 unduplicated new visits by English-speaking patients that included a chlamydia and/or gonorrhea test. Of these, presumptive treatment for chlamydia and/or gonorrhea was given to patients at 3438 (28%) visits, leaving 9056 (72%) visits that did not involve presumptive treatment on the day of visit. Patients tested in Periods 1 (n = 3624), 2 (n = 3931), and 3 (n = 1501) were demographically similar. However, minor gender differences across periods were noted (Table 1).

After optional online results were offered (Period 2), 1616 of 3931 (41%) opted to receive online results. The remainder was considered to have opted for telephone results. After online results were assigned (Period 3), 1292 of 1501 (86%) accepted an assigned access code. The remainder was considered to have been unable to sign up for online results. Among those in Period 2 who opted for online results, 1198 of 1616 (74%) actually received results compared to 1431/2315 (62%) of those who had opted to receive results through the telephone (P < 0.0001). Among those in Period 3 who were assigned online results, 960 of 1292 (74%) received results compared to those who were unable to sign up, 85 of 209 (41%) (P < 0.0001). However, the overall proportion of patients who received their results (either by phone or online) did not change significantly before (2446/3624; 67%) or after (2629/3931; 67%) the online option became available, neither did it change significantly after online results were assigned (1045/1501; 70%). The proportion of patients calling the clinic for results decreased significantly from 2446/3624 (67%) before optional online results were offered to 1985/3931 (51%) after optional online results were offered (P < 0.0001) and decreased further once online results were assigned 537/1501 (36%) compared to Period 2 (P < 0.0001). A display of these data can be seen in Table 2. Trends in result receipt over the 3 periods can be seen in Figure 1.

To further elucidate the associations we found between opting for online results or being assigned online results and receiving results, we conducted a series of multivariate analyses. For those in Period 2, receiving results was significantly associated with being 20 years of age or older, white or in the other race/ethnicity category, and opting for online results. After controlling for these demographic factors, opting for online results remained significantly associated with actually receiving results (adjusted odds ratio: 1.7, 95% confidence interval: 1.5–2.0). Among those in Period 3, receiving results was significantly associated with being white and opting for online results (adjusted odds ratio: 4.3, 95% confidence interval: 3.2–5.9, Table 3).

During Periods 1, 2, and 3, respectively 916, 1073, and 367 patients were diagnosed with either chlamydia or gonorrhea. Of these, 623 (68%), 708 (66%), and 220 (60%) were treated on the day of visits, leaving 293, 365, and 147 patients who were not treated on the day of visit. Of the latter group, respectively 193 (66%), 240 (66%), and 110 (75%) received their test results (P = 0.08), and of these 0%, 16%, and 41% received their results online (data not shown). Of these patients for whom the receipt of results was documented, treatment within 30 days of diagnosis was confirmed for 81.3%, 82.1%, and 70.9% (NS). Of the group of patients for whom the receipt of results was not documented, very few (<5% in all 3 groups) had treatment recorded within 30 days of diagnosis.

Back to Top | Article Outline

STI Clinic Survey Results

Of 442 patients asked to complete the survey during Period 2, 429 (97%) agreed to participate. The demographic composition of survey participants was similar to that of general clinic patients (data not shown). Thirty-six percent of surveyed patients reported signing up for the online results service on that day. The main reasons given for accepting the service were that results could be accessed any time of day (75%) and patients believed they would receive results faster than calling (37%). Main reasons given for declining the service were that patients preferred to call the clinic (43%) or that they had limited access to the Internet (32%, Table 4). In response to the open-ended question, “What is the most important reason you declined online results,” the most frequent response (47% agreement) was that patients had limited Internet or computer access.

Back to Top | Article Outline


In our study, opting for and/or being assigned online results was positively associated with actually receiving test results. Yet, the overall proportion of patients receiving results did not increase after the online results system was implemented. A reasonable explanation for these paradoxical findings is that online results appealed to patients who were technologically knowledgeable and proactive and therefore more likely to receive results anyway. Consequently, a large proportion of people who would have obtained results regardless of modality may have opted for online results. However, after adjusting for demographic variables that might predict results receipt, opting for online results was still associated with receiving results. Thus, the association between opting for online results and receiving results must be explained by factors that were not measured, such as personal characteristics of patients.

The online service appeared to improve clinic operational efficiency by reducing the proportion of phone calls to the clinic pertaining to chlamydia and gonorrhea results, potentially reducing staff time spent answering these calls. The majority of patients attending the STI clinic have negative chlamydia and gonorrhea test results, and so the online service is a feasible method for providing results that do not necessitate staff involvement. Although a formal cost analysis was beyond the scope of this evaluation, we believe that the online service may have resulted in significant clinic savings. To illustrate, a reduction of clinic calls for test results by approximately 50% (as was found in our study) in a clinic like the DMHC that conducts 15,000 chlamydia/gonorrhea tests per year, could result in freeing up 30% to 50% of clerical time that may be spent on other tasks. Obviously, these potential savings need to be weighed against the costs and availability of resources that allow for the development of an online results system. In our experience, having an electronic medical records system was a prerequisite for the development of the online results program. The cost of developing the online results interface (approximately $60,000) was compounded by the desire of our funders to arrive at a stand-alone module that could be adapted for use in other clinics at minimal cost. We are currently in the process of working with other clinics to test the applicability of our online system in other settings.

The individual unknowingly infected with chlamydia or gonorrhea is at risk of ongoing transmission, and so the considerable percentage of infected patients who did not receive results in all 3 periods (10%–12% among all infected persons and 25%–34% among infected persons not treated on the day of visit) is of concern. Whereas assigning access codes in Period 3 appeared to increase the overall receipt of results, there was a concomitant drop in the number of infected patients for whom treatment could be documented after receipt of results during this period. Although these trends were not significant, and while the number of observations in Period 3 was relatively small, our data do indicate that a significant number of patients with chlamydia or gonorrhea that are not presumptively treated on the day of visit, remain untreated. Our clinic uses several methods to contact patients for treatment; however, interventions need to be considered to further ensure notification and treatment of infected patients.

Survey responses suggest that limited Internet and computer access is a deterrent for patients considering online results. Perhaps recommending public locations where patients can access the Internet would encourage participation. Of note, data from previous clinic surveys indicate that computer and Internet access has increased steadily in the past 10 years (unpublished data, DMHC). However, data from this study also indicate that, despite these developments, remnants of the “digital divide” persist. An additional limitation of our study is that our online results system has only been accessible to English-speaking patients (a Spanish version is being developed). Nonetheless, to accommodate all patients, test results must continue to be available via the telephone or in person.

In conclusion, offering online results successfully decreased the proportion of patients calling the clinic for results and did not affect the overall proportion of patients who received results. Further, we found that a de facto “opt-out” Web-based test results system where patients are assigned access codes is more effective than an “opt-in” system where patients sign up with personally created access codes. Future research should focus on determining whether online results or other patient communication technology could further increase the proportion of infected patients receiving results. Our study also suggests that a formal cost analysis would be helpful when considering the adoption of an online results system.

Back to Top | Article Outline


1. Workowski KA, Berman SM. Sexually transmitted infections treatment guidelines, 2006. MMWR Recomm Rep 2006; 55:1–94.
2. Baldry M, Cheal C, Fisher B, et al. Giving patients their own records in general practice: Experience of patients and staff. Br Med J (Clin Res Ed) 1986; 292:596–598.
3. Menon-Johansson AS, McNaught F, Mandalia S, et al. Texting reduces time to treatment for genital Chlamydia trachomatis infection. Sex Transm Infect 2006; 82:49–51.
4. Liederman EM, Lee JC, Baquero VH, et al. The impact of patient-physician Web-messaging on provider productivity. J Healthc Inf Manag 2005; 19:81–86.
5. Bergmo TS, Kummervold PE, Gammon D, et al. Electronic patient-provider communication: Will it offset office visits and telephone consultations in primary care? Int J Med Inform 2005; 74:705–710.
6. Mimiaga MJ, Tetu AM, Gortmaker S, et al. HIV and STD status among MSM and attitudes about Internet partner notification for STD exposure. Sex Transm Infect 2008; 35:111–116.
7. Tomnay JE, Pitts MK, Fairley CK. General practitioners' use of Internet-based patient materials for partner notification. Sex Transm Dis 2007; 34:613–616.
8. Matheny ME, Gandhi TK, Orav EJ, et al. Impact of an automated test results management system on patients' satisfaction about test result communication. Arch Intern Med 2007; 167:2233–2239.
9. Ralston JD, Carrell D, Reid R, et al. Patient Web services integrated with a shared medical record: Patient use and satisfaction. J Am Med Inform Assoc 2007; 14:798–806.
10. Kassirer JP. Patients, physicians, and the Internet. Health Aff 2000; 19:115–123.
11. Greenhalgh T, Wood GW, Bratan T, et al. Patients' attitudes to the summary care record and HealthSpace: Qualitative study. BMJ 2008; 336:1290–1295.
12. Katz SJ, Moyer CA. The emerging role of online communication between patients and their providers. J Gen Intern Med 2004; 19:978–983.
13. Kohn R, Williams D, Klausner JD. Getting STD test results over the Internet: San Francisco, 2005. 2006 National STD Prevention Conference, Jacksonville, Fl, 2006. Abstract #143. Available at: Accessed August 4, 2008.
14. Jaarverslag 2007 Soa Polikliniek GGD Amsterdam (Annual Report 2007 STI Clinic, Amsterdam Health Department - in Dutch).
© Copyright 2010 American Sexually Transmitted Diseases Association