The Institute of Medicine's 1997 report on the need for a national response to unacceptably high rates of sexually transmitted diseases (STDs)1 and the Centers for Disease Control and Prevention's (CDC) 2002, 2006, and soon-to-be-released 2010 STD Treatment Guidelines2,3 underscore the need for health care providers to improve STD prevention and care. However, only 20% to 35% of primary care providers perform recommended STD/human immunodeficiency virus (HIV) risk assessment, screening, and treatment.4–11 Furthermore, the persistently high incidence of chlamydia, growing racial and ethnic disparities in STD rates, and the resurgence of syphilis12 suggest that better prevention, detection, and treatment of STDs are still needed, more than a decade after the Institute of Medicine report.
Efforts to increase adoption of STD guidelines by updating providers' knowledge through didactic presentations and printed materials have been insufficient to change clinical practices.13–15 However, adding skills practice16–21 and one-on-one academic detailing22–24 have resulted in measurable improvements. Training that addresses worksite obstacles to new practices also promotes change.11,20,21 It is not known whether combined experiential and didactic training that addresses worksite obstacles can lead to changes in clinical STD practice. This evaluation assesses whether participation in an experiential–didactic 3-day training presented by the Denver STD/HIV Prevention Training Center (PTC) improved knowledge and skills, and increased frequency of performing recommended STD risk assessment, screening, and treatment in clinical settings 6 months after training.
The Denver PTC, as part of CDCs National Network of STD/HIV PTCs, offers clinical STD training to small groups of clinicians who serve patients at heightened risk for STDs.25 The 3-day training incorporates lectures, videos, case studies, discussions of cutting-edge STD diagnostic and treatment protocols and 10 hours of practicum experience, case management discussions, and laboratory demonstrations. The practicum resembles academic detailing with one-on-one mentoring in an STD clinic. Training objectives include disseminating STD guidelines, developing clinical skills, and helping clinicians overcome barriers to change.
Between 2001 and 2004, the Denver PTC presented 27 3-day STD trainings for 110 clinicians engaged in STD care. Participants self-selected to attend or were sent by their employer. Participants completed a demographic survey, pretraining assessment of knowledge, skills, and STD clinical practices, and immediate post-training assessment of knowledge and skills. Six months later, a survey assessing the same knowledge, skills, and clinical practices was mailed to 95 participants who had completed the demographic survey, at least 2 of 3 pretraining assessments, and at least 1 immediate post-training assessment. The survey also asked how often participants faced specific barriers to implementation.
Knowledge was consistently measured by percentage of correct responses to 20 multiple choice questions based on case vignettes covering nongonococcal urethritis, secondary syphilis, genital warts, herpes simplex virus infection, and bacterial vaginosis. Missing responses counted as incorrect. The general linear model for repeated measures with Bonferroni adjustment was used to compare knowledge scores pre-, post-, and 6 months post-training for 53 sets of matched responses. One-way analysis of variance and Kruskal-Wallis analysis were used to compare gains in knowledge among participants in practice ≤5 years, 6 to 10 years, and ≥11 years.
Participants rated their proficiency to perform 27 clinical skills using a scale of “poor” (1), “fair” (2), “good” (3), “very good” (4), and “excellent” (5) before, immediately after, and 6 months post-training. Skill proficiency was analyzed using the following 3 scales: communication/risk assessment; (6 items including risk assessment, sexual history, partner notification; Cronbach α 0.863; N = 28 matched responses); clinical examination/specimen collection (15 items including male examination, female examination, and specimen collection/preparation; Cronbach α 0.993; N = 26); and diagnosis/treatment (6 items; Cronbach α 0.914; N = 27). Likert scale ordinal data for skills were treated as interval data because they met normal distribution assumptions.26 Response values were summed for each scale; missing data counted as zero. The generalized linear model with Bonferroni adjustment was used to compare mean scores for matched responses for each skill and each scale across the 3 time periods.
Respondents self-reported how often they performed 4 sets of recommended STD/HIV practices before and 6 months after training. Response options were “never” (1), “rarely” (2), “sometimes” (3), “usually” (4), and “always” (5). Communication practices focused on sexual history and risk assessment. The second set of practices included screening asymptomatic sexually active females and males with new or multiple partners for chlamydia or gonorrhea. The third set concerned performing wet mounts for symptomatic females and ordering HIV and syphilis serology. The fourth set asked trainees who had seen a patient with a genital ulcerative lesion how frequently they ordered specific diagnostic tests.
Mean frequencies of clinical practices 6 months post-training were compared to pretraining means using paired samples t test. As not all respondents cared for males, females, adolescents, and adults, the number of matched responses for the 4 sets of practices varied from 34 to 12. Cases with >20% missing values for any scale were excluded.27 For communication practices, if 1 or 2 values were missing, person mean substitution was used to maintain respondent variability and psychometric properties of the scale.27,28 For diagnosing and treating genital ulcerative lesions, medians and nonparametric Wilcoxon rank sign were used to analyze the 12 matched responses.
Nurse practitioners, physician assistants, and registered nurses accounted for 77% of trainees. An additional 8% were physicians. Most participants were females (84%) and white (79%); 11% identified as Latino/a. The majority (65%) worked in a state in which chlamydia and gonorrhea rates exceeded the national average at the time.29 Participants worked primarily in public health (44%) and private practice (17.2%) settings although students from academic, family planning, correctional facilities, Indian Health Services, and managed care settings also attended. On average, trainees devoted 31% of their time to STD/HIV care (range: 0%–100% since some were not yet in practice). About half (51%) had worked in their primary occupation for 5 years or less. Gender, race, profession, clinical role, employment setting, and education level did not significantly differ among the 62 who responded to the 6-month survey (65% response rate) and the nonresponders.
Knowledge Attainment and Retention
Mean knowledge scores increased from 58% correct pretraining to 75% post-training (P < 0.001), and declined to 69% 6 months later (not statistically significant). After 6 months, STD knowledge was significantly greater than pretraining (P = 0.002). Immediate postcourse knowledge scores did not differ between those who did and did not complete the 6-month follow-up period (P = 0.195). Six-month gains in knowledge from pretraining did not differ on the basis of years of experience in STD care (P = 0.446 comparing ≤5 years [N = 23], 6–10 years [N = 9], ≥11 years [N = 8]).
Skill Proficiency and Skill Retention
Findings about skills and practices are all based on self-reports. Figure 1 illustrates significant self-reported gains in skill proficiency (P < 0.001) from pretraining to immediately and 6 months after training for all 3 skill scales: communication/risk assessment; clinical examination; and diagnosis/treatment. Self-reported proficiency gains from immediately post-training to 6 months after were not statistically significant.
Skill proficiency gains 6-months post-training were significant for each of the 27 skills (P < 0.05). For communication/risk assessment skills, mean gains ranged from 0.5 to 1.46 Likert scale increments and 6-month mean proficiency exceeded very good. The largest gain was for discussing partner notification which improved from good (2.86) pretraining to very good (4.32). Six-month mean proficiency was lowest for eliciting a sexual history (4.04 = very good) although there was significant improvement from before training (3.04 = good).
Six-month mean proficiency was very good to excellent for 5 clinical examination skills, and good to very good for the remaining 10. Detecting tenderness or palpating masses in the uterus and adnexa improved 1.7 increments to very good (4.32). Palpating scrotal contents and inspecting the penis during the male examination improved from fair pretraining (2.27 and 2.39, respectively) to very good (3.81) at 6 months. At all time points, mean proficiency for performing an anorectal examination rated lowest but improved 1.05 increments from fair (1.95) pretraining to good (3.0) 6 months later. Proficiency applying universal precautions increased 1.38 increments from good pretraining (3.31) to very good (4.39) at 6 months.
Significant gains in self-reported diagnostic and treatment proficiency ranged from 1.19 to 1.67 increments. The 3 largest improvements (exceeding 1.5 increments) were for making appropriate diagnoses, prescribing proper medications, and collecting specimens to diagnose genital ulcer disease.
Changes in Clinical Practices
Six months post-training, participants reported greater frequency of performing all practices except laboratory tests for genital ulcerative lesions. Increases were statistically significant for 23 of 35 practices (P < 0.05). Six months post-training, a significantly greater number of trained clinicians reported usually or always asking adult patients about their STD/HIV history, number of recent sex partners, use of condoms, reproductive history (for female patients), and partners' STD/HIV risk history (P < 0.05) (Table 1). Frequency of asking about recreational drug use increased but bordered on significance (P = 0.05). Asking about partner gender increased but not significantly.
Six months post-training, participants reported asking adolescent patients significantly more often about the number and gender of sex partners and partners' STD/HIV risk history. The frequency of asking adolescents about drug use increased, but not significantly. Before training, participants reported that they already asked adult and adolescent patients usually or always about condom use, STD risk factors, and female reproductive history.
At 6 months, participants reported screening asymptomatic females for chlamydia and gonorrhea significantly more often (Table 2). Specifically, respondents reported usually or always screening for chlamydia and gonorrhea in asymptomatic females if pregnant, under age 26, or reporting a new sexual partner in the preceding 3 months. For women with exclusively female sex partners, asymptomatic screening increased from a mean between rarely and sometimes pretraining to between sometimes and usually at 6 months.
After 6 months, respondents reported screening of each risk category of asymptomatic males for gonorrhea significantly more frequently than before training. Self-reported chlamydia screening increased significantly only for sexually active asymptomatic males younger than 26 years.
Six-month self-reported changes in ordering or performing appropriate laboratory tests for asymptomatic patients reporting high-risk behaviors showed significant improvement for ordering syphilis serology and performing wet mounts for females (Table 3). However, mean post-training frequency of ordering syphilis and HIV serology for asymptomatic high-risk patients was less than sometimes, and was sometimes for performing wet mounts for asymptomatic high-risk females. The 12 clinicians who had seen patients with a genital ulcer both before and 6 months after training reported no significant post-training change in mean frequency of ordering dark field microscopy, STAT RPR, chancroid culture, or MHA-TP or FTA-ABS laboratory tests.
A total of 53 participants indicated how the training changed STD care at their worksites. Over one-third had shared STD prevention or treatment information with colleagues (39.6%). Half (50.9%) reported improved STD clinic practices and policies, 58.4% reported increased resources for STD diagnosis and treatment, and 49.1% reported somewhat or a lot of increased coworker interest in STD care. Self-reported changes in participants' clinical practices included improved: patient STD education (71.7%); female STD examinations (67.9%); male STD examinations (47.2%); identification of patients needing STD screening (64.1%); and following best-practice STD treatment protocols (60.3%).
A total of 53 respondents reported barriers they faced trying to implement recommended practices. Barriers included the following: inadequate time for partner evaluation (52.9%) and for risk assessment and counseling (43.4%); inadequate facilities or equipment (51.5%); and inadequate staffing (47.1%). For 37.7%, treatment was not available sometimes, often, or always due to cost or access.
This study asked whether clinician knowledge, skills, and performance of recommended guidelines changed after attending an interactive 3-day experiential–didactic STD/HIV training. Findings showed that knowledge and skills needed for competent STD diagnosis, treatment, and risk-reduction counseling increased. The self-reported frequency of performing recommended screening and treatment also increased modestly 6 months post-training, which is consistent with other studies showing that experiential training improves clinical practices.23,30,31
In this evaluation, STD knowledge, self-reported skills, and self-reported practices improved for physicians, physician assistants, nurses, and nurse practitioners whether new to the field or seasoned clinicians. This is important for 2 reasons. First, it indicates that improvement was not just a result of the natural learning curve expected for those new to the field. Second, it shows the potential for experiential–didactic training to improve STD practices among clinicians with varying levels of clinical STD experience.
Despite solid mean gains in knowledge and skills, for some, knowledge attainment missed the desired 85% mark and skill proficiency fell short of the desired very good or excellent level 6 months post-training. In response, training content, delivery, and practicum experience have been strengthened to address shortcomings. Training centers may need to offer booster sessions to participants demonstrating insufficient knowledge or skills post-training to increase their capacity to change clinical practices.
At first glance, several practice patterns appear lagging. But increasing chlamydia and gonorrhea screening of asymptomatic males from between rarely and sometimes pretraining to between sometimes and usually at 6 months may be appropriate. The 2006 STD Treatment Guidelines3 suggest that clinicians consider asymptomatic males' risk behaviors, STD/HIV history, partner STD/HIV history, as well as local prevalence when deciding whom to screen for chlamydia and gonorrhea. Accordingly, not all asymptomatic males should be screened even if they are young, have a new partner, or have sex with men.
It is not clear why respondents 6 months post-training reported only rarely or sometimes screening asymptomatic patients for HIV and syphilis. These responses may reflect a flaw in the evaluation that omitted wording indicating the screening was for new asymptomatic patients with high-risk behaviors. Without those qualifiers, rarely and sometimes may accurately reflect providers' intention to consider asymptomatic patients' risk history and testing history before ordering syphilis or HIV serology. It is also difficult to interpret the lack of change in ordering ulcerative disease diagnostics after training as only 12 trainees reported examination of patients with genital ulcers.
The pending release of the CDC's 2010 STD Treatment Guidelines in 2010 highlights the importance of identifying effective ways to disseminate guidelines since only 20% to 35% of US primary care providers currently practice recommended STD screening of asymptomatic women.4–11 Six months post-training, clinicians who attended the training reported changing from sometimes to usually conducting chlamydia and gonorrhea screening of asymptomatic women aged <26 and women with a new sex partner. Screening asymptomatic women at their first prenatal visit increased from usually to always. Although the sample providing information on screening practices is small, these findings suggest that experiential–didactic training can increase recommended STD screening to decrease partner transmission, morbidity in pregnant women, and transmission to newborns.
Although trainings attempted to help participants anticipate and overcome system-level obstacles in implementing clinical guidelines in the real world, barriers persisted for many. Of the 53 who responded to this question, approximately half were hindered sometimes, often, or always by inadequate time and staffing for risk assessment, counseling, and partner services, and/or by inadequate facilities or equipment. Such barriers indicate that improving STD/HIV clinical knowledge and skills alone may be insufficient to change practices widely enough to reduce STD incidence.
The PTCs are an eminent training resource for building the capacity of the nation's clinicians in essential hands-on STD examination and diagnostic skills. However, as individual- level training is resource intensive and limited in its ability to reach all clinicians who would benefit from training, training centers may be able to amplify their effect by promoting systems-level change. The findings from this evaluation suggest that individual trainees are well aware of obstacles to system-wide implementation of current practice guidelines, and many would welcome training to address such challenges. Accordingly, the Denver PTC has begun providing tailored training and technical assistance to STD clinic managers to promote systems-level improvements in care.
The PTCs' extensive experience and expertise put them in a unique position to offer clinic-level trainings, such as Partnership for Health, that not only train individual clinicians in prevention skills (in this case, HIV prevention with patients living with HIV/AIDS), but also give all clinic staff members a role in implementing and supporting the intervention. Such an approach may be particularly beneficial if health care reform shifts greater responsibility for STD diagnosis and treatment to primary care physicians and physician extenders, who currently may not perceive an individual need for in-depth STD training.
Training centers can also take a central role in disseminating innovative interventions of a more structural nature that address barriers and promote recommended practices. For example, at the Denver Metro Health STD clinic, the electronic medical record has introduced algorithm-based decision aids that prompt clinicians to take certain actions when indicated. Examples include a checklist of topics for client-centered counseling, a prompt to suggest expedited therapy for partners of patients diagnosed with chlamydia or gonorrhea, and a prompt to facilitate reproductive health counseling for eligible men and women. Another systems-level innovation involves a triage system that fast-tracks asymptomatic patients into a no-examination express visit, effectively redistributing limited clinical resources to patients needing more extensive services.32
Although the training centers have a role in diffusing such innovations, other dissemination channels should be explored to develop and distribute innovations such as a generic electronic medical record, web-based partner notification tools, and computer-assisted individual risk assessments so that clinics do not need to incur costs individually to develop their own systems. Together with individual skills training, structural interventions promise to enhance the reach and dissemination of clinical practice improvement that cannot be attained by individual trainings alone.
This study has several limitations. First, without a control group we could not definitively link practice improvements to the training. Second, the sample size constrained analysis and generalizability of results. Third, the sampling frame did not assure that participants represented the population of clinicians who see patients at risk for STD. Although the professions of trainees approximate the composition of providers caring for STD patients, indicated in the literature,7,8 and at STD clinics and family planning clinics in the Rocky Mountain region,33 most PTC trainees self-selected or were required by their employer to attend the training. Ideally, more primary care physicians would have participated in this training, but they reported difficulty justifying a 3-day training on a topic not considered a priority.
A fourth limitation was the potential for self-assessment to result in over- or under-reporting of true performance due to participants' desire to represent themselves in a positive light or give correct answers. Although studies have shown self-report by clinicians can be comparable to observed measures,34–36 the validity and reliability of self-reported measures should be tested for this course. Ideally, self-assessment would be validated by standardized observed measures of trainee performance. Preceptor observations of practicum performance during this course have lacked adequate consistency to be considered reliable. Insufficient resources have precluded rigorous evaluation with a control group, larger number of trainees, and standardized expert observation to rate clinical performance. To date, self-assessment is the most affordable measure of change in skill proficiency and performance of recommended STD practices affordable.
Although the purpose of this training is to increase rates of national adoption of STD practice guidelines and to ultimately prevent STDs, funding the rigorous measurement of those effects is not currently feasible. As nearly half the participants said they shared the training with colleagues, one strategy to amplify the effect might be to add an experiential learning component about accurately teaching colleagues about recommended practices. Conducting a more rigorous follow-up of a cohort of trainees could also measure how sharing information from the training does or does not translate into changing practice patterns in a clinical setting. A case-control study of changes in provider practices and STD rates of primary care clinics trained after the release of the 2010 Guidelines could provide a relatively low-cost assessment of the potential for experiential–didactic training to turn national practice guidelines into real-world practices at greater scale.
1.Institute of Medicine Committee on Quality of Health Care in America. The Hidden Epidemic: Confronting Sexually Transmitted Diseases. Washington, DC: National Academy Press, 1997.
2.Centers for Disease Control and Prevention. Sexually Transmitted Diseases Treatment Guidelines, 2002. MMWR Morb Mortal Wkly Rep 2002; 51:1–82.
3.Centers for Disease Control and Prevention. Sexually Transmitted Diseases Treatment Guidelines, 2006. MMWR Morb Mortal Wkly Rep 2006; 55:1–94.
4.Burstein GR, Lowry R, Klein JD, et al. Missed opportunities for sexually transmitted diseases, human immunodeficiency virus, and pregnancy prevention services during adolescent health supervision visits. Pediatrics 2003; 111:996–1001.
5.Gift TL, Hogben M. Emergency department sexually transmitted disease and human immunodeficiency virus screening: Findings from a national survey. Acad Emerg Med 2006; 13:993–996.
6.Huppert J, Goodman E, Khoury J, et al. Sexually transmitted infections screening test rates differ by symptoms and clinic specialty. Obstet Gynecol 2003; 101:103S.
7.Montaño DE, Phillips WR, Kasprzyk D, et al. STD/HIV prevention practices among primary care clinicians: Risk assessment, prevention counseling, and testing. Sex Transm Dis 2008; 35:154–166.
8.Moran JS, Kaufman JA, Felsenstein D. Survey of health care providers: Who sees patients needing STD services, and what services do they provide? Sex Transm Dis 1995; 22:67–69.
9.St. Lawrence JS, Montano DE, Kasprzyk D, et al. STD Screening, testing, case reporting, and clinical and partner notification practices: A national survey of US physicians. Am J Public Health 2002; 92:1784–1788.
10.Wimberly YH, Hogben M. Physicians' STD diagnosis and screening practices in the South. South Med J 2004; 97:624–630.
11.Wiesenfeld HC, Dennard-Hall K, Cook R, et al. Knowledge about sexually transmitted diseases in women among primary care physicians. Sex Transm Dis 2005; 32:649–653.
12.Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance, 2008. Atlanta, GA: US Department of Health and Human Services, 2009.
13.Glasziou P, Haynes B. The paths from research to improved health outcomes. Evid Based Med 2005; 10:4–7.
14.Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.
15.Ockene JK, Zapka JG. Provider education to promote implementation of clinical practice guidelines. Chest 2000; 118(suppl 2):33S–39S.
16.Beach MC, Gary TL, Price EG, et al. Improving health care quality for racial/ethnic minorities: A systematic review of the best evidence regarding provider and organization interventions. BMC Public Health 2006; 6:104.
17.Bero LA, Grilli R, Grimshaw JM, et al. Closing the gap between research and practice: An overview of systematic reviews of interventions to promote the implementation of research findings. BMJ 1998; 317:465–468.
18.Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health. A review of systematic reviews. Int J Technol Assess Health Care 2005; 21:380–385.
19.Davis DA, Mazmanian PE. Continuing medical education and the physician as a learner: Guide to the evidence. JAMA 2002; 288:1057–1060.
20.Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: An overview of systematic reviews of interventions. Med Care 2001;39 (suppl 2):II2–II45.
21.Grimshaw JM, Eccles MP, Walker AE, et al. Changing physicians' behavior: What works and thoughts on getting more things to work. J Contin Educ Health Prof 2002; 22:237–243.
22.Davis D, O'Brien MA, Freemantle N, et al. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999; 282:867–874.
23.Davis DA, Thomson MA, Oxman AD, et al. Changing physician performance. A systematic review of the effect of continuing education strategies. JAMA 1995; 274:700–705.
24.Davis NL, Willis CE. A new metric for continuing medical education credit. J Contin Educ Health Prof 2004; 24:139–144.
25.Judson FN, Boyd WA. The Denver Sexually Transmitted Diseases Prevention Training Center: A two-year performance evaluation. Sex Transm Dis 1982; 9:183–187.
26.Jaccard J, Choi W. LISREL Approaches to Interaction Effects in Multiple Regression. Thousand Oaks, CA: Sage Publication, 1996.
27.Downey RG, King CV. Missing data in Likert ratings: A comparison of replacement methods. J Gen Psychol 1998; 125:175–191.
28.Roth PL, Switzer FS, Switzer DM. Missing data in multiple items scales: A Monte Carlo analysis of missing data techniques. Organ Res Methods 1999; 2:211–232.
29.Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance, 2005. Atlanta, GA: US Department of Health and Human Services, 2006.
30.Cantillon P, Jones R. Does continuing medical education in general practice make a difference? BMJ 1999; 318:1276–1279.
31.Forestlund L, Bjornal A, Rashidian A, et al. Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009, Issue 2. Art No.: CD003030.
32.Shamos SJ, Mettenbrink CJ, Subiadur JA, et al. Evaluation of a testing-only “express” visit option to enhance efficacy in a busy STI clinic. Sex Transm Dis 2008; 35:336–340.
33.Denver STD/HIV Prevention Training Center. Region VIII Clinical Advisory Meeting Report. Denver STD/HIV Prevention Training Center, 2009.
34.D'Eon MF, Sadownik L, Harrison A, et al. Using self-assessments to detect workshop success. Am J Eval 2008; 29:92–99.
35.Davis DA, Mazmanian PE, Fordis M, et al. Accuracy of physician self-assessment compared with observed measures of competence. JAMA 2006; 296:1094–1102.
36.Biernat K, Simpson D, Duthie E, et al. Primary care residents self assessment skills in dementia. Adv Health Sci Educ Theory Pract 2003; 8:105–110.