Teaching Evidence-Based Medicine on a Busy Hospitalist Service: Residents Rate a Pilot Curriculum

Nicholson, Laura J. MD, PhD; Shieh, Lisa Y. MD, PhD

Academic Medicine:
Research Report

Purpose: To increase evidence-based medicine (EBM) instruction within the confines of reduced resident work hours.

Method: In 2001–02, the authors designed and implemented an EBM curriculum for residents on an inpatient medicine service at Stanford University Medical Center. Thirty-six residents were assigned the hospitalist rotation in its pilot year. Attendings introduced EBM concepts and Internet resources. During daily rounds, housestaff presented patient-based EBM literature search results. After the rotation, residents were given a questionnaire on which they were asked to rate the impact of the curriculum on their understanding of 20 EBM terms or practice skills (1 = no effect to 5 = strong effect).

Results: Twenty-three residents (64%) completed the questionnaire. The results were very positive with average effect of more than 4 (somewhat strong effect/impact) for 16 of the 20 questions. High-speed Internet access and EBM Web resources were critical to efficient delivery of the curriculum during inpatient care.

Conclusion: The pilot curriculum successfully introduced the practice of EBM during active inpatient care without requiring additional hours from housestaff schedules. To further evaluate and expand this project, EBM skills will be tested before and after the rotation, and faculty development will allow consistent delivery in additional clinical settings.

Author Information

Dr. Nicholson is assistant clinical professor of medicine, University of California, San Diego, School of Medicine, San Diego, California.

Dr. Shieh is assistant clinical professor of medicine, Stanford University School of Medicine, Palo Alto, California.

Correspondence should be addressed to Dr. Nicholson, University of California, San Diego Hospital Medicine Program, 9300 Campus Point Drive, MC 7828, La Jolla, CA 92037-7828; telephone: (858) 657-6090; e-mail: 〈lnicholson@ucsd.edu〉.

Article Outline

Evidence-based medicine (EBM) is clinical expertise combined with the conscientious and judicious use of the current best evidence in making decisions about patient care.1 A major goal in teaching EBM is to impart the skills of identifying and applying the best available evidence for use in clinical decision making. More traditional EBM curricula have used a classroom setting, following the Users’ Guides to the Medical Literature2 to demonstrate careful critique of published articles. Several authors have described these workshop and journal club formats for teaching EBM to residents.3–7 While methodical literature appraisal is important and central to understanding EBM terminology, it can leave the learner with the impression that EBM requires hours of isolated study to answer a single patient-care question. In addition, one survey of medicine residency programs found that only 37% provided dedicated curricular time for EBM,8 likely due at least in part to competing demands for residents’ time.

A criticism of EBM instruction has been its failure to demonstrate applicability during the time constraints posed by active patient care.2 Fortunately, many new tools exist to help the clinician rapidly sort through the large volume of published literature to locate the best evidence to answer a clinical question. These tools take the form of prefiltered EBM Internet resources such as ACP Journal Club©, which critiques individual articles; Cochrane Library© and Clinical Evidence©, which publish disease-based evidence reviews; and InfoPOEMs® and PIER©, which appraise articles and produce summary guidelines. With the addition of EBM limits available on the Medline® platforms Ovid® and PubMed®, a search of the best evidence can often be performed in five minutes or less.

In this study, we implemented a curriculum to teach EBM to internal medicine residents during a clinical rotation already in place. Our objective was to teach evidence-based practice during real time patient care, thereby demonstrating that it can and should occur within the time constraints of a busy inpatient service. Curricular goals included formulating well-structured EBM-style questions based on specific patients; performing information searches to include the newer EBM online resources; presenting the search results, with critical appraisal of any original articles obtained; and, where appropriate, applying the results to patient care. Additionally, we evaluated our curriculum's impact on residents’ EBM skills through residents’ self-assessment following the rotation.

Back to Top | Article Outline


From June 2001 to July 2002, we piloted the EBM curriculum at Stanford University Medical Center, Stanford, California, on a general medicine inpatient service taking admissions and consultations within a 450-bed tertiary care center. Except for some weekend coverage, we shared exclusive teaching and attending responsibility for the team. To prepare for the year, we reviewed the Users’ Guides to the Medical Literature2 and a textbook on how to practice and teach EBM.1 In 2001, we took the two-day course at the University of California San Francisco entitled “Evidence-based Medicine: A Hands-on Workshop for Clinicians and Policy Makers,” updating our Web-based resource knowledge and reinforcing our literature appraisal skills. Institutional review board approval was waived for a study of learners’ evaluation of an educational intervention.

Thirty-six of Stanford's 92 total internal medicine housestaff were assigned the hospitalist rotation in its pilot year. The participating hospitalist team rotated monthly and consisted of the attending, one second- or third-year resident, and two interns. This team was chosen for the pilot year because it was unique among the five medicine services in being attended only by Stanford's two hospitalists and accepting new admissions every day but requiring no overnight call. During rounds, a daily EBM session took place with the attending generally using the first three sessions of the month to present definitions and principles of EBM, briefly demonstrate literature appraisal, and share some EBM Internet resources. Searches about therapies and diagnostic tests were specifically emphasized, since these query types are most often raised in the course of inpatient care. Following this introduction, the housestaff were asked to formulate at least one focused clinical question for each patient admitted. They were then expected to choose two of their questions for literature searches, use EBM principles to evaluate search results, and present their findings to the group during subsequent rounds. Presentations included the original question, the search strategy and results, an appraisal of the findings, and whether the data supported any alteration in patient care (see List 1).

After the residents completed the rotation, we gave them an anonymous questionnaire regarding their learning experience on the hospitalist team. Using a five-point scale (1 = no effect; 5 = strong effect), residents were asked to rate the impact of the curriculum on their understanding of 20 EBM terms or practice skills. Residents returned the questionnaires to us by mail, some with a lag time of up to 24 weeks. During the preliminary year, data were pooled and not identified separately by time of year, time to questionnaire completion, or attending. For each question, we computed a mean response and standard deviation.

Back to Top | Article Outline


All thirty-six residents completed the curriculum. Twenty-three (64%) returned the questionnaire. Answers were very positive with an average effect of more than 4 (somewhat strong effect/impact) for 16 of the 20 questions (see Table 1). All responses, inclusive of the standard deviation ranges, were in the positive half of the scale (≥ 2.5 out of 5). Lowest scores were for the curriculum's impact on outpatient EBM use and on appraisal of meta-analysis, an article type that was evaluated only if recovered during a search and therefore not with every team. Our rotation does appear to encourage extrapolation of EBM skills to other settings, given the mean responses to questions regarding EBM for outpatients and EBM use during later rotations.

The Users’ Guides to the Medical Literature provided a consistent framework for teaching evidence appraisal. We often returned to the Users’ Guides even when prefiltered EBM resources had already critiqued the information, to check levels of validity and hierarchy of evidence. The residents frequently commented that when confronted with little time and an article to appraise on their own, the Users’ Guides tutored efficient perusal of the methods to help them quickly discard low-quality literature. This skill was mastered by repetition during the month, and was especially important for recent studies that had not yet been evaluated by the EBM Web sites.

The prefiltered EBM Web sites such as ACP Journal Club and Cochrane Library were important for rapid evidence searches. In addition, using EBM limits for Medline, such as the “Clinical Queries” service on PubMed, allowed accelerated searches for the most recent high-quality studies. With these EBM filters, the housestaff learned that highly valid evidence could be quickly obtained and applied, often without their own systematic appraisal of the individual studies. During the pilot year of the curriculum, several new EBM Internet resources became available, and a major challenge was to keep abreast of these new Web sites. In addition, some sites we initially enjoyed for free later levied expensive subscription fees. These changes meant that the residents used slightly different resources depending on when they took the curriculum. ACP Journal Club and Cochrane Library were the two preappraised resources that were consistently available from any hospital workstation throughout the study period. To use these EBM Web sites, our curriculum is fully dependent on available high-speed Internet access from computers throughout the medical center.

Back to Top | Article Outline


We found that role modeling and teaching EBM is possible on a busy inpatient service. With our curriculum added to an existing inpatient rotation, no additional dedicated course time was required within the busy resident's schedule, a major consideration given the 80-hour work week restriction. In addition, although the principles and methods of EBM were reinforced by some traditional literature appraisal during rounds, the setting of the active ward team allowed much of the curriculum to occur at the bedside. For example, PICO questions (see List 1) were often raised at the time of admission, and housestaff later described performing rapid literature searches on-the-fly using prefiltered EBM Web sites from emergency room and hospital ward computers. In this manner, residents adopted EBM skills and behaviors during active patient care rather than requiring a process transition to bring these skills to the clinical arena from the journal club or classroom settings described in other published EBM curricula.

In the questionnaire, residents reported significant improvement in their EBM skills through participation in the hospitalist service and its EBM curriculum. They also reported that they continued to use these skills on later rotations (see Table 1). In the pilot year, we chose to measure residents’ perceptions of their own improvement after taking the curriculum; a plan under way directly measures change in their EBM skills using pre- and posttesting.9 Posttesting twice, immediately after and six months after completion of the rotation, will check for sustainability of these skills. Additional measures we intend to add include how often EBM searches change the plan for the source patient's care and whether the curriculum significantly improves participating residents’ EBM knowledge compared with EBM skills in control housestaff groups taking the same pre- and posttests.

A major limitation for delivering this curriculum to a broader resident audience will be establishing and maintaining the EBM knowledge base of their teaching faculty. In order to do this, we have initiated a faculty development course focused on general EBM principles, EBM Web sites, and literature appraisal skills. Faculty members vary more than housestaff in their baseline EBM skills and confidence in using computerized resources, and we are seeking to provide an instruction standard in order to teach our curriculum to residents in many different patient care settings.

Back to Top | Article Outline


1 Sackett DL, Strauss SE, Richardson WS, et al. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. London: Churchill Livingstone, 2000.
2 Guyatt G, Rennie D. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago: AMA Press, 2002.
3 Green ML, Ellis PJ. Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med. 1997;12:742–50.
4 Smith CA, Ganschow PS, Reilly BM, et al. Teaching residents evidence-based medicineskills: a controlled trial of effectiveness and assessment of durability. J Gen Intern Med. 2000;15:710–5.
5 Cramer JS, Mahoney MC. Introducing evidence based medicine to the journal club, using a structured pre and post test: a cohort study. BMC Med Educ. 2001;1:6.
6 Ross R, VerdieckA. Introducing an evidence-based medicine curriculum into a family practice residency—is it effective? Acad Med. 2003;78:412–7.
7 Haines SJ, Nicholas JS. Teaching evidence-based medicine to surgical subspecialty residents. J Am Coll Surg. 2003;197:285–9.
8 Green ML. Evidence-based medicine training in internal medicine residency programs: a national survey. J Gen Intern Med. 2000;15:129–33.
9 Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003;326:319–21.
© 2005 Association of American Medical Colleges