Secondary Logo

Journal Logo

Connecting Education to Quality: Engaging Medical Students in the Development of Evidence-Based Clinical Decision Support Tools

Crabtree, Elizabeth A. MPH; Brennan, Emily MLIS; Davis, Amanda MPH, RD; Squires, Jerry E. MD, PhD

doi: 10.1097/ACM.0000000000001326
Innovation Reports
Free
SDC

Problem Evidence-based practice (EBP) skills are crucial for delivering high-quality patient care. It is essential that medical students learn EBP concepts through a practical, in-depth research project. To date, literature on preparing students in this manner is limited.

Approach In academic year 2014–2015, the Medical University of South Carolina’s (MUSC’s) Center for Evidence-Based Practice (now known as the Value Institute) partnered with College of Medicine faculty to revitalize the undergraduate medical student EBP curriculum. Without adding to the number of the lecture hours, the curriculum was restructured to be more process driven, project based, and clinically relevant. The resulting yearlong EBP course partnered small teams of medical students with interprofessional clinical teams to engage the students in developing evidence-based clinical decision support tools.

Outcomes The content developed during the EBP projects is currently being used to develop evidence-based clinical practice guidelines and accompanying order sets.

Next Steps It is likely that this model will serve as a new framework for guideline development and will greatly expand the breadth of evidence-based content currently produced and available for clinicians at the MUSC. It would be feasible to offer a similar course within the MUSC to other disciplines and colleges, or at other institutions, if there were support from administration, interest on the part of clinicians and medical faculty, and individuals with the required expertise available to develop the curriculum and facilitate the course. It is worth considering how to improve the course and evaluating opportunities to implement it within other settings.

E.A. Crabtree is currently director, Clinical Integration and Evidence-Based Practice, assistant professor, Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon, and PhD candidate, University of Texas School of Public Health, Houston, Texas. At the time of writing, E.A. Crabtree was director, Center for Evidence-Based Practice (now known as the Value Institute), assistant professor, Department of Library and Information Sciences and Department of Public Health Sciences, Medical University of South Carolina, Charleston, South Carolina, and PhD candidate, University of Texas School of Public Health, Houston, Texas.

E. Brennan is research informationist and assistant professor, Department of Library Science and Informatics and Department of Pediatrics, Medical University of South Carolina, Charleston, South Carolina.

A. Davis is clinical evidence-based practice analyst, Center for Evidence-Based Practice (now known as the Value Institute), Medical University of South Carolina, Charleston, South Carolina.

J.E. Squires is medical director of transfusion medicine and associate professor, Department of Pathology and Laboratory Medicine, Medical University of South Carolina, Charleston, South Carolina.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A378 and http://links.lww.com/ACADMED/A379.

Correspondence should be addressed to Elizabeth A. Crabtree, MBS 355, 3181 SW Sam Jackson Park Rd., Portland, OR 97239; telephone: (503) 494-9344; e-mail: crabteli@ohsu.edu.

Back to Top | Article Outline

Problem

Evidence-based practice (EBP) skills are crucial for delivering high-quality patient care. Clinicians must be able to search for, appraise, apply, and evaluate evidence, as well as have the ability to adapt to new information and changing circumstances throughout their careers.1 EBP is more than theory and skills; it is a way of practicing medicine and, therefore, should be integrated into student education and clinician training to influence the institutional culture.2 For these skills and culture to solidify, it is essential that medical students learn the concepts of EBP through a practical, in-depth research project.3 To date, literature on preparing students in this manner is limited.

Back to Top | Article Outline

Approach

The Medical University of South Carolina (MUSC) is a large academic medical center in Charleston, South Carolina, that consists of a 700-bed academic medical center (MUSC Health) and six health sciences colleges (including the College of Medicine, which had 181 second-year students at the beginning of academic year 2014–2015). The MUSC also has a Center for Evidence-Based Practice (CEBP, now known as the Value Institute), whose purpose is to support clinicians in providing high-quality, evidence-based care through the development of evidence-based guidelines and clinical decision support tools. The CEBP is housed jointly in the MUSC Library and the quality management department of the MUSC Medical University Hospital. In academic year 2014–2015, the CEBP partnered with College of Medicine faculty to revitalize the undergraduate medical student EBP curriculum.

To accomplish this without adding to the number of lecture hours, we restructured the second-year EBP curriculum to be more process driven, project based, and clinically relevant. It is unique for an undergraduate medical curriculum to be tied to a clinically relevant project that results in the development of clinical decision support tools for use in clinical practice even though it has been shown that clinically integrated teaching of EBP is more likely to bring about changes in skills, attitudes, and behavior than stand-alone teaching.3 Positioning this course in the second year of medical school allows the students to understand and participate in the steps of EBP before their clinical years, and to apply their findings at a system level to a population of patients. In addition, during their clinical years, students have required EBP projects in their third-year pediatrics and internal medicine clerkships, which allow them to apply and reinforce the EBP concepts learned in this course at a patient level.

In this restructured second-year EBP curriculum, we taught evidence-based skills and theory through a combination of self-paced online recordings, in-person lectures, hands-on activities, and small-group work. This mix of delivery methods ensured that faculty taught to different learning styles and that students reaped the benefits of each method, such as enhanced information retrieval skills from the hands-on project and better critical judgment from lectures.4 Such blended instructional approaches have been shown to be effective at improving student attitudes toward EBP as well as at increasing self-reported use of EBP in clinical practice.5

The resulting project-based, yearlong EBP course partnered 40 small teams of medical students with 11 interprofessional clinical teams (see below) to engage the students in developing evidence-based clinical decision support tools for the MUSC Medical University Hospital. By the end of the course, students were expected to be able to apply the steps of EBP (ask question, search literature, appraise evidence, apply evidence, and evaluate results) to complete an evidence summary.

The project topics reflected strategic organizational priorities for which the chief quality officer had identified the need for clinical practice guidelines and accompanying electronic health record order sets. To prepare for the course, the 11 interprofessional teams each identified 2 to 6 clinical questions of interest and developed 40 practice case studies for these questions. We then divided all 181 second-year medical students into 40 teams of 4 to 5 members and gave each team a case study. Using their assigned case study, the student teams framed one clinical question, searched the literature, and appraised, evaluated, and summarized evidence. (Although the interprofessional teams developed the case studies from clinical questions, we gave the student teams the case studies instead of the questions so that they could practice the EBP step of framing a question.) At the end of the academic year, the student teams completed their EBP projects by presenting their evidence summaries to the interprofessional teams.

The interprofessional teams comprised three to five clinical faculty members selected by the CEBP and/or recommended by other clinical faculty members who had previously served on a guideline content expert team. Many of the interprofessional team clinicians had completed an interprofessional EBP course offered by the CEBP and were well-versed in EBP methods. Because the student teams, with help from CEBP and library staff, focused on the research evidence component needed to develop the guidelines, the interprofessional team clinicians were selected to offer clinical expertise to the interpretation of the evidence and to represent patient values and preferences when the evidence summaries were translated into clinical decision support tools.

At the beginning of the academic year, the chief quality officer of the MUSC Medical University Hospital gave an introductory lecture on EBP, highlighting the impor tance of integrating best evidence into everyday clinical practice. Over the course of the academic year, members of the CEBP and library staff provided lectures and individual consultations on framing clinical questions, conducting comprehensive literature searches, and appraising and summarizing evidence using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) criteria,6 and faculty from the MUSC’s Department of Public Health Sciences delivered lectures on biostatistics.

Student evaluations, and the students’ completed projects (i.e., evidence summaries), helped the CEBP assess the effectiveness of the course content and the methods of delivery. However, we ultimately measured success by the translation of the student teams’ projects into guidelines.

Back to Top | Article Outline

Outcomes

Students provided feedback through an in-person feedback session for all students facilitated by the CEBP after the EBP projects were presented to the clinicians. Students noted that condensing the project and moving the overall/final deadline for it to earlier in the year would improve their ability to manage the workload. They also felt that eliminating long stretches of time between activities would sustain the momentum of the project, decrease repetition among lectures, and ensure that the project did not compete with board exam preparation. In addition, students requested that project deadlines fall in the beginning of each academic block and not during study or exam weeks. Furthermore, students noted that the time between when the interprofessional clinical teams first identified their questions of interest and developed the students’ evidence summaries into guidelines was more than a year, so a shorter timeline might keep clinicians more engaged in these projects as well.

Because the EBP projects were integrated within existing biostatistics lectures given by faculty in the Department of Public Health Sciences, the students noted both redundancies and contradictions in content. Going forward, the lectures will be shared between faculty members ahead of time to standardize the delivered content.

Each student team met with the librarian about their search strategy, and with the CEBP director or analyst about their GRADE evidence appraisal table. Although only one student from each team was required to attend these consultations, usually at least two students participated; the students found these meetings so valuable that they requested additional small-group consultations. Because there were 40 student teams, this was a huge time commitment for the CEBP, and so it is something to consider for future offerings of this course.

The student feedback suggested that the hands-on search session should occur earlier during the literature search and that additional librarians should be made available during this session to answer questions. Some students also expressed concerns about the real-world issues of applying a team-based approach to the project, including the (1) equitable division of work, (2) time required to complete the EBP projects, and (3) negative ramifications due to the extended timeline (i.e., forgetting the clinical question and project deadlines). However, these are all issues that come up in practice given the team-based nature of medical care, so clinicians must learn to overcome them.

As noted above in the student feedback, there were a number of limitations to the course. The information provided by students will be used to enhance the course for offerings in the coming academic years. In addition, students will be given a pre- and postcourse survey to assess their skills, attitudes, and beliefs regarding EBP, and any change in their self-reported utilization of EBP resources in future offerings of the course.

Interprofessional team clinicians provided feedback on the EBP projects via e-mail directly following the presentations and through an online survey once work on developing the projects into guidelines had begun. Data from these surveys were collected and managed using REDCap (Research Electronic Data Capture, Vanderbilt University, Nashville, Tennessee) tools hosted at the MUSC. The clinicians completing the survey (n = 11) thought that the literature searches were comprehensive and accurate (1.82/5.00, where 1.00 = strongly agree, 5.00 = strongly disagree); found the evidence summaries informative to their practice (2.09/5.00); believed they could use the summaries to support practice and/or develop new guidelines (2.18/5.00); and were satisfied enough with the course to participate in the future (1.82/5.00).

The feedback from interprofessional team clinicians on the EBP projects obtained through the survey was overwhelmingly positive. Clinicians appreciated the way the best-practice evidence was presented and felt that the students understood the value of the projects. One clinician described the EBP projects as a “great use of a huge resource, and an excellent opportunity for the students to learn this technique early in their education.” Other clinicians’ suggested improvements included meeting with the student teams prior to their presentations to further infuse clinical expertise into the projects’ clinical practice recommendations, and allowing time for additional conversations about the clinical applications of the evidence. Many clinicians were also very interested in making specific requests for the clinical topic areas of interest for the next year’s EBP projects.

Since the completion of the course, the CEBP has partnered with the interprofessional teams (as well as representatives from related health professions and specialties based on the topic area) to translate the evidence summaries from 20 of the student projects into 4 guidelines and 11 order sets. (The 4 developed guidelines span both emergency department and inpatient settings and address multiple components of management, resulting in the 11 order sets.) For an example evidence summary and guidelines, see Supplemental Digital Appendixes 1 and 2 at http://links.lww.com/ACADMED/A378 and http://links.lww.com/ACADMED/A379; see Figure 1 for an example of the guideline development process.

Figure 1

Figure 1

The expectation is that the remaining evidence summaries from the other 20 student projects will be translated into seven additional guidelines as time permits. As the CEBP and interprofessional teams continue to translate the student teams’ work into guidelines and order sets, the students are invited to participate, allowing them a chance to see the EBP process in action.

The guidelines and accompanying order sets developed to date are for sepsis, pediatric status epilepticus, pediatric sickle cell disease/acute chest syndrome, and Kawasaki disease. The guidelines and accompanying order sets resulting from the remaining student projects will cover processes for other diseases that reflect a high volume of admissions at the hospital, such as total hip and knee replacement and depression.

Back to Top | Article Outline

Next Steps

This course provides a real, concrete model for how to connect education to patient care via quality. It moves EBP education from artificial exercises to something of real use that can be applied in practice. While the development of evidence-based guidelines using this approach was very resource intensive for the CEBP, on the basis of the positive feedback from the interprofessional team clinicians, it is likely that this model will serve as a new framework for guideline development and will greatly expand the breadth of evidence-based content currently produced and available for clinicians at the MUSC Medical University Hospital.

The method used to select topics for guideline development in the course was effective. In future offerings of this course, the CEBP will continue to partner with the chief quality officer and service line leadership to identify additional topics that reflect either strategic organizational priorities or a large volume of hospital admissions for which no standard approach to care currently exists. This will prevent the possibility of clinical questions being duplicated and will ensure that content developed by the student teams is both relevant and useful to the clinical enterprise.

This course, offered to 181 medical students, was led by two staff members from the CEBP, a medical librarian, and one faculty member from the Department of Public Health Sciences. It would be feasible to offer a similar EBP course within the MUSC to other disciplines and colleges, or at other institutions, if there were support from administration, interest on the part of clinicians and medical faculty, and individuals with the required expertise available to develop the curriculum and facilitate the course. The CEBP is currently exploring the possibility of offering a similar course to nursing students and partnering students in the MUSC College of Nursing with practicing bedside nurses to develop and implement best practices in nursing care. Given the variability of academic medical centers, this model may not be highly generalizable but could be adapted by the right innovators.

In our experience, this innovative model effectively engaged medical students in a clinically relevant EBP project, while simultaneously resulting in the development of evidence-based clinical decision support tools for use in a clinical enterprise. Therefore, it is worth considering how to improve the model based on experiences and feedback, and evaluating opportunities to implement it within other settings.

Back to Top | Article Outline

References

1. Dawes M, Summerskill W, Glasziou P, et al.; Second International Conference of Evidence-Based Health Care Teachers and Developers. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1.
2. Till A, Banerjee J, McKimm J. Supporting the engagement of doctors in training in quality improvement and patient safety. Br J Hosp Med (Lond). 2015;76:166169.
3. Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004;329:1017.
4. Zee M, de Boer M, Jaarsma AD. Acquiring evidence-based medicine and research skills in the undergraduate medical curriculum: Three different didactical formats compared. Perspect Med Educ. 2014;3:357370.
5. Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ. 2015;15:39.
6. Guyatt GH, Oxman AD, Vist GE, et al.; GRADE Working Group. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924926.

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2016 by the Association of American Medical Colleges