Medical specialists, both practicing physicians and those in training, are expected to work according to the principles of evidence-based medicine (EBM). We follow the literature in defining evidence-based practice as decisions about health care that are based on information regarding the best available, current, valid, and relevant evidence; the patient's clinical and physical circumstances; and the patient's preferences and likely actions.1,2 These decisions should be made by those receiving care, informed by the tacit and explicit knowledge of those providing care, within the context of the available resources.1 Yet various studies showed that medical specialists find it difficult to use new evidence in their daily practice. Insufficient time is the most important barrier mentioned to implementing EBM.3–5 Other limiting factors are the recognition of questions, formulating search questions, performing literature searches, formulating answers to questions from the available literature, and translation of those answers to the individual patient.3,6 Even with the rising availability of aggregated evidence in clinical practice guidelines or evidence-based textbooks (e.g., Clinical Evidence, Pier, UpToDate), implementing new evidence into practice remains adifficult task.4 Changing a physician's existing ideas and habits can make implementation difficult as well.4
Residents may well have different barriers to practicing EBM than established physicians. Changing their own practice might be easier because their habits have not formed as strongly yet. However, despite the best evidence and the preferences of both the resident and the patient, the educational system and preferences of the clinical supervisors may also influence the practice of the resident.
In this study, therefore, we aimed to systematically review the literature on the barriers residents experience in the application of EBM in their daily practice.
Identification of studies
We performed a search for studies in MEDLINE (PubMed), EMBASE (Ovid), the Cochrane Library, CINAHL (EBSCOhost), and ERIC (EBSCOhost) from the earliest available date for each database until the end of January 2008. The search terms used are presented in List 1. There was no language restriction on the literature search. We screened the reference lists of retrieved studies for relevant publications. Additionally, online available abstract reports from the annual or biannual meetings of the Society of Medical Decision Making, Society of General Internal Medicine, Association for Medical Education in Europe, Evidence-Based Health Care Teachers & Developers, and Ottawa conferences between January 2001 and January 2008 were manually searched for relevant abstracts. To complete our search for relevant studies, we contacted experts in the field by e-mail, asking them if they were aware of any additional studies for our review.
We used the following criteria to screen studies for inclusion in our review:
- Study design: Original studies, both qualitative and quantitative (observational and experimental), focusing on the barriers that residents experience in applying EBM in their daily practice.
- Population: Residents or those working on a similar level (MDs at postgraduate, but premedical specialist level, working under the supervision of a medical specialist).
- Outcomes: Pertaining to barriers toward the practice of EBM, prevalence of different barriers toward the practice of EBM, or perceived/measured importance of factors mentioned to be barriers toward the practice of EBM.
Studies not describing residents, not reporting on original studies, or not related to the practice of EBM were excluded.
Two of us (N.v.D. and M.W.d.W.) independently reviewed the titles and abstracts of the retrieved studies for potentially eligible studies (Figure 1 describes this process). For each potentially eligible study, the full paper was read by two reviewers (N.v.D. and M.W.d.W.) independently. They both assessed whether the study fulfilled the inclusion criteria. Studies about which both reviewers had doubt about inclusion, and studies about which the reviewers disagreed about inclusion, were discussed in a consensus meeting. In any case of persisting disagreement, the third author (L.H.) was consulted and a decision was made by consensus of all authors.
Assessment of study quality
Included studies were assessed for methodological quality independently by two reviewers (N.v.D. and M.W.d.W.). We assessed the quality of quantitative studies using a modified version of the recommendations described by the Strengthening the Reporting of Observational Studies in Epidemiology initiative7,8 (List 2). Quality assessment of qualitative studies was performed using the modified criteria proposed by Giacomini and Cook9 (List 2). Finally, we assessed the quality of randomized controlled trials (RCTs) according to the methods recommended by the Cochrane Collaboration.10 In studies in which it was unclear whether the quality criteria had been met, the reviewers contacted the first author of the publication to obtain additional information.
The results of the included studies were extracted by two reviewers (M.W.d.W. and N.v.D.). A prespecified data extraction form was used to extract and collect information from the included studies on (1) study design, (2) characteristics of residents (specialty, gender, age, number of responders), and (3) outcomes as published by authors. We resolved disagreements about data extraction using a consensus meeting with all reviewers. Because we expected to find no homogeneous studies in this area, no pooling of data was attempted.
Search results and selection of studies for inclusion
After duplicate studies were removed, the searches for articles resulted in 511 titles. From these, 37 were considered potentially eligible for inclusion in the study on the basis of title and abstract. We obtained no additional studies by screening the references of the retrieved publications. Of the 37 potential studies, 8 were included. The other 29 were excluded because they did not report about residents (n = 5), barriers (n = 21), or both (n = 3). The searches of the meeting abstracts resulted in 84 potentially relevant titles, of which 4 fulfilled the inclusion criteria. Three of the included abstracts11–13 were already identified and included as full papers,14–16 resulting in 1 new included study.17 None of the 3 studies suggested by the experts fulfilled all inclusion criteria. The final review therefore included 9 studies (8 full articles and 1 abstract) evaluating the attitudes of residents toward EBM and the barriers toward practicing EBM.14–21 Of these studies, 5 were identified through MEDLINE,15,19–22 and the others were identified through MEDLINE and CINAHL,14 MEDLINE and EMBASE,18 the Cochrane Library,16 and the abstracts of scientific meetings.17 Agreement between both reviewers on the inclusion of studies was 92% after individual review of the papers and 100% after discussion. Agreement on data abstraction was 100%. See Figure 1 for a graphic representation of the selection process.
Quality of included studies
Our criteria for the quality of the individual studies are shown in Table 1. Most studies we found were of high quality. The qualitative studies14,15,18 scored high on all criteria, such as proper selection of study participants, appropriate methods, and comprehensive data collection. The quantitative studies varied in quality, with some unclear aspects in the study (abstract only) of Ho et al,17 but none had such poor quality as to be excluded from the study.
Of the nine included studies, four described the results of quantitative surveys,20–22 one described the results of quantitative interviews,19 one described the results of an RCT,16 and three described the results of qualitative studies. Of these, one used focus groups,15 and two used a combination of individual interviews and focus groups.14,18 A more detailed description of the nine studies is displayed in Table 2. Most studies were performed in countries where English is the primary language (four in the United States, three in Canada, one in Ireland, and one in Chile). Most studies included between 12 and 97 residents from various medical specialties as subjects. Only three studies14,19,22 describe whether the residents had received any training in EBM before participating in the study. Green and colleagues19 report that 52% of the residents had participated in an EBM curriculum for one or more years; the neurology residents assessed by Burneo et al22 participated in biweekly, 90-minute evidence-based clinical practice sessions, and the internal medicine residents from the Mayo Clinic study participated in a clinical decision-making conference.14 The other studies did not report on the presence or absence of EBM training of the residents.15–18,20
Barriers to implementing all steps of the evidence-based decision-making process were reported by various studies. The main barriers reported were time, attitude, knowledge and skills, and resident-specific barriers.
The most often mentioned and primary barrier for residents was limited available time. In all qualitative studies,14,15,18 time was spontaneously mentioned as a barrier. The neurology residents responding to the survey of Burneo and colleagues22 all responded that time constraints were the main reason why they do not use EBM all the time. Three studies17,20,22 showed that 28% to 85% of the residents report time as an important limiting factor for using EBM concepts. In contrast, Allan et al21 found that time was not a substantial barrier. Other studies15,18 suggest that a lack of priority and competing responsibilities, due to a lack of allocated time, constitute a barrier.
The two central themes in the study of Montori et al14 were time and expertise. These two themes influenced the decision-making style of the residents and the information sources used. The preferred style of the residents was to discuss the evidence with the patients and incorporate patient preferences into the decision-making process. However, when time was limited or residents felt insecure about their expertise, they were more likely to consult an expert as an information source instead. In the first study by Green et al,19 residents reported identifying two new questions for every three patients. Questions were not answered in 81% of the cases. Lack of time (60%) and forgetting the question (29%) were the most frequently mentioned reasons for not searching for the answer.
Both Bhandari et al18 and Green and Ruff15 describe factors such as personal initiative, lack of motivation, and interest as barriers toward residents' use of EBM. In other studies, the attitude of residents toward EBM is described as moderately positive (53% of residents)20 or positive (70% of residents).21 However, Green and colleagues19 mention lack of interest in only a minority of the cases (4%) as the reason for not researching a clinical question using the principles of EBM.
Knowledge and skills.
In the qualitative study by Green and Ruff15 residents reported experiencing barriers on all steps involved in the EBM process, such as forgetting clinical questions, low searching skills, and difficulty knowing when to stop searching. Also reported by Bhandari et al,18 a lack of education and a lack of critical appraisal skills were considered an important barrier. Ho and colleagues17 found that only 20% of the residents noted that the practice of EBM was easy, although it was never reported as being very easy. Allan et al21 described lack of basic computer skills as a barrier, especially for immigrant residents (compared with U.S. and Canadian residents).
In four studies, specific barriers related to the position of residents were mentioned. Green and Ruff15 found that the learning climate influenced the residents' motivation for practicing EBM. Bhandari et al,18 for surgical residents, identified staff-surgeons-based barriers, institutional and health care system factors, and resident barriers (knowledge) influencing the integration of EBM in day-to-day routines. The main staff-surgeons-based barrier was staff disapproval.18 Several residents reported fear of repercussions from staff members when confronting them with new evidence, thereby indirectly telling them their current practices are outdated. Other staff factors were a lack of interest and low motivation for implementing EBM.18 However, Burneo et al22 reported that only 1 of 12 residents reported the disinterest of some attending physicians as a reason for using EBM concepts only sometimes.
Resident shortage, lack of funding of health care, and inadequate information resources were mentioned as examples of institutional and health care system factors.18 This may make institutional factors or learning environment an important barrier.15 Unfortunately, this was not examined by most studies. On the other hand, Green and colleagues19 found the perception of inadequate resources to be the reason for not pursuing clinical questions in only 2% of the cases. In a later study, Green and Ruff15 describe access to electronic information resources and available information technology as varying widely between hospitals.
Only the findings of Letelier et al16 focused on language as a barrier toward the practice of EBM. That study found that Chilean residents reading a paper in English used longer time (12.6 versus 11.8 minutes) and had more difficulty interpreting (low score 34.7% versus 16.7%) a Cochrane abstract than residents reading in Spanish.
Three of the studies suggested potential solutions to overcome important perceived barriers. Bhandari et al18 provide a long list of strategies to improve the practice of EBM, such as EBM training, preappraisal of resources, and journal clubs. Residents studied by Green and Ruff15 suggested the use of handheld devices to overcome barriers in access problems and time and specially designed Web sites assisting residents in their searches. Letelier et al16 suggest formal training in English language and translation of papers as solutions to overcome language barriers.
Although many studies report on the barriers of medical specialists toward the application of EBM, only a few studies report on the specific barriers that residents experience. Residents encounter similar problems to practicing physicians regarding time and searching skills, but they also encounter the additional and potentially negative influences of clinical supervisors, the lack of role models, and practical and institutional barriers, which make the use of EBM even more difficult. This in combination with less experience in EBM and the high workload of residents23 could reduce their motivation to practice EBM to a minimum. Because residents could be important motivators for the propagation of EBM, educators should be made aware that EBM is now considered a principal component of good clinical practice,1 and resident-specific barriers should be identified and integrated into their EBM-training programs.
The few studies we found that focused on the barriers residents experience to applying EBM in daily practice were of high quality and recent date. Although the overall quality of the studies is high, most quantitative studies did not fulfill all subcriteria regarding the definition of the outcomes. Most studies focused on one or two requirements (i.e., knowledge) for EBM and not on the EBM process until the application of the results to the patient. To obtain an overall view on the process of implementing EBM, a measurement of the barriers in all five steps of EBM would be required.
Another limitation of this review is the location in which the studies took place. All but two studies are of North American origin. Differences in educational and health care systems between North America and other regions might limit the generalizability of the results.
All studies included in this review are single-center studies. The culture (EBM and attending supervision in general) might vary significantly per center and medical specialty. The resources for, quality, and amount of available evidence differ significantly per medical specialty, and new evidence might be difficult to implement. Learning a new surgical technique requires more efforts than prescribing another type of medication. Also, the influence of supervisors might differ per medical specialty. Some surgeons dismiss clinical studies on the basis of their perceived quality of the study surgeons (judging that they have poor skills and that, therefore, the results are poor).18 Residents can retrieve new evidence (techniques), but if they are not allowed to practice it on patients by their supervisors, this new evidence will less likely be applied in later practice.18 Also, barriers could differ per resident. In a study by Doran et al24 of preregistration house officers, 41% of the residents agreed with the statement that educational supervisors support critical appraisal skills in everyday practice, but this was significantly less so for female than for male house officers. Differences between groups of residents have not been studied thus far.
It is unclear from these studies what definition of EBM the participants had in mind. Most of the included studies seem limited to the search and use of original studies and do not cover other aspects of EBM. A broader definition of what research evidence is (not only original studies, but also and more easily retrieved in synopses, summaries, and systems25) and the practice of integrating research evidence with the circumstances of the patient and patient preferences by using clinical expertise2 might lead to a shift in perceived barriers.
Implications for medical education
The main resident-specific barriers toward implementing EBM were a lack of interest by the staff or even staff disapproval of EBM. A positive learning environment, with staff members as EBM role models, might therefore be one of the most important factors influencing the behavior of residents.
Besides reserving specific time for teaching about EBM in the training programs of residents, another step in reducing the time-related barrier could be to provide a clear definition and corresponding expectations of the practice of EBM.26 Finding and appraising all relevant original papers on all clinical questions that rise during a consultation is of course not useful or attainable for residents.26 Studies assessing the amount of time needed to find all relevant original studies indicate that even skilled searchers like clinical librarians need 45 minutes.27 One important solution is the use of preappraised papers or evidence-based synopses, summaries (systematic reviews and evidence-based clinical practice guidelines), or evidence-based decision support systems when available.25,28 These resources need to be approached critically as well,29 but using them can save a lot of time without reducing the quality of the retrieved evidence. Also, evidence preappraised by the resident's own institution and stored in CAT databases or computer-based clinical support systems25,30,31 could save time while still encouraging the practice of EBM. A study by Sackett and Straus28 reported that an EBM cart, consisting of secondary sources from their own department, textbooks, and computerized (summary) sources such the Cochrane Library and MEDLINE, was used during rounds. They found that the presence of the cart raised the number of searches for evidence and that 90% of the searches were completed in the time available on rounds (90 seconds). Also, multiple other interventions are available and have been tested. Various Web-based tools,32,33 EBM curricula,34 teach-the-teacher programs,35 active involvement of clinical librarians,36 EBM resident rotations,37 and many other educational interventions have been shown to be effective in reducing specific barriers to practicing EBM. Only clinically integrated teaching, however, has been reported to improve not only knowledge but also skills, attitudes, and behavior toward EBM.38
Implications for research
To create a complete overview of the barriers of residents in the application of EBM in daily practice, studies on the barriers experienced when combining the evidence with the patients' clinical and personal circumstances and preferences are required. In these studies, a clear definition of EBM should be used, including these patient-factors and all the evidence-based information sources that can be applied in clinical practice.25 Measuring the implementation of EBM as a decision process, however, remains difficult.
The next step in overcoming barriers is to quantify the effects of the various barriers on residents' application of EBM. Currently, the relative importance of each barrier in the process of EBM is unknown. Although time is mentioned often as an important barrier, the willingness to practice EBM and the recognition of answerable clinical questions should be present before time is necessary to find the answers.39 Barriers that are located earlier in the process of EBM could therefore be more important, and should be overcome, before interventions reducing barriers, such as lack of time or computer facilities, are most useful. Additionally, the importance of different barriers for specific (groups of) residents should be studied. Methods for identifying the barriers for individual residents are required to identify their specific learning needs. These needs can then be used as a basis for optimizing the learning process of the residents and creating a learning environment in which residents are stimulated to overcome their own barriers.
Besides the more general barriers for the practice of EBM that practicing physicians are subject to, residents experience the additional and potentially negative influences of clinical supervisors, the lack of role models, and practical and institutional barriers. These barriers should be dealt with when attempting to integrate evidence-based practice into the training programs of residents.
Department of General Practice, Academic Medical Center, University of Amsterdam; Dutch Cochrane Centre.
1 Dawes M, Summerskill W, Glasziou P, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1.
2 Haynes RB, Devereaux PJ, Guyatt GH. Physicians' and patients' choices in evidence based practice. BMJ. 2002;324:1350.
3 Ely JW, Osheroff JA, Ebell MH, et al. Obstacles to answering doctors' questions about patient care with evidence: Qualitative study. BMJ. 2002;324:710.
4 Grol R, Grimshaw J. From best evidence to best practice: Effective implementation of change in patients' care. Lancet. 2003;362:1225–1230.
5 Freeman AC, Sweeney K. Why general practitioners do not implement evidence: Qualitative study. BMJ. 2001;323:1100–1102.
6 Schaafsma F, Hulshof C, van Dijk F, Verbeek J. Information demands of occupational health physicians and their attitude towards evidence-based medicine. Scand J Work Environ Health. 2004;30:327–330.
7 Von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Epidemiology. 2007;18:800–804.
8 Vandenbroucke JP, von Elm E, Altman DG, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. Epidemiology. 2007;18:805–835.
9 Giacomini MK, Cook DJ. Users' guides to the medical literature: XXIII. Qualitative research in health care A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 2000;284:357–362.
10 The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions. Available at: http://www.cochrane-handbook.org
. Accessed January 6, 2010.
11 Montori VM, Tabini CC, Ebbert JO. The clinical decision-making conference: A qualitative assessment of learners' experience. J Gen Intern Med. 2000;15(suppl):40.
12 Ruff TR, Green ML. Why residents fail to answer their clinical questions: A qualitative study. J Gen Intern Med. 2004;19(suppl):240.
13 Letelier LM, Zamarin N, Andrade M, et al. Exploring language barriers to evidence based medicine: Is it different to read a Cochrane review abstract in English or in Spanish for Chilean post graduate medical students? Paper presented at: International Conference of Evidence-Based Health Care Teachers & Developers; Taormina, Italy; Nov 2–6, 2005.
14 Montori VM, Tabini CC, Ebbert JO. A qualitative assessment of 1st-year internal medicine residents' perceptions of evidence-based clinical decision making. Teach Learn Med. 2002;14:114–118.
15 Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med. 2005;80:176–182.
16 Letelier LM, Zamarin N, Andrade M, et al. Exploring language barriers to Evidence-based Health Care (EBHC) in post-graduate medical students: A randomised trial. Educ Health (Abingdon). 2007;20:82.
17 Ho P, Kim P, Hong JJ, Fontelo P. Virtual evidence cart and resident use of EBM. J Gen Intern Med. 2006;21(suppl 4):161.
18 Bhandari M, Montori V, Devereaux PJ, Dosanjh S, Sprague S, Guyatt GH. Challenges to the practice of evidence-based medicine during residents' surgical training: A qualitative study using grounded theory. Acad Med. 2003;78:1183–1190.
19 Green ML, Ciampi MA, Ellis PJ. Residents' medical information needs in clinic: Are they being met? Am J Med. 2000;109:218–223.
20 Amin M, Saunders JA, Fenton JE. Pilot study of the knowledge and attitude towards evidence based medicine of otolaryngology higher surgical trainees. Clin Otolaryngol. 2007;32:133–135.
21 Allan GM, Manca D, Szafran O, Korownyk C. EBM: A challenge for international medical graduates. Fam Med. 2007;39:160.
22 Burneo JG, Jenkins ME, Bussiere M. Evaluating a formal evidence-based clinical practice curriculum in a neurology residency program. J Neurol Sci. 2006;250:10–19.
23 Fletcher KE, Underwood W III, Davis SQ, Mangrulkar RS, McMahon LF Jr, Saint S. Effects of work hour reduction on residents' lives: A systematic review. JAMA. 2005;294:1088–1100.
24 Doran T, Maudsley G, Zakhour H. Time to think? Questionnaire survey of pre-registration house officers' experiences of critical appraisal in the Mersey Deanery. Med Educ. 2007;41:487–494.
25 Haynes RB. Of studies, syntheses, synopses, summaries, and systems: The “5S” evolution of information services for evidence-based healthcare decisions. Evid Based Med. 2006;11:162–164.
26 Akl EA, Maroun N, Neagoe G, Guyatt G, Schunemann HJ. EBM user and practitioner models for graduate medical education: What do residents prefer? Med Teach. 2006;28:192–194.
27 Gorman PN, Ash J, Wykoff L. Can primary care physicians' questions be answered using the medical journal literature? Bull Med Libr Assoc. 1994;82:140–146.
28 Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: The “evidence cart.” JAMA. 1998;280:1336–1338.
29 AGREE collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: The AGREE project. Qual Saf Health Care. 2003;12:18–23.
30 Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: Bootstrapping multiple methods. J Biomed Inform. 2005;38:189–199.
31 Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: A systematic review. JAMA. 1998;280:1339–1346.
32 Belda TE, Gajic O, Rabatin JT, Harrison BA. Practice variability in management of acute respiratory distress syndrome: Bringing evidence and clinician education to the bedside using a Web-based teaching tool. Respir Care. 2004;49:1015–1021.
33 Bell DS, Fonarow GC, Hays RD, Mangione CM. Self-study from Web-based and printed guideline materials. A randomized, controlled trial among resident physicians. Ann Intern Med. 2000;132:938–946.
34 Green ML, Ellis PJ. Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med. 1997;12:742–750.
35 Cartwright CA, Korsen N, Urbach LE. Teaching the teachers: Helping faculty in a family practice residency improve their informatics skills. Acad Med. 2002;77:385–391.
36 Earl MF, Neutens JA. Evidence-based medicine training for residents and students at a teaching hospital: The library's role in turning evidence into action. Bull Med Libr Assoc. 1999;87:211–214.
37 Akl EA, Izuchukwu IS, El-Dika S, Fritsche L, Kunz R, Schunemann HJ. Integrating an evidence-based medicine rotation into an internal medicine residency program. Acad Med. 2004;79:897–904.
38 Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004;329:1017.
39 Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–1465.