Many of the same implementation issues emerged across all curricula irrespective of learner level (i.e., undergraduate or postgraduate). However, some factors were more commonly cited as important factors only for curricula targeting residents (e.g., time pressures and the need for ongoing financial, educational, institutional, and operational support), perhaps because of the greater inclusion of QI or PS projects in curricula for residents. A barrier unique to curricula that targeted medical students in the preclinical years was their perception of the unimportance of the material compared with traditional clinical content.
We identified 41 QI and PS curricula that specifically targeted medical students or residents. Concepts of continuous QI, systems thinking, and root cause analysis constituted the most common topics covered, and specific projects undertaken often involved chart audits. Despite the heterogeneity in educational content and teaching methods, most curricula were well accepted and led to learners' knowledge acquisition. Resident involvement in experiential QI projects such as chart audits also frequently led to significant improvements in processes of care.
Few studies demonstrated changes in learners' behavior or potential patient benefits. Although some reports suggest that educational interventions have the potential to change behavior or improve health outcomes, most studies lack good-quality evidence to support their findings.60 There are examples of well-designed continuing medical education interventions that are sequenced and make use of interactive techniques that lead to changes in learners' behaviors and health outcomes. However, those studies often centered on screening, smoking cessation, and communication skills and may not translate to more complex curricular content areas, such as QI and PS.61–63 In fact, for QI and PS, improving patient outcomes as a result of educational efforts represents a particularly daunting task, given that intensive, large-scale QI efforts often fail to demonstrate improvements in health outcomes.64,65 Also for some tools of QI and PS, including ones that commonly appeared in the curricula we reviewed (e.g., root cause analysis), little empiric evidence guides recommendations on how to design or use these tools.65 Consequently, even with optimal delivery of the target educational content, the degree to which organizational or patient outcomes might improve remains unclear.
Our results complement those of a systematic review of educational efforts in QI for clinicians in general13 in that well-established adult learning techniques (e.g., experiential learning) were identified as key factors for success in delivering curricula in QI and PS. However, our review, which included 34 newer reports of curricula specifically targeting trainees, demonstrated that residents' involvement in QI and PS curricula can lead to meaningful improvements in clinical processes, a novel finding compared with those of the previous review.
Our review also identified important barriers and facilitators to implementation that are likely unique to curricula in the undergraduate and postgraduate settings. Many of the studies identified barriers commonly encountered with new curricular initiatives in general.17 For example, most of the curricula relied on small numbers of faculty members with a personal interest in QI or PS to teach the curriculum, often resulting in burdensome time commitments. Many reports highlighted the need for greater faculty development to achieve sufficient numbers of teachers of QI and PS topics for both medical student and resident curricula. Some curricula addressed these issues by developing teaching materials that circumvented the need to have faculty experienced in QI or PS.36
Competing educational demands and achievement of learner buy-in also represented major issues for curricula at all levels. However, the only two reports24,35 that noted these as potentially insurmountable obstacles were ones that targeted medical students at preclinical stages. Learners reported significant dissatisfaction with key elements of the curricula, which suggests that clinical experience represents a prerequisite for appreciating the importance and relevance of QI or PS concepts.
Curricula that targeted residents may require special consideration, perhaps because such curricula more commonly involved the learners in experiential projects, adding to time pressures and increasing the need for supporting infrastructure. Many residents did not complete their projects because of time constraints. Some programs addressed this problem by scheduling their curricula during less busy clinical rotations or research years.19 Having adequate personnel, financial, and technological resources to support curricula involving experiential projects was also important. For example, studies that made use of chart audits required administrative support to retrieve charts. Also, many QI projects depended on efficient availability of clinical data through information systems to determine whether improvements occurred.
Finally, a number of studies emphasized the importance of a local “safety culture,” substantially enhancing the curricular success when present and undermining it when absent. Other curricula that target nonmedical competencies (e.g., professionalism) also highlight the importance of the so-called hidden curriculum, where there is a discrepancy between the concepts trainees learn in formal educational venues and what trainees observe when supervised by attending staff in routine clinical practice.66–68 Preparing trainees for the fact that behavior of faculty in routine practice, design of the delivery systems in which they work, and institutional culture may not conform to accepted principles of QI and PS may reduce the discomfort reported by participants in some of the curricula we reviewed.
Our systematic review had several limitations. The literature examining the effectiveness of educational interventions in QI and PS exhibited substantial heterogeneity in terms of the content delivered, educational methods used, learners targeted, and learning outcomes reported. Also, many curricular evaluations involved weak study designs, occurred in single centers, had small numbers of learners, and often exhibited other methodological concerns. Consequently, we did not regard quantitative synthesis as appropriate.
Our thematic textual analysis of all of the included curricular reports identified a number of potentially important factors that promote or hinder implementation efforts. However, most of those reports did not have the identification of facilitators and barriers to implementation as their primary aim. Consequently, authors may not have recognized or reported aspects of the curricular implementation in a systematic fashion. Moreover, the vast majority of reports did not comment on the degree to which curricula had been sustained.
Improving the quality and safety of patient care has gained widespread acceptance as a central activity for the health care system. Clinicians will be expected to have acquired core concepts in QI and PS in order to apply them to improve their personal practices and help support institutional improvement efforts. Consequently, a consensus has emerged that QI and PS should be broadly taught to trainees, with ACGME9 and CanMEDS10 mandating such education and some students actively requesting it.69 Despite this emerging consensus, in 2006, few medical schools in the United States and Canada reported having explicit curricula in QI and PS,70 although it is likely that now (2010) there are more schools with such curricula.
The existing literature indicates that educational curricula focused on QI and PS are generally well accepted by trainees, effectively improve knowledge in these domains, and can even lead to important improvements in processes of care. Programs undertaking the development of curricula in QI or PS must recognize the significant time pressures and competing educational demands for trainees, as well as the requirements for adequate numbers of faculty with appropriate expertise and support for their contributions. To succeed, these curricula require engagement of educational and organizational stakeholders to promote adoption. Future research must better characterize the learner, faculty, and institutional factors that facilitate or hinder the promotion of sustained educational efforts focused on QI and PS for medical students and postgraduate trainees.
Dr. Wong received an honorarium from the Association of the Faculties of Medicine of Canada to write an earlier version of this review that appeared as a chapter in a monograph entitled “The Future of Medical Education in Canada.” Dr. Shojania receives general salary support from the Government of Canada Research Chairs Program. Neither funding body played any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the systematic review.
The opinions expressed in this article are those of the authors alone and do not reflect the views of the Association of the Faculties of Medicine of Canada.
A preliminary version of the review appears as a chapter in a monograph entitled “The Future of Medical Education in Canada” (http://www.afmc.ca/fmec/activities-env-literature.php), published by the Association of Faculties of Medicine of Canada.
1Fletcher KE, Underwood W 3rd, Davis SQ, Mangrulkar RS, McMahon LFJ, Saint S. Effects of work hour reduction on residents' lives: A systematic review. JAMA. 2005;294:1088–1100.
2Ogden PE, Sibbitt S, Howell M, et al. Complying with ACGME resident duty hours restrictions: Restructuring the 80-hour workweek to enhance education and patient safety at Texas A&M/Scott & White Memorial Hospital. Acad Med. 2006;81:1026–1031.
3Okie S. An elusive balance—Residents' work hours and the continuity of care. N Engl J Med. 2007;356:2665–2667.
4Shojania KG, Fletcher KE, Saint S. Graduate medical education and patient safety: A busy—and occasionally hazardous—intersection. Ann Intern Med. 2006;145:592–598.
5Kennedy TJ, Regehr G, Baker GR, Lingard L. Preserving professional credibility: Grounded theory study of medical trainees' requests for clinical support. BMJ. 2009;338:b128.
6Vidyarthi AR, Arora V, Schnipper JL, Wall SD, Wachter RM. Managing discontinuity in academic medical centers: Strategies for a safe and effective resident sign-out. J Hosp Med. 2006;1:257–266.
7Association of American Medical Colleges. Report X: Contemporary Issues in Medicine: Education in Safe and Effective Prescribing Practices. Medical School Objectives Project. Available at: http://www.aamc.org/meded/msop
. Accessed March 24, 2010.
8Association of American Medical Colleges. Report V: Contemporary Issues in Medicine: Quality of Care. Medical School Objectives Project. Available at: http://www.aamc.org/meded/msop
. Accessed March 24, 2010.
10Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29:642–647.
11Scheele F, Teunissen P, Van Luijk S, et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach. 2008;30:248–253.
12Shojania KG, Levinson W. Clinicians in quality improvement: A new career pathway in academic medicine. JAMA. 2009;301:766–768.
13Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: A systematic review. JAMA. 2007;298:1023–1037.
14Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.
16Kirkpatrick DL. Evaluation of training. In: Craig RL, Bittel LR, eds. Training and Development Handbook. New York, NY: McGraw-Hill; 1967:87–112.
17Hughes MT. Implementation. In: Kern DE, Thomas PA, Howard DM, Bass EB, eds. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore, Md: The Johns Hopkins University Press; 1998.
18Bechtold ML, Scott S, Nelson K, Cox KR, Dellsperger KC, Hall LW. Educational quality improvement report: Outcomes from a revised morbidity and mortality format that emphasised patient safety. Qual Saf Health Care. 2007;16:422–427.
19Canal DF, Torbeck L, Djuricich AM. Practice-based learning and improvement: A curriculum in continuous quality improvement for surgery residents. Arch Surg. 2007;142:479–482.
20Coleman MT, Nasraty S, Ostapchuk M, Wheeler S, Looney S, Rhodes S. Introducing practice-based learning and improvement ACGME core competencies into a family medicine residency curriculum. Jt Comm J Qual Saf. 2003;29:238–247.
21Coyle YM, Mercer SQ, Murphy-Cullen CL, Schneider GW, Hynan LS. Effectiveness of a graduate medical education program for improving medical event reporting attitude and behavior. Qual Saf Health Care. 2005;14:383–388.
22Djuricich AM, Ciccarelli M, Swigonski NL. A continuous quality improvement curriculum for residents: Addressing core competency, improving systems. Acad Med. 2004;79(10 suppl):S65–S67.
23Frey K, Edwards F, Altman K, Spahr N, Gorman RS. The ‘Collaborative Care’ curriculum: An educational model addressing key ACGME core competencies in primary care residency training. Med Educ. 2003;37:786–789.
24Gould BE, Grey MR, Huntington CG, et al. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Acad Med. 2002;77:1011–1018.
25Gunderson A, Tekian A, Mayer D. Teaching interprofessional health science students medical error disclosure. Med Educ. 2008;42:531–532.
26Halbach JL, Sullivan LL. Teaching medical students about medical errors and patient safety: Evaluation of a required curriculum. Acad Med. 2005;80:600–606.
27Henley E. A quality improvement curriculum for medical students. Jt Comm J Qual Improv. 2002;28:42–48.
28Holmboe ES, Prince L, Green M. Teaching and improving quality of care in a primary care internal medicine residency clinic. Acad Med. 2005;80:571–577.
29Kerfoot BP, Conlin PR, Travison T, McMahon GT. Web-based education in systems-based practice: A randomized trial. Arch Intern Med. 2007;167:361–366.
30Madigosky WS, Headrick LA, Nelson K, Cox KR, Anderson T. Changing and sustaining medical students' knowledge, skills, and attitudes about patient safety and medical fallibility. Acad Med. 2006;81:94–101.
31Mohr JJ, Randolph GD, Laughon MM, Schaff E. Integrating improvement competencies into residency education: A pilot project from a pediatric continuity clinic. Ambul Pediatr. 2003;3:131–136.
32Moskowitz E, Veloski JJ, Fields SK, Nash DB. Development and evaluation of a 1-day interclerkship program for medical students on medical errors and patient safety. Am J Med Qual. 2007;22:13–17.
33Newell P, Harris S, Aufses AJ, Ellozy S. Student perceptions of medical errors: Incorporating an explicit professionalism curriculum in the third-year surgery clerkship. J Surg Educ. 2008;65:117–119.
34Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med. 2004;19:496–500.
35Ogrinc G, West A, Eliassen MS, Liuw S, Schiffman J, Cochran N. Integrating practice-based learning and improvement into medical student learning: Evaluating complex curricular innovations. Teach Learn Med. 2007;19:221–229.
36Oyler J, Vinci L, Arora V, Johnson J. Teaching internal medicine residents quality improvement techniques using the ABIM's practice improvement modules. J Gen Intern Med. 2008;23:927–930.
37Patey R, Flin R, Cuthbertson BH, et al. Patient safety: Helping medical students understand error in healthcare. Qual Saf Health Care. 2007;16:256–259.
38Peters AS, Kimura J, Ladden MD, March E, Moore GT. A self-instructional model to teach systems-based practice and practice-based learning and improvement. J Gen Intern Med. 2008;23:931–936.
39Tomolo A, Caron A, Perz ML, Fultz T, Aron DC. The outcomes card. Development of a systems-based practice educational tool. J Gen Intern Med. 2005;20:769–771.
40Varkey P, Karlapudi SP, Bennet KE. Teaching quality improvement: A collaboration project between medicine and engineering. Am J Med Qual. 2008;23:296–301.
41Varkey P, Reller MK, Smith A, Ponto J, Osborn M. An experiential interdisciplinary quality improvement education initiative. Am J Med Qual. 2006;21:317–322.
42Voss JD, May NB, Schorling JB, et al. Changing conversations: Teaching safety and quality in residency training. Acad Med. 2008;83:1080–1087.
43Weingart SN, Tess A, Driver J, Aronson MD, Sands K. Creating a quality improvement elective for medical house officers. J Gen Intern Med. 2004;19:861–867.
44Ziegelstein RC, Fiebach NH. “The mirror” and “the village”: A new method for teaching practice-based learning and improvement and systems-based practice. Acad Med. 2004;79:83–88.
45Cosby KS, Croskerry P. Patient safety: A curriculum for teaching patient safety in emergency medicine. Acad Emerg Med. 2003;10:69–78.
46Esselman PC, Dillman-Long J. Morbidity and management conference: An approach to quality improvement in brain injury rehabilitation. J Head Trauma Rehabil. 2002;17:257–262.
47Farquhar D, Myers K, Benjamin D. Education in quality of care in an internal medicine residency program. Acad Med. 2001;76:562.
48Gosbee J. A patient safety curriculum for residents and students: The VA healthcare system's pilot project. ACGME Bulletin. November 2002:2–6.
49Gould BE, O'Connell MT, Russell MT, Pipas CF, McCurdy FA. Teaching quality measurement and improvement, cost-effectiveness, and patient satisfaction in undergraduate medical education: The UME-21 experience. Fam Med. 2004;36(suppl):S57–S62.
50Krajewski K, Siewert B, Yam S, Kressel HY, Kruskal JB. A quality assurance elective for radiology residents. Acad Radiol. 2007;14:239–245.
51Paulman P, Medder J. Teaching the quality improvement process to junior medical students: The Nebraska experience. Fam Med. 2002;34:421–422.
52Rosenfeld JC. Using the Morbidity and Mortality conference to teach and assess the ACGME General Competencies. Curr Surg. 2005;62:664–669.
53Schillinger D, Wheeler M, Fernandez A. The populations and quality improvement seminar for medical residents. Acad Med. 2000;75:562–563.
54Singh R, Naughton B, Taylor JS, et al. A comprehensive collaborative patient safety residency curriculum to address the ACGME core competencies. Med Educ. 2005;39:1195–1204.
55Thompson DA, Cowan J, Holzmueller C, Wu AW, Bass E, Pronovost P. Planning and implementing a systems-based patient safety curriculum in medical education. Am J Med Qual. 2008;23:271–278.
56Varkey P. Educating to improve patient care: Integrating quality improvement into a medical school curriculum. Am J Med Qual. 2007;22:112–116.
57Weeks WB, Robinson JL, Brooks WB, Batalden PB. Using early clinical experiences to integrate quality-improvement learning into medical education. Acad Med. 2000;75:81–84.
58Wong RY, Hollohan K, Roberts M, Hatala R, Ma IW, Kassen BO. A descriptive report of an innovative curriculum to teach quality improvement competencies to internal medicine residents. Can J Gen Intern Med. 2008;3:26–29.
59Morrison LJ, Headrick LA, Ogrinc G, Foster T. The quality improvement knowledge application tool: An instrument to assess knowledge application in practice-based learning and improvement. J Gen Intern Med. 2003;18(suppl 1):250.
61Belfield C, Thomas H, Bullock A, Eynon R, Wall D. Measuring effectiveness for best evidence medical education: A discussion. Med Teach. 2001;23:164–170.
62Davis D. Does CME work? An analysis of the effect of educational activities on physician performance or health care outcomes. Int J Psychiatry Med. 1998;28:21–39.
63Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874.
64Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: Systematic review. BMJ. 2008;336:1491–1494.
65Wu AW, Lipshutz AK, Pronovost PJ. Effectiveness and efficiency of root cause analysis in medicine. JAMA. 2008;299:685–687.
66Brainard AH, Brislen HC. Viewpoint: Learning professionalism: A view from the trenches. Acad Med. 2007;82:1010–1014.
67D'Eon M, Lear N, Turner M, Jones C. Perils of the hidden curriculum revisited. Med Teach. 2007;29:295–296.
68Gofton W, Regehr G. What we don't know we are teaching: Unveiling the hidden curriculum. Clin Orthop Relat Res. 2006;449:20–27.
69Nazem AG. A piece of my mind. Teach us how. JAMA. 2008;300:2463–2464.
70Alper E, Rosenberg EI, O'Brien KE, Fischer M, Durning SJ. Patient safety education at U.S. and Canadian medical schools: Results from the 2006 Clerkship Directors in Internal Medicine survey. Acad Med. 2009;84:1672–1676.