Proven quality improvement (QI) methodologies have been an important part of industry standards during the past half century and are needed (and long overdue) in health care to improve quality and safety in patient care. The preparatory work with QI education should ideally start early in a medical professional’s career. Most medical schools and residency/fellowship training programs, however, have minimal to no QI educational programs, despite the Accreditation Council for Graduate Medical Education’s (ACGME’s) more recent emphasis on including QI education.
In this Innovation Report, we describe a pilot for QI education that was implemented as a required curriculum for the Mayo Clinic Combined Critical Care Fellowship (CCF) program’s 2010–2011 academic year. Considered successful, the pilot is now in its fourth academic year as an established and integral part of the fellowship’s curriculum and training.
The objective of the CCF QI education pilot was to deliver just-in-time QI training that could be effective and immediately applied to improve patient care as an integral part of the demanding CCF curriculum, and as a model for other fellowship programs to follow.
Delivery of the QI education curriculum involved two experienced Mayo Quality Academy (MQA) instructors to teach and act as coaches, as well as five experienced physicians who were content experts and functioned as project champions.
The QI education curriculum consisted of 17 weekly 90-minute training sessions over a five-month period, beginning at the start of the academic year in July 2010. Instructions included didactic teaching and workshops. The content of the sessions included project discussion sessions (break sessions) blended with didactic classes. The didactic classes were designed to cover Bronze-Level (QI terminology) and Silver-Level (gaining knowledge in core QI methods including selection, initiation, and prioritization of QI projects along with appropriate conduct of a patient-outcome-driven QI project) material. (See Supplemental Digital Appendix 1, https://links.lww.com/ACADMED/A220). These break sessions were used to discuss the quality projects and enable greater fellow participation in the process.
To accommodate the fellows’ schedules, most of the sessions were done over the lunch time and included food, which improved their ability to participate.1 The training sessions were recorded and the materials made available if fellows could not attend. To ensure that the cohort stayed on track, a test was administered to the fellows after reviewing the online material to ensure comprehension of the session’s materials, with a passing rate of 80%. At the end of each of the four main subject areas (that included quality management tools, selecting QI methods, applied quality essentials, and finally champions training [see Supplemental Digital Appendix 1, Table B at https://links.lww.com/ACADMED/A220), the fellows individually took the computer-delivered test (pass rate was also 80%) offered by the Mayo Quality Fellows curriculum. Those who did not pass studied the session materials in more detail and, if requested, received additional individual training.
After the first month of instruction, the fellows began selecting projects and were assigned to teams. Potential QI projects were discussed and investigated, then assigned by the group into an Impact and Effort prioritization tool (Supplemental Digital Figure 1 at https://links.lww.com/ACADMED/A220). Projects were selected by coaches, champions, and faculty on the basis of their education value and their potential to improve patient care with the least effort or difficulty.
Participants and surveys
All 20 critical care internal medicine (CCIM) fellows who were in training from 2010 to 2011 participated in the QI education curriculum. We surveyed the fellows three times. The first (30 days into the curriculum) and second (3 months after the curriculum’s completion) surveys were sent to all fellows to deter mine whether the curriculum’s content and delivery were effective (see Supplemental Digital Table 1 at https://links.lww.com/ACADMED/A220). We administered the third survey 10 months after graduation to evaluate the impact of QI training on the fellows’ career development. The primary outcome was to assess learner satisfaction, knowledge, and skill transfer. Secondary outcomes were to evaluate the effectiveness of the curriculum in the fellows’ careers (Supplemental Digital Table 2 at https://links.lww.com/ACADMED/A220) and its impact on patient care processes. We used a five- point Likert scale for all surveys. We used Research electronic data capture (REDCap) Survey Software (version 1.3.10; Vanderbilt University, 2013) to collect the survey data. We used Wilcoxon sign-rank test to compare the survey results before and after the intervention. For the statistical summaries and analyses, we used JMP software version 9.0.1 (SAS Institute, Cary, North Carolina). This study was exempted by the Mayo Clinic Institutional Review Board.
Descriptions of the participant and curriculum characteristics are shown in Table 1. Of the 20 fellows enrolled in the 2010–2011 CCIM program, most were male (17/20, 85%). One fellow was trained in emergency medicine and another subspecialized in infectious disease prior to starting their CCF. All other fellows were board certified in internal medicine. Four fellows graduated from U.S. medical schools, whereas the remaining 16 fellows were international medical graduates.
Learner satisfaction and impact on curriculum
Implementing the QI training program involved a substantial increase in the amount of material and in-person, mandatory didactic sessions to the fellows’ weekly schedule. The fellowship program leadership and the course coaches and champions were very interested in determining the value of these additions to an already-busy fellowship program. Indeed, several fellows expressed initial concern at the additional lecture time, especially with no precedent to establish the utility of these sessions.
The response rates to the three surveys, mentioned above, were 65% (13/20), 45% (9/20), and 75% (15/20), respectively. The aggregate results of the surveys were shared with the class, coaches, and champions (Supplemental Digital Tables 1 and 2 at https://links.lww.com/ACADMED/A220).
The fellows’ qualitative comments were nearly all positive and included suggestions for future improvement, such as a more condensed curriculum and the continued emphasis on projects with real-time and patient impact. Overall, the fellows seemed to enjoy discovering the practical importance of QI expertise and recognized the advantages of this training as they entered the job search process. Three fellows identified a future goal of achieving the Gold-Level Quality Fellowship, four offered to become quality mentors for the next academic year, and two fellows made professional decisions to focus on QI as a subspecialty.
In response to the statement in the first and second surveys, “I felt prepared to direct QI activities in my future practice,” the fellows’ mean rating increased significantly after training from a mean near 2 (“Disagree”) to just above 4 (“Agree”) on a five-item Likert scale. Using the Wilcoxon sign-rank test, this increase was statistically significant (Δ mean 2.35, 95% CI: 1.84–2.86, P < .0001) (Figure 1).
In our seven-question follow-up survey conducted 10 months after graduation, we wanted to measure the QI curriculum’s impact on the physicians in their professional practice to improve quality and, after a time of reflection, to give an overall rating of this aspect of the CCF curriculum (Supplemental Digital Table 2 at https://links.lww.com/ACADMED/A220). Although 12 (80%) responded that they have initiated or are currently participating in QIs in health care, which was exceptionally encouraging, we were concerned that 3 fellows (20%) felt that the QI curriculum had displaced other important topics. We used this information to streamline the teaching and project application process in the following years as well as include additional discussion on the justification and relevance of QI training in health care.
Knowledge and skill transfer
Fourteen of the initial 20 fellows (70%) achieved the Mayo Bronze and Silver Quality Fellowship. Of the 6 fellows that did not obtain Silver, 1 left the program early, 2 did not pass the exams, and one team representing 4 fellows did not successfully get their project approved prior to graduation, mainly because of a lengthy and confusing quality project approval process. The 14 that were successful fulfilled the requirements of passing four exams and completing a QI project, which was peer reviewed by each other, coaches, and champions, and approved by the MQA and the Mayo quality review board. It should be noted that one of the tests, “Applied Quality Essentials,” was designed to fulfill the maintenance of certification and continuing education requirements for quality and has been approved by the American Board of Internal Medicine and eight other medical boards.
The five selected QI projects involved multidisciplinary collaboration among physicians, nurses, respiratory therapists, and pharmacists. All projects had the underlying requirement that there could be no risk to patient safety. The teams used the DMAIC (Define, Measure, Analyze, Improve, and Control) methodology2 to structure their efforts and add format consistency to presentations. The results of the QI projects were reported and peer reviewed by each other and the coaches and champions during semiannual presentations. Professional posters were presented at the Mayo Clinic Quality Academy Conference.
All five projects had a positive impact on patient safety and care and resulted in subsequent projects in the next academic year (Table 2).
We used the Kirkpatrick validated model to evaluate the efficacy of our QI training model.3 We evaluated the satisfaction of fellows to the QI training by surveys, assessed the extent of their learning (on the basis of how many became eligible for Silver-Level certification), assessed changes in their behavior (patient outcome), and evaluated the QI curriculum’s impact on their career goals. On the basis of our data, we showed that implementation of a new curriculum for QI training is feasible and can be successful. With new requirements for QI training by regulating bodies like ACGME, and a lack of standard QI training models, our QI education model could be adopted by other fellowship training programs. In fact, three other fellowship and residency programs at the Mayo Clinic have adopted our curriculum to train their fellows and residents in QI, showing that our QI education model can be generalized and may be applicable to other training programs.
Integrating yearly trainee education with sustained improvements in care
With their QI projects, the teams were able to reach an initial stage of project completion, completing a basic DMAIC cycle. The nature of postgraduate medical education is that new learners constantly rotate in and out of the curriculum. Achieving results in a short amount of time and sustaining the results (the Control phase of DMAIC) can be difficult for any QI project, especially under these circumstances. We attempted to eliminate this by engaging subsequent groups of fellows into related next-project phases of some of the initial projects described above. For example, one of the projects in the 2012–2013 academic year was “Noise Reduction in the Intensive Care Unit (ICU),” which was directly related to the “Improve Compliance with Sleep Enhancement” project in the ICU from the 2010–2011 academic year. Because QI is a cyclical, never-ending process, the nature of these projects is such that new energy and new ideas will fuel their maintenance and further development. In so doing, learners come to understand the longitudinal nature of QI, and patients will benefit from a sustained effort at bettering the clinical practice.
Future improvements and application
The 2010–2011 pilot was determined successful and has continued in the future academic years. Some of the most significant improvements were incorporating former quality-trained fellows as mentors and teachers; a preassessment of quality knowledge at the beginning of the year; increased and earlier use of allied health to assist in determining and implementing projects; additional training on navigating the project approval process and quality review board; formal presentations of the projects at the Mayo medical critical care grand rounds and to outside organizations; and the use of a Blackboard education system to deliver materials, communications, and assessments.
It should also be noted that there is often a distinct paradigm shift in residents to think of QI separately from pure research and scientifically based projects; this lack of understanding can be a real obstacle to accomplish fast and meaningful QI improvements in health care. This created a delay in the Silver-Level certification process for four of the fellows. For this reason, we included the topic of “QI versus research: differences and similarities” in the following-year curriculum.
There are limitations to our study. Our sample size was small (limited to one year of a fellowship program that recruits 20 fellows), and the study was conducted in a single institution and in a single fellowship program.
Improvement in the quality of patient care and patient safety is not only a moral obligation, it has become a professional and economic necessity in light of the reforms ushered in by the Affordable Care Act of 2010 and ACGME accreditation changes of 2006, 2009, and 2013. Our external obligations to legislative and accrediting bodies, along with Mayo Clinic’s drive to constantly improve the value of care, compel training programs to plan for explicit training in QI. Although there are no established guidelines for QI education, the CCF program has been able to incorporate advanced QI methodology training for the fellows within the critical care core curriculum. Subsequent survey data suggest that fellows considered the QI training program to be educationally valuable, and it has also measurably improved the care of ICU patients and boosted fellows’ academic productivity. As we advance further into this era of closely monitored quality and safety in health care, it is absolutely necessary to investigate the efficacy of the training modules in a national and global collaborative manner to be able to establish better and more efficient training resources in the future.
1. Segovis CM, Mueller PS, Rethlefsen ML, et al. If you feed them, they will come: A prospective study of the effects of complimentary food on attendance and physician attitudes at medical grand rounds at an academic medical center. BMC Med Educ. 2007;7:22
2. Pyzdek T, Keller PA The Six Sigma Handbook: A Complete Guide for Green Belts, Black Belts, and Managers at All Levels. 20103rd ed New York, NY McGraw Hill
3. Kirkpatrick DL, Kirkpatrick JD Evaluating Training Programs. 20063rd ed San Francisco, Calif Berrett-Koehler Publishers, Inc.