Quality improvement (QI) is center stage in medicine, promoted by the Institute of Medicine roundtable on evidence-based medicine in 20071 and the new requirement by the Accreditation Council for Graduate Medical Education (ACGME) in 2012 for QI training for graduate medical education trainees (e.g., residents).2 The learning health care system is defined as the consistent refinement of best practices developed from new knowledge generated in the health care environment through scientific inquiry, care culture, and informatics, and resulting in better health and health care for patients.3 Incorporating principles of a learning health care system for the purpose of system redesign is a goal of educators and practitioners throughout the health care system. The nature of such a change involves transformation at all levels of the system—policy and regulation, information technology, and cultural shifts across medicine and within specific institutions. The goal is to evolve into a system that can learn from real-time data to solve health care system problems.3 Techniques for this evolution include analyzing and applying “big data,”4,5 pragmatic trials at the point of care,6 feedback, and evidence-based policy change.3
Residency training is fundamentally challenging and intensive. Learning QI is not only an important and necessary skill to be acquired during this period but is now mandated.7 Evaluation of residents’ experiences with QI have generally been limited to assessments of QI knowledge acquisition, skills learned, and satisfaction with or attitudes about specific programs (e.g., didactic sessions).8–10 However, this evaluation does not address the deeper cultural change that is one of the goals of a learning health system. Learning the skills, vocabulary, and methods of QI is not the same as acquiring a positive attitude toward the practice and an intention to engage in the activity in the future. These latter variables are signs of a cultural change—that is, an internalization of the principles of QI in terms of both value and practice. A positive attitude towards QI is important if physicians are going to be leaders in activities to promote a learning health care system throughout their careers. Training efforts (both formal and informal) are more successful with a motivated population.11 Understanding how residents think about and experience QI can inform curricular offerings and promote facets of culture change. Many studies have demonstrated that physician leadership is key to success, and this assumption is the basis for the ACGME Clinical Learning Environment Review Pathways to Excellence initiative.12
The purpose of this study was to characterize resident attitudes, experiences, and beliefs of the QI experience in ACGME-accredited programs at our institution.
We conducted four focus groups with residents at the University of Utah School of Medicine. Residency program directors from five programs were approached for resident recruitment after an initial recruitment targeting residents was unsuccessful. We chose these five programs because they were part of a larger project examining residency geriatrics skills training. Of the five residency program directors, two declined participation on behalf of their residents (orthopedics and anesthesia) because of prior commitments during the focus group times. Three programs (neurology; physical medicine and rehabilitation; and emergency medicine) agreed to permit residents to participate during regularly scheduled educational times. Each focus group contained residents from a single program but across varied training years. We presented a brief study overview to residents. If they agreed to participate, they completed informed consent and provided information about their program and year of training. The institutional review board at the University of Utah approved all procedures. No direct compensation for participation was offered, but focus groups conducted around mealtimes did include a light meal.
We conducted the focus groups in accord with recommended procedures.13,14 An investigator with extensive focus group experience (J.B.) and the study program coordinator conducted the focus groups during September and October 2014. Facilitators had no role in assessing or evaluating any focus group participants. An identical script was used for each group—with an introduction to the purpose of the study followed by a series of questions. Questions were developed by the authors through a process of consensus and were designed to elicit broad information about resident experience with QI. Questions included a query about residents’ first impressions when hearing about QI, and a series of questions addressing experiences with QI generally and with specific projects (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A404 for the focus group moderator guide). The focus groups lasted approximately one hour. Each session was audio-recorded and transcribed with identifying information removed. We analyzed transcripts using a qualitative software analysis program (Atlas.ti; 2015, Scientific Software Development GmbH, Berlin, Germany).
We conducted qualitative analysis using an approach adapted from the principles of grounded theory15—beginning with the words of the participants, using multiple reviewers (including two behavioral scientists and a physician specializing in medical education), and iteratively generating conceptual themes. The goal was to review the transcripts beginning with the smallest concept identified by reviews and classifying these comments into precodes.16 These precodes were discussed by the group by looking at the associated quotations and then organized into higher-level constructs based on their perceived associations using the network display in ATLAS ti. The initial code list included 67 unique codes that we refined based on content similarity to a final list of 41 codes. Each transcript was reviewed and discussed multiple times. We resolved any disagreements by discussion and consensus. After no new concepts emerged, we grouped codes by overarching conceptual similarity and identified themes.14,17
A total of 45 residents in three programs participated in the focus groups (see Table 1). The sample size was constrained by residency program director permission for recruitment and the number of residents willing to participate.
Perceptions of meaning and QI experiences
The foundational perspective that emerged in this study was the residents’ mental model of QI. A mental model is a representation of external reality, hypothesized to play a major role in cognition, reasoning, and decision making.18 Residents reported that QI included practical tasks such as “projects,” “safety measures,” and “evaluation of systems.” Residents reported formal and informal experiences with didactic and experiential training opportunities. Six Sigma and Lean principles were mentioned as inextricably related to QI and part of the training in QI that residents received.19,20 QI efforts were perceived to include hospital-wide programs, specific department-level projects, and individually led activities. Residents described QI as mandated “to ensure that physicians or providers stick to guidelines that are measures for quality,” and “scrutiny.” Another category was related to cost; for example, QI was for “saving money.” This element was described both positively (“I don’t know if it affects patient outcomes or not, but you’re saving the hospital money, that was a useful idea”) and more negatively (“trying to find a way to avoid penalties and that’s frustrating because our interest is the patient. I don’t care how much.… If it costs the hospital an extra $10, an extra $15, I want my patient to get the best care possible”). Reponses to this question suggested that residents had a mixed impression of QI. The diversity and inconsistency of responses suggest a lack of a commonly held mental model of the meaning of QI. The residents’ mental model of QI appeared to drive their approach, beliefs, attitudes and the perceived quality of their experience. These mental models constituted the perceived meaning of QI and in many ways appeared splintered, seemed composed of partial facts about QI, and varied substantially across the residents. Under this overarching general perspective, five themes—four barriers and one facilitator for doing QI—emerged from the qualitative analysis.
Of the barriers to and facilitator of QI, each theme is reported briefly in List 1 along with representative quotes. The full list of quotes is available by request from the authors.
Themes Identified From Focus Groups With 45 Residents Discussing Quality Improvement (QI), From a Study of Resident Attitudes About QI, University of Utah, 2014
QI Participation Barriers
Theme 1 (Understanding the vision is challenging): Competing goals make understanding the vision, purpose, and philosophy of QI challenging.
- “You’re not relying on the status quo … [but] showing that you’re trying to improve and I think patients can see that.”
- “I think the attendings … probably don’t have a lot of awareness about this because it is something that maybe like is newer medicine, I guess … questioning the existing protocols.”
- “An important process … but a difficult one politically to pull off.”
- “How do you address pain? With a lot of our patients, addressing pain is counterproductive. We are dwelling on the pain rather than recovery or functionality…. I don’t want to ask them to rate pain but … they’re asking us to rate pain.…”
- “I don’t know like the value of comparing across specialties because for instance with pain, we talked about this a lot like specifically in neurology … we need to kind of hold back on medications to preserve their neurologic exam and so that’s a little bit unique to us as a specialty so you feel like you’re … between a rock and hard place.”
- “There’s like the central line infection protocols…. If you suspect that anybody has any type of bacteremia, you don’t do a blood culture, you just do a urine culture and pull the lines … we just don’t even test for it because the quality improvement then like marks you off.”
- “This hasn’t really made any difference to the patients. Like this checklist we do on rounds, like I don’t know. Maybe it has.”
- “Quality improvement is this cash word that gets thrown out there that is certainly on the quality meeting list, but kind of worthless.”
- “Somebody’s PhD project.”
- “More of like a business-driven process rather than necessarily like a health care process.”
- “I feel like patient ratings are really … the institution is making a push for patient satisfaction, which is important, but I think there have been studies that show that changing your actions to please patients’ expectations may not be the best thing especially when it comes to pain and when it comes to things that are safety issues for patients as well, that maybe the physician is better qualified to do and that’s just something in general.”
Theme 2 (Confusion): We don’t know what is going on.
- “I feel like those residents … don’t really get that much feedback on what the quality improvements are, what the projects are, and then what the results are…. All the resident is doing is asking them for help in order to collect the data.”
- “Just that lack of understanding and potentially also among attendings.”
- “You start off with a lot of gusto and momentum and it fizzles out before they complete. Maybe a half a year ago or so we had a big movement and flurry of e-mails saying that we needed to secure all of our computers and install some software and lock them down and I use that in medical school and makes a computer [unintelligible] swell, and then you get e-mail, e-mail, e-mail, and two weeks later everything is gone and then no one talks about it anymore so where did it go? Those things got up and went, they carry themselves out but they [unintelligible] [laughing].”
- “I don’t feel like there’s a lot of dissemination of information, there’s not a lot of promotion of … there are some of the quality initiatives. I don’t even know what process to go about working on or implementing or who to talk to to begin a quality improvement initiative.”
- “Nobody sat down and said if you want us to do a quality improvement project here is how you could get started on it…. Everybody is saying, you better publish papers here…. You really don’t have time to do both, so which one do you want us to do?”
- “Maybe [after] a few months of e-mails I got some data, but then like during a presentation there were people whom I had never heard of and never met who had different data that I would like to have.”
- “You’re asked to go data mining, do chart review, and you want to.”
- “So all our [clinical events] are monitored…. There’s someone … kind of watching, making sure we have like certain markers and timing and I am aware that there’s people who are not directly involved with patient care who are watching us a lot, but I don’t know the results.”
- “I think that there’s a lot of talk and not a lot of action objectively. Patient satisfaction, callbacks. This has been a huge _______ for us that I don’t think has been demonstrated to show improved outcomes.”
- Theme 3 (Resident value): We are not a valued/valuable part of the process.
- “I think the biggest problem with these initiatives is that we’re told to do them. There’s no buy-in from us, right? So somebody, some faceless person says, this is supposed to be an improvement in patient outcomes and you will do it.”
- “Somebody is forcing her, somebody is twisting her arm.”
- “We don’t need to discuss it every day to know that it’s important…. Marking it down on a sheet of paper that you’ve checked that box isn’t important to [residents]…. I think it’s best when kind of from the ground level up, but often I find it is top down.”
- “I think no matter where any of us go, these are going to be enforced upon us by our hospital administrators to save money for reimbursement.”
- “You don’t know your role other than to collect data…. I mean that probably is your only role, which is kind of a sucky role.”
- “When we’re asked to implement an intervention and study the results and we’re not allowed to implement an intervention. That is frustrating.”
- “Your work is pretty futile…. You’ve tried to provide that data and you’ve also tried to come up with solutions.”
- “It just seems we’re not valued.”
- “Sometimes we have no authority to implement [QI].”
Theme 4 (Overload): Prioritizing our QI work is difficult given other responsibilities.
- “Truly the first thing I think of when I hear [QI] is going to make more work for residents.”
- “We’re doing the vast majority of the work actually.”
- “Everybody is strapped for time as is. So if we don’t have any kind of set-aside time when we’re supposed to.”
- “Timing like mental bandwidth…. If you have to think about like some of the little jots and tittles you miss some of the big things….”
- “It feels like a lot of extra work for something that for residents doesn’t feel that important and we don’t even know if all these questions that we’re filling out and things we’re suggesting to do will make any difference in our scores.”
- “We don’t know what we get from it.”
- “It directly affects us, I think, that project so if we have a patient that falls they’ll often stay in the hospital longer…. It’s more work when the patient stays an extra day and so we obviously don’t want that to happen.”
QI Participation Facilitators
Theme 5 (Positive aspects): The QI experience is positive when:
- “I did a research project that was not initially designed as a QI project, but then they found the results that we got … we revamped our inpatient program … because of the data we got…. It fit into both categories, it actually had an academic research purpose but it also has a quality improvement purpose with it.”
- “That’s actually one of the reasons I like it because it’s like an immediate.”
- “An opportunity to kind of direct … what is actually going to happen in the future.”
- “An obvious goal, easily achievable,” and “we all agreed [it] was useful and we all supported [it].”
- “Doing some of the basic literature search is kind of educational.”
- “Overall [QI] is very educational. I wish that we had some more time set aside for it.”
- “It was communicated to the chief [of the division] … and then the chief communicated that to his attendings and to the inpatient attendings, and then the inpatient attendings, we built groups around our kind of shared interests.”
- “I did a research project that was not initially designed as a QI project, but based on the results we got in comparison to other research we revamped our inpatient protocol ... because of the data we got from the paper.”
Theme 1: Understanding the vision is challenging.
Competing goals make understanding the vision, purpose, and philosophy of QI challenging.
On the one hand, residents reported intense eagerness to improve care for patients. On the other hand, residents reported a sense that QI disrupted and interfered with the system of care in important ways and had political ramifications. There were concerns that QI goals were in opposition to patient or provider care goals—potentially even frequently—with examples cited that included distracting checklists and care protocols that were seen as interfering with the highest-quality patient care. QI was considered not particularly effective (e.g., “somebody’s PhD project”). The residents perceived a disconnect between quality processes and care processes—many seemed to perceive saving money or achieving quality milestones as of minimal significance to patient care and were concerned about the focus in the health care system on patient satisfaction. Within this theme, residents’ statements reflected the concern that doing QI competes with providing high-quality care. There was an indication that residents did not always understand the administration’s goals or always agree with them.
Theme 2: Confusion.
Residents expressed profound confusion about the QI process and their role within the process: “We don’t know what is going on.”
Many residents reported a general feeling of disconnect from the process of QI and reported the impression that attending physicians also didn’t understand the QI process well. Many residents reported difficulty in getting information about ongoing initiatives or follow-up about completed projects. There were several reports that maintaining awareness of ongoing QI efforts was difficult. There was a general sense that QI involved close monitoring or “watching,” but for many projects residents didn’t feel informed about the results of that monitoring.
Residents reported the sense that QI projects were desirable but that there were difficulties with understanding how to manage the process—how to begin, the tension of choosing between QI and research, how to disseminate the results, and how to manage other responsibilities.
Those who participated in QI directly had difficulties particularly with retrieving and analyzing data. Although residents understood that QI was potentially related to research and publishing, could improve patient care, and could contribute to career advancement, the specifics were not clearly identifiable, as two illustrative comments show:
Maybe (after) a few months of e-mails I got some data, but then like during a presentation there were people whom I had never heard of and never met who had different data that I would like to have.
This is not going to be enough to do a study … quality improvement may or may not even be published.
In synthesis, the goals of QI were difficult for residents to discern. There was pervasive confusion about all phases of the process from choosing to participate to disseminating results.
Theme 3: Resident value.
Residents did not feel valued as part of the QI process: “We are not a valued/valuable part of the process.”
There was a general sense that QI was inflicted on residents. Vivid language, such as “forcing” and “twisting her arm,” was used. In addition, residents referred to a sense that QI originates with hospital administrators “no matter where we go.” This language suggests that unknown (or barely known, faceless) powers are compelling residents’ actions.
Residents had a sense that their contributions were not particularly important or valuable to faculty or others, with roles that were unknown or viewed as low level. Some residents reported experiences in which they conceived of their own QI projects but then experienced significant difficulties implementing them. In some cases the perception was that their superiors blocked implementation, and this was a source of frustration. Residents used words such as “futile” and felt a distinct lack of authority to compel any actions. In other words, residents felt deeply frustrated when ideas or projects were rejected and, overall, reported very little control over their QI-related activities.
Theme 4: Overload.
“Prioritizing our QI work is difficult given other responsibilities.” Residents continuously expressed the idea that they were overworked. A lack of dedicated time for QI was a particular problem. There were concerns about cognitive overload—that important patient care might be neglected to “check a box.”
Participating in QI projects was seen as burdensome—with particular concern that it might not benefit the residents’ career. In contrast, residents saw greater value in participation if a direct benefit to the resident was perceived. One resident remarked:
It directly affects us, I think, that project so if we have a patient that falls they’ll often stay in the hospital longer.… It’s more work when the patient stays an extra day and so we obviously don’t want that to happen.
Residents were more invested in supporting QI projects when they perceived a tangible personal benefit to the project outcomes—suggesting that projects that increased resident workload might be difficult to implement with resident involvement.
Theme 5: Positive aspects.
“The QI experience is positive when.…” Residents reported many positive aspects of involvement in QI. The participation of a mentor was seen as an essential component of an excellent QI project experience. Residents saw QI as influential—affecting both current and future practice within medicine. There was a sense that QI was particularly beneficial when there was broad support from stakeholders with “shared interests” incorporated into the project. QI was rarely mentioned as part of a learning program directly, but when it was, it was viewed as a positive part of the QI process overall. Residents indicated that QI was a positive process when it was an effort of shared vision and stakeholder support, with potential to affect future practice in the institution.
Physician involvement in QI is critical for any specific QI project’s success as well as to the overall effectiveness of a learning health care system transformation. The envisioned cultural change requires that physicians’ knowledge, attitudes, and behaviors are part of the foundation for transforming institutions into learning health care systems.3 The findings from our qualitative study suggest that implementation of the ACGME mandate for resident training in QI has room for improvement to foster a positive attitude among residents of buy-in and the perceived quality of their QI experience.
The overarching frame of this work is that residents are unclear, unsure, and not in agreement about the purposes and meaning of QI initiatives as part of their training. The direct consequence is a failure to appreciate the extra effort, to feel valued, and to understand how to be effective. Improving alignment on this very important institutional goal may be a foundational step in the transformation process.
Residents reported a tension between viewing QI activities as research (to be conducted with rigor) versus viewing QI activities as patient safety improvement (more emphasis on changing clinical processes). Our observation is that this particular tension undergirds much of their negative experience. A review of resident experience in QI found that academic medical centers do not value QI as an academic pursuit (in contrast to research),21 which appeared to be the case among the residents in this study and contributes to the related “muddy” mental models of the meaning of QI. Clarifying the need for rigor in QI and providing more tools, such as point-of-care trials, to conduct a scientific inquiry at the bedside may be a solution to changing resident attitudes by promoting QI as an important academic activity.
Consistent with other research, residents in our study described a wide variation in QI experiences. Resident involvement primarily included training in QI principles (e.g., Lean and Six Sigma) and participation in projects, both individual and hospital-wide,22,23 and didactic and experiential components employed in many programs.8,24 Consistently, residents were enthusiastic about promoting real change and improving patient care. Although this was not seen as a constant outcome related to QI, many residents saw this as a possibility or greater good and a way to impact current and future care. This is consistent with other research that has found a relationship between experience and the sense of the value of QI.25
The residents’ experiences may reflect a lack of commonly held purpose throughout the institution. Resident statements in this study reflect a tension between the care that providers want to deliver to patients and what they perceive the “bean counters” want to do. Interestingly, no residents mentioned that cost-saving measures might benefit patients as well as the hospital, suggesting that framing QI as supporting patient needs may be important. Residents expressed skepticism about the relationship between patient satisfaction and care quality—a skepticism that has been explored in a number of ways in other studies, with results suggesting that patient satisfaction is associated positively with some quality indicators and negatively with others.26
Many evaluations of programs supporting QI project involvement or practice-based learning and improvement focus on increasing QI knowledge and satisfaction,27 but it may also be important to capture the affective valence surrounding QI. The sense of “hurt feelings” revealed in this study is unlikely to be revealed in more structured measures with closed-ended questions. For an effective learning health care system, QI ideas and activities must be generated by clinicians on the ground.28,29 Initiatives seen as “top-down” were perceived as very far away from clinical care in the trenches. To transform the QI experience for residents, it is important to incorporate their ideas.
Limitations and strengths
This study was qualitative and not designed to be a comprehensive assessment of QI at the single institution where it was conducted. We selected resident participants only from the three programs in which their participation was supported by the residency program director. Residents in other programs may have had different experiences. Study strengths include the frank voices of the resident participants who were clear about their frustrations.
Implications for residency training programs
If resident attitudes about QI do not improve, the culture of QI is threatened. It is critical to address these attitudes and improve the sense of value of QI.
Based on the experience of the residents in the current study, we suggest the following tactics to incorporate residents in QI in a successful way.
Provide a mentored experience.
A mentor with adequate time to support residents in QI is able to guide residents to useful topics likely to be accepted by colleagues and superiors, reducing frustrations related to ideas being rejected. Ideal mentors would have greater knowledge about data access and supports and could introduce residents to appropriate processes and resources. It is critical that such mentors have necessary skills and dedicated time to support residents.
Clarify implementation procedures and disseminate results.
Residents could benefit from a consistent pattern of QI dissemination explaining project origins and goals, and showing results in methods frequently accessed by today’s residents. A Twitter feed might be much more helpful than a flyer, for instance.
Incentivize the process for residents.
Developing incentives that illustrate residency benefits and later career benefits, or even monetary incentives, would reduce frustrations and confusion about the time burden of participation and potential benefits. In addition, a mentor could provide a tangible benefit to a resident’s career as well as highlighting future career benefits. Financial and mentor incentives have been successfully implemented at other institutions.30,31
Assuage concerns about “business” versus “clinical” goals.
Fundamentally, a well-functioning health care system attends to multiple areas of importance. Saving money through safe quality measures enhances high-quality clinical care.
Highlight successful QI strategies.
These may include strategies that have been integrated into clinical care, such as the use of ultrasound guidance and sterile draping for placing central lines, to demonstrate to residents the important role QI activities play in patient care improvement.
Enhancing the quality of residents’ understanding and experience of QI is likely to result in improved attitudes about QI and advance the important vision of achieving a learning health care system.
Acknowledgments: The authors wish to thank the D.W. Reynolds Foundation for its generous support of this research; the residency program directors at the University of Utah School of Medicine who facilitated data collection for this study; and Madison Briggs for her able coordination support at all phases of data collection and manuscript preparation.
1. Olsen L, Aisner D, McGinnis JM; Roundtable on Evidence-Based Medicine, Institute of Medicine of the National Academies. The Learning Healthcare System: Workshop Summary. 2007.Washington, DC: National Academies Press.
2. Accreditation Council for Graduate Medical Education. ACGME common program requirements. http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_07012015.pdf
. Published 2015. Accessed September 9, 2016.
3. Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2013.Washington, DC: National Academies Press.
4. Angus DC. Fusing randomized trials with big data: The key to self-learning health care systems? JAMA. 2015;314:767768.
5. Grumbach K, Lucey CR, Johnston SC. Transforming from centers of learning to learning health systems: The challenge for academic health centers. JAMA. 2014;311:11091110.
6. Klein S, Hostetter MK. In focus: Learning health care systems. Quality Matters. August/September 2013. http://www.commonwealthfund.org/publications/newsletters/quality-matters/2013/august-september/in-focus-learning-health-care-systems
. Accessed September 9, 2016.
7. Holland R, Meyers D, Hildebrand C, Bridges AJ, Roach MA, Vogelman B. Creating champions for health care quality and safety. Am J Med Qual. 2010;25:102108.
8. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: A systematic review. JAMA. 2007;298:10231037.
9. Sockalingam S, Stergiopoulos V, Maggi J, Zaretsky A. Quality education: A pilot quality improvement curriculum for psychiatry residents. Med Teach. 2010;32:e221e226.
10. Wilper AP, Smith CS, Weppner W. Instituting systems-based practice and practice-based learning and improvement: A curriculum of inquiry. Med Educ Online. 2013;18:21612.
11. Noe RA, Schmitt N. The influence of trainee attitudes on training effectiveness: Test of a model. Pers Psychol. 1986;39:497523.
12. Weiss KB, Bagian JP, Wagner R, Nasca TJ. Introducing the CLER Pathways to Excellence: A new way of viewing clinical learning environments. J Grad Med Educ. 2014;6:608609.
13. Krueger RA, Casey MA. Focus Groups: A Practical Guide for Applied Research. 2000.Thousand Oaks, CA: SAGE Publishing.
14. Morgan DL. Focus Groups as Qualitative Research. 1997.2nd ed. Thousand Oaks, CA: SAGE Publishing.
15. Bryant A, Charmaz K. The SAGE Handbook of Grounded Theory. 2007.Los Angeles, CA: SAGE Publishing.
16. Patton MQ. Qualitative Research & Evaluation Methods. 2002.3rd ed. Thousand Oaks, CA: SAGE Publishing.
17. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:12771288.
18. Craik KJW. The Nature of Explanation. 1943.Cambridge, England: University Press.
19. Womack JP, Jones DT, Roos D. The Machine That Changed the World: The Story of Lean Production. 1994.New York, NY: Productivity Press.
20. Varkey P, Reller MK, Resar RK. Basics of quality improvement in health care. Mayo Clin Proc. 2007;82:735739.
21. Patow CA, Karpovich K, Riesenberg LA, et al. Residents’ engagement in quality improvement: A systematic review of the literature. Acad Med. 2009;84:17571764.
22. Keitz SA, Holland GJ, Melander EH, Bosworth HB, Pincus SH; VA Learners’ Perceptions Working Group. The Veterans Affairs Learners’ Perceptions Survey: The foundation for educational quality improvement. Acad Med. 2003;78:910917.
23. Majka AJ, Cook KE, Lynch SL, et al. Teaching quality essentials: The effectiveness of a team-based quality improvement curriculum in a tertiary health care institution. Am J Med Qual. 2013;28:214219.
24. Ogrinc G, Headrick LA, Mutha S, Coleman MT, O’Donnell J, Miles PV. A framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003;78:748756.
25. Lipstein EA, Kronman MP, Richmond C, White KN, Shugerman RP, McPhillips HA. Addressing core competencies through hospital quality improvement activities: Attitudes and engagement. J Grad Med Educ. 2011;3:315319.
26. Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: A national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172:405411.
27. Varkey P, Karlapudi SP. Lessons learned from a 5-year experience with a 4-week experiential quality improvement curriculum in a preventive medicine fellowship. J Grad Med Educ. 2009;1:9399.
28. Williamson JW, Walters K, Cordes DL. Primary care, quality improvement, and health systems change. Am J Med Qual. 1993;8:3744.
29. Holmboe ES, Cassel CK. The role of physicians and certification boards to improve quality. Am J Med Qual. 2007;22:1825.
30. Vidyarthi AR, Green AL, Rosenbluth G, Baron RB. Engaging residents and fellows to improve institution-wide quality: The first six years of a novel financial incentive program. Acad Med. 2014;89:460468.
31. Rosenbluth G, Tabas JA, Baron RB. What’s in it for me? Maintenance of certification as an incentive for faculty supervision of resident quality improvement projects. Acad Med. 2016;91:5659.