Voss, John D. MD; May, Natalie B. PhD; Schorling, John B. MD, MPH; Lyman, Jason A. MD, MS; Schectman, Joel M. MD, MPH; Wolf, Andrew M.D. MD; Nadkarni, Mohan M. MD; Plews-Ogan, Margaret MD
Our conversations about quality and safety in health care have begun to change, thanks to the Institute of Medicine reports To Err is Human and Crossing the Quality Chasm.1–3 Real change will come, however, as our next generation of physicians is “raised up” on this new conversation about teamwork and systems, continuous process improvement, and practice based on evidence and outcomes. Training programs have a unique opportunity to catalyze change by preparing residents and faculty leaders in these new disciplines. With the future of health care in mind, the Accreditation Council for Graduate Medical Education (ACGME) established six competencies, broadening the traditional training focus from medical knowledge and patient care to underscore that physicians must be skilled in systems-based practice (SBP), practice-based learning and improvement (PBLI), professionalism, and communication. The ACGME also recognizes that these competencies must be measured in meaningful outcomes. Improvements in quality and patient safety are such outcomes and can be powerful tools for teaching these competencies. Under Title VII, the Health Resources and Services Administration (HRSA) provides national leadership and resources to improve patients' access to culturally competent, high-quality health care. With HRSA support, complemented by funding from the Robert Wood Johnson Foundation (RWJF), we developed an innovative curriculum in quality and patient safety for internal medicine residents at the University of Virginia, in order to teach competency in elements of SBP and PBLI.
Building a Foundation
Our curriculum evolved throughout six years to reach its present form. Beginning in 2000 with a Title VII Academic Administrative Unit grant, we built a research and curriculum infrastructure within the division of general medicine that targets the quality and safety of care for underserved populations. Using this foundation, we developed an interdisciplinary patient safety team to investigate ambulatory adverse events and near misses in our residents' clinic. This program produced a 70-fold increase in error reporting and important improvements in care.4,5 From the experience gained, the faculty designed our initial patient safety curriculum for internal medicine residents.
At the same time, under the RWJF Take Care to Learn (TCTL) chronic care collaborative,6 we developed an experiential learning curriculum to teach chronic illness care and quality improvement (QI) to a multidisciplinary group of resident and faculty learners. Our faculty also participated as teachers in the Achieving Competency Today program, a Harvard Pilgrim/RWJF follow-on program to the TCTL that instructs graduate nursing students and residents how to design and implement QI projects.7
On the basis of the success of these endeavors, a core general internal medicine faculty group plus an informatics-trained physician used our HRSA Academic Administrative Unit 2003 and 2005 grant renewal funding to develop methods to teach the ACGME PBLI and SBP competencies to internal medicine residents, including developing the Systems and Practice Analysis for Resident Competencies (SPARC), a Web-based resident-profiling system.8 With an HRSA Title VII Faculty Development Program grant award from 2003 to 2006, we expanded our QI training to create the Chronic Illness and Quality Improvement Faculty Development Program. During these six years of continuous Title VII funding, supplemented by RWJF support, we built the infrastructure and faculty expertise to create and implement an experiential curriculum in internal medicine (Chart 1) to teach SBP and PBLI competencies that uses quality and patient safety as an educational thread to guide participants' experiences.
Case Part A: Mr. S. is a 69-year-old man hospitalized for congestive heart failure and pneumonia. On admission, intermittent pneumatic compression device calf boots (ICDs) were ordered for deep vein thrombosis (DVT) prophylaxis. On hospital day four, he was noted to have an abnormally rapid heart rate, and his ICDs were found on the floor beside his bed. A computed tomographic pulmonary angiogram demonstrated a pulmonary embolism.
This case study illustrates one of the QI and patient safety projects completed by first- and second-year residents at the University of Virginia. Excerpts from the case study appear throughout this article to demonstrate how parts of our curriculum correspond to actual clinical outcomes. Our two-year curriculum for internal medicine residents allows learners to acquire and practice new skills under the direction of faculty selected for their ability to model these competencies. We incorporated principles of learning organizations9 to create a “safe space” for residents' inquiry into and conversations about safety and quality. This curriculum reflects the values and interests of general internal medicine physicians as underscored by recent calls to redesign residency training, and it clarifies the roles and expectations of clinician–educators.10 The two-year curriculum is divided into four, six-month-long periods (fall and spring) each year. In these fall and spring periods, each resident has a one-month mandatory ambulatory rotation that contains four, three-hour seminars scheduled weekly. The ambulatory rotation and seminars are repeated each month within the fall and spring periods. Approximately 35 residents participate in each year of the curriculum, and in each seminar one or two faculty teach four to six residents. The topics for first- and second-year residents differ (see Chart 1), and the curriculum for each training year builds on topics and skills previously learned. By the end of the curriculum, residents have participated in 16 seminars comprising 48 contact hours of instruction. In the remainder of this article, we describe the seven QI and patient safety seminars (21 contact hours) of the curriculum indicated by the unshaded cells in Chart 1. The other nine seminars (not further described) teach chronic illness care, health economics, and advanced communication skills.
First-Year Resident Fall Seminars
In the initial two first-year resident seminars, we introduce learners to a comprehensive, integrated model for viewing the behavior of health systems from a quality and safety perspective termed the Outside–Inside (OSIS) model (Figure 1). OSIS combines the disciplines of systems thinking and human factor analysis to examine the behavior of systems ranging from the complex organizational or macro-level (systems thinking) to the interaction of individuals with the larger system and individual's own cognitive and perceptive processes (human factors).
The systems thinking and QI seminar provides residents with a method for viewing health care as a set of related processes producing recurring events (and sometimes errors) driven by deeper forces. Residents achieve this understanding by learning to see processes of care as composed of interdependent steps with unique properties and results that emerge from the interactions of those steps and processes. To ground these theoretical ideas in residents' practical experiences, first-year residents analyze the competing demands of patient care and resident education by playing a computer simulation that shows the numbers of residents scheduled and seeing patients, patients receiving care (or walking out if waiting too long), attendings staffing the clinic, and room and nursing staffing numbers. To “win” the simulation, residents make adjustments to see a predetermined number of patients, achieve financial objectives, and receive sufficient teaching from attendings. As they do so, faculty guide residents to discover the organizational and financial forces that create the tension between service and learning, thereby illustrating the application of systems thinking.
The method of human factor analysis examines individuals' actions using knowledge about human perception and cognition applied to the complex and often ambiguous health care environment. To learn about human factors and their relationship to safety and quality in the human factors and cognition seminar, residents view a visual awareness video to experience selective attention and to witness its effect on outcomes.11 Human factors are presented from the perspective of how we view the world and perform “sense making,” that is, how physicians give meaning to perceptions and make decisions. Residents see examples of human-factors-related safety problems, including medical visual illusions, handwriting problems, difficult-to-read IV bag labels, and cognitive decision-making biases drawn from residents' own cases. The seminar concludes with a faculty-led informal root cause analysis (RCA) of a case selected from the residents' own experiences that illustrates human factors concepts. Residents gain a new perspective on how and why mistakes happen, and they are introduced to more sophisticated concepts of causation, QI, and error prevention. This exercise not only teaches human factor and patient safety skills, it also illustrates the explanatory power of SBP and PBLI principles in a real-world context. Related basic QI concepts such as Plan Do Study Act cycles and brainstorming are also described.
Second-Year Resident Fall Seminars
Case: Part B. An RCA of Mr. S.'s case conducted by second-year residents during the patient safety seminar series revealed a number of root causes amenable to intervention. First, low knowledge and high forgetting among residents were identified as important human factors causing underutilization of appropriate DVT prophylaxis. Second, a survey of nurses revealed inconsistent use of ICDs attributable to equipment problems and difficulties with ordering, replacing, and monitoring the equipment, and patient dissatisfaction with the boots. A review of a former resident's data previously collected on use of DVT prophylaxis in medical patients indicated low utilization of DVT prophylaxis and no institutional guidelines for standardization of practice, with resultant underutilization of subcutaneous heparin.
In three fall seminars (Patient Safety, Root Cause Analysis 1, and Root Cause Analysis 2), second-year residents learn how to perform an RCA on an adverse event or near miss, choose an event to analyze, design an intervention to address an identified cause, and develop outcome measures to follow. The first seminar teaches basic safety knowledge and RCA skills, including the principles of causation,12 event mapping,13 and the “Five Whys”14 methodology. Using previously investigated real cases, learners practice RCA methods,13 role-play interviewing participants, brainstorm interventions, and review the strengths and weaknesses of their proposed interventions. They also conceive data-collection methods and outcome measures to evaluate the success or failure of the proposed intervention. “Homework” for the intervening week is to identify a recent inpatient or outpatient event from their own experience that they might choose to investigate.
In the second seminar, second-year residents discuss near-miss or adverse event cases, form teams of two or three, and choose one case per team to investigate. Faculty help learners select cases and plan the investigation. Residents spend the next two weeks in the investigation process, which includes interviews of nurses, pharmacists, or other professionals and often requires site visits to understand the context and the environment. The residents then prepare a detailed RCA, including a proposed intervention and outcome measures.
At the final fall seminar, second-year residents present their RCAs, consisting of detailed event descriptions, causal diagrams, and proposed interventions with outcome measures. During the presentations, the strength and breadth of the causal relationships, the strengths of the proposed interventions, and the next steps necessary for implementation are discussed. In some cases, residents elect to take their cases into implementation. Otherwise, faculty pursue implementation or share the RCAs and proposed solutions with the appropriate operations or quality program staff for action. In each case, a concerted effort is made to “close the loop.”
Second-Year Resident Spring Seminars
The final series of safety and quality seminars occurring in the spring of the second year features two activities to help residents learn additional skills and integrate their newly acquired expertise in the quality and safety aspects of SBP and PBLI. In the SPARC, PBM, and QI seminar, residents are introduced to the SPARC tool,7 a locally developed, population-based medicine (PBM) tool. SPARC is a Web-based software program that provides population-based reports to residents about the patients they care for in their primary care practices, using the University of Virginia's Clinical Data Repository, a health-system-wide data warehouse, and additional clinical or administrative databases. Residents compare their individual data with (1) data for all residents assigned to their same firm/small practice group, (2) data for all residents in the same year of training, and (3) data for all residents and faculty in the general medicine clinic. SPARC shows population-level deidentified data, but identifiers are available as needed for focused chart reviews. Faculty also teach the principles of PBM, and residents are introduced to SPARC by an informatics-trained physician. After initial SPARC familiarization, residents begin their final project for the quality and safety curriculum.
The residents' final quality project, initiated in the same seminar, includes a resident clinic population-level practice review and a detailed analysis of a specific quality problem. For the population-level review, residents use SPARC to compose a cross-sectional demographic snapshot of their ambulatory practice. This requires them to examine and summarize data about their clinic patients' sociodemographic characteristics and their outpatient practice continuity measures. Residents also compare their results for preventive and chronic care measures, and they prepare written summaries of the strengths and weaknesses of their individual practice compared with their resident peers.
For their QI analyses, residents can select any problem, although they are encouraged to select a problem from their SPARC work. Residents create a process map, perform an RCA, identify several candidate improvement interventions, analyze the effort required versus yield obtained from implementing their candidate improvements, select an improvement to implement, identify specific measures of improvement, brainstorm potential implementation barriers, and propose solutions to barriers. Whereas most activities residents perform in this work capitalize on previously learned skills, additional concepts and methods introduced include principles of value stream mapping,15 measuring and identifying common and special cause variation,16 and the concepts of identifying and ameliorating bottlenecks in flow processes.17 Residents have three weeks to complete their analysis projects that are discussed collectively at the final QI report and wrap-up seminar. Reports are four to eight pages in length, follow a specific format, and include resident-prepared process flowcharts. All resident QI project ideas are forwarded to the resident ambulatory clinic QI task force or the head of the patient safety committee for inpatient QI or safety-improvement initiatives.
Curriculum Assessment and Outcomes
To analyze the outcomes of our curriculum, we have used several methods of evaluation that have evolved in parallel with our content. To assess resident skills and curriculum-generated care improvements, we tracked the results of all second-year residents' patient safety projects for academic years 2004–2005 and 2005–2006. Thirty-eight residents completed a total of 27 investigations of 11 near misses and 16 adverse events. There were 10 cases involving hand-off of care, 9 medication cases, 3 lab or X-ray cases, 2 knowledge/training cases, and 3 cases involving administrative processes. Of the 25 unique interventions devised by residents, 15 have been implemented, 6 are in progress, and 4 have not yet been attempted. Overall, 13 of 25 interventions were systems changes, 11 involved information technology, and 1 involved education of residents. Examples of implemented interventions include (1) development of a direct telephone link for physicians to clinic nurses for urgent patient care referrals, (2) practice guideline changes to permit urgent blood transfusions in clinic while patients wait for hospital admission, (3) hospital-wide implementation of guidelines for DVT prevention, using computer prompts at admission, (4) removal of H2O2 from the computer order set as a choice for gastrostomy flushes, and (5) an educational series for first-year medicine residents on reading X-rays, and the use of X-rays in morning reports.
We used qualitative analysis to understand the value to residents of reflecting on and analyzing an adverse event and to assess subsequent attitudinal shifts among residents. Thirteen of 15 eligible residents in 2005, and 14 of 19 residents in 2006, participated in hourlong individual interviews conducted and transcribed by a PhD-qualified qualitative analyst. Residents were asked to thoroughly describe their adverse event investigations, including how they chose their incident, what surprised them about their findings, and how well prepared they felt to conduct the study.
They discussed changes in the ways they thought about or responded to adverse events after this exercise. Residents also talked about the barriers to patient safety at our institution and their sense of self-efficacy in bringing about change. Data were analyzed using a variation of Spradley's domain analysis.18 Residents' responses indicate that the seminars and hands-on experiences were worthwhile to most of them, and the training was successful in shifting residents' thinking about patient safety to a systems-based approach. Residents not only were able to describe the basic concepts and mechanics of systematic safety improvement, but also acknowledged that their role and responsibility extends beyond medical knowledge to include patient safety. One resident stated that he was now able to think and talk about incidents without focusing on blaming an individual, which would “hopefully make things better, hopefully that's the end goal.”
The interviews also revealed a disconnect between residents' ability to discuss patient safety from a systems-based perspective and their ability to participate effectively as change agents. Although residents were articulate and often passionate about the shift away from the “blame and shame” approach to patient safety, they were reluctant to complete quality reports for a variety of reasons, including fear of getting someone in trouble. Residents were surprised to learn just how complicated health systems are, which contributed to their reluctance to initiate change. Given this complexity, residents sometimes felt unable to make change happen because of the perception that they were such small cogs in a large organization or because their tenure at the institution was so short. On the other hand, all residents felt that when they were in their own practices, they would be able to use the tools they learned to identify and implement necessary changes when they had more control over processes of care.
We also surveyed residents as part of our program's comprehensive online assessment of resident activities. Using a separate, confidential evaluation system, residents completed 237 assessments of the seminars' processes and outcomes during a three-year period for the curriculum presented herein. Table 1 presents the process results, indicating that learners felt that seminars generally met predefined learning objectives, were interactive, and were enjoyable. Self-evaluated learner skill attainment is presented in Table 2. Potential self-evaluation bias notwithstanding, these strongly positive responses show that learners believed they gained important skills in all topics prospectively identified by the faculty. Similar gains in content knowledge (not presented) were found. This demonstrates a high degree of resident satisfaction with both seminar process and outcomes.
Discussion and Lessons Learned
At the University of Virginia, we aim to train the next generation of physicians in the skills and basic science of patient safety and QI while creating a learning environment where QI and patient safety are valued and modeled. As a result of our curriculum, residents gain important skills while actively improving our health system processes. Our curriculum development efforts also have produced faculty with special knowledge and skills in SBP and PBLI. These faculty teach other faculty, increasing our effectiveness as role models for residents. The sum of these efforts has created a collaborative spirit among the general medicine faculty that has raised morale and stimulated further collaboration and innovation.
Central to both the learning work of the residents and the learning and teaching work of the faculty is the role of conversations. We use primarily a behavioral educational model to teach specific skills, but we also borrow from both social learning19 and constructivist20 theories in asking learners to work together in small groups to create new knowledge for themselves through a process of investigation and discovery. Our qualitative analysis reveals that these informal conversations are important and powerful learning tools and that these conversations have the ability to begin to reframe the cultural dialogue within residency programs.
In the course of our work, we have learned four lessons that are worth specific comment. We identify these observations as (1) Making it real, (2) Closing the loop, (3) Less is more, and (4) It really does take a village.
Making it real.
It doesn't take much to make patient safety and quality real to the residents. They have experienced error, and they can immediately appreciate the importance of safety if encouraged to reflect on their work experience. Giving residents a window into their own “data” is an effective method of engaging them in quality-of-care conversations. When one resident investigated an event in which H2O2 was inadvertently used as a gastrostomy tube flush instead of H2O, he was able to use his human factors skills to understand how the structure of the order entry menu made this type of error occur. After enlisting our computer systems staff to remove H2O2 from the drop-down menu of routine flushes, he was surprised and gratified that he could make substantial and durable changes to prevent such an event from recurring.
Closing the loop.
Our qualitative evaluation shows that actually implementing change and measuring success is important to trainees, yet this is a labor-intensive process encumbered by residents' and training programs' other demands. Without opportunities to implement their work, residents may feel impotent to make real, systemic changes to prevent harm to their patients. They can become cynical about the likelihood of change, amplifying their disinterest in quality and safety endeavors. If our goal is to engage physicians early in their training to improve quality and safety, closing the loop becomes a crucial step in the curriculum. When the residents' evaluation determined that a clinician-to-clinician phone conversation could have prevented the adverse event they investigated, they proposed the previously noted telephone line dedicated to clinicians for handling urgent referrals. Nine months elapsed before this resident-initiated change occurred, and the residents who had proposed this change were close to finishing their residency. Closing the loop takes time, perseverance, and a quality infrastructure to track the status of projects, but is necessary to show residents that their work has meaning.
Less is more.
Attempting too many projects may lead to failure to implement meaningful change because implementing real-life quality or safety interventions takes substantial time and energy. After the first year of the curriculum, the authors reduced the number of required resident projects by half and were subsequently able to bring a greater absolute number of projects to completion.
It really does take a village.
The importance of connecting to or creating a quality and safety infrastructure cannot be overstated. A curriculum in quality and safety needs an operational home if it is to accomplish real change. With attention and resolve, curricular projects can be connected to the quality infrastructure of the institution, and local infrastructure can be created in educational settings such as resident clinics. Access to operations-side change agents and decision makers is crucial to implement curricular projects, and access to clinical data is important to plan improvements and to evaluate project successes. These connections can benefit both the institution and the training program because real improvement in patient outcomes is the mutual goal of both entities.
Case: Part C.The DVT prophylaxis case illustrates the collaborative effort required to make such significant, institution-wide changes. The interventions proposed by the resident team were (1) to develop institutional guidelines for DVT prophylaxis, (2) to implement guidelines using computer prompts at admission with imbedded criteria to guide choices for ordering physicians, and (3) to educate residents in all residency programs housewide about DVT prophylaxis. Residents worked with the senior associate dean for clinical services and the information systems staff to finalize the guideline and to design computer prompts. They also performed brief educational interventions for medical and surgical residency programs. Pre- and postguideline outcomes measures established were % DVT prophylaxis in eligible medical and surgical patients, and DVT, PE, stroke, and significant bleeding complications pre- and postguideline implementation. Hospital performance improvement staff were enlisted to help with data collection.
Guidelines were implemented approximately six months after the completion of the RCA, and analysis of the data one year after implementation is currently under way. The resident who collected the initial data is now in primary care practice locally. The residents who performed the original RCA in their second year are now finishing their residency. The lead resident on this project also used this work as her third-year resident research study21and has enlisted rising third residents to continue these efforts.
New first-year residents may not know that the DVT guidelines were the result of resident-directed QI work, but when this is pointed out, it becomes a useful story in resident empowerment to improve quality and safety.
With Title VII and local institutional support, we are revising our program of evaluation for the SBP and PBLI competencies to include three new instruments. The first tool is a 33-item prepost test that measures residents' knowledge and attitudes about curricular topics in SBP and PBLI and that is administered at the beginning, midpoint, and end of the curriculum. The second instrument is a series of short-answer cases based on the work of Ogrinc,22 which we have modified for use with cases specifically directed toward quality or safety issues. A third assessment method uses a portfolio to collect assessments of all available information described herein to derive a more global assessment of each individual's competency in SBP and PBLI. All instruments are undergoing validity testing.
It is not likely that any amount of small-group work, Web-based exercises, or design of QI projects divorced from actual implementation can supplant the learning derived from working on a multidisciplinary team planning and implementing real-time change. Our curriculum tilts toward the former position, largely because of large-scale forces operating at institutional and departmental levels. To offer residents additional opportunities to participate in hands-on QI activities, we are now offering third-year residents the chance to participate in institutional QI initiatives or to carry out faculty-coached QI work as their third-year scholarly project. Resident participation in this type of additional QI work is voluntary.
Although our work has been received well by our residents, faculty, and administration, substantial challenges remain. Topic areas need periodic revisions to improve educational effectiveness and to incorporate new thinking. Finding the resources to support teaching efforts remains problematic in a financially driven culture, and time within the resident educational calendar remains at a premium, given the ACGME workweek mandates.
Improving patient safety and quality in health care is one of the most pressing issues facing medicine today. We are in the midst of dialogues about how to meet that challenge. Faculty are learning the new language of systems of care, human factors, teamwork, and evidence-based and outcomes-based care. Although educators of all specialties will need to develop solutions that fit their residents' needs, we suggest that, given their positions and interests, academic generalists play unique and useful roles teaching quality and safety. With the support of Title VII, academic medicine can be the catalyst for change in the profession, engaging the next generation of physicians in this new dialogue, a conversation which leads to safer and better-quality care for every patient.
This work was supported by:
* Health Resources and Services Administration Academic Administrative Unit Grant: 09/01/00–08/31/06 (# D54 HP000040);
* Academic Administrative Unit Grant: 09/01/05–08/31/06 (SPARC Supplement) (# D54 HP000040);
* Faculty Development Grant: 7/01/03–06/30/06 (D55 HP00235); and
* The Robert Wood Johnson Foundation-funded Partnerships for Quality Education “Take Care to Learn” and “Achieving Competency Today” Grants.