In 2006, the Residency Review Committee in Internal Medicine of the Accreditation Council on Graduate Medical Education (ACGME) brought to fruition what had been planned for a number of years: its Education Innovation Project (EIP), an alternative pathway to accreditation for high-performing programs that would be based on educational and clinical outcome measures, the centrality of patient safety and quality of care, and creativity rather than on the more process-based accreditation standards most programs must follow.1–4 The committee invited all programs with outstanding recent accreditation history (at least eight years for the most recent two cycles) to submit proposals. Ultimately, 17 programs were chosen in the first round, including our program, and four more were chosen in the second round of the EIP.5 In this article, we report the first 18 months (July 2006 to January 2007) of our program’s experience as the Beth Israel Deaconess Internal Medicine Residency EIP Project.
Many organizations have called for an increase in outcomes-oriented education in residencies, education that will be relevant to current and future practice environments6–8; however, little has been written about how to provide such education.9–12 Traditionally, residents work as passive followers, rather than active contributors, to institutional quality improvement (QI) initiatives. Residents are told what guidelines to follow, instructed in new systems of care, and taught what outcomes are being measured. They are not asked whether the guidelines are relevant to daily care, what new systems are necessary, or what outcomes mean the most to their patients—even though the residents are often the physicians most involved with the day-to-day processes directly contributing to the quality of care on the inpatient services. It is our contention that such passive resident involvement not only compromises institutional efforts to improve quality of care but also ignores a central responsibility to train and produce physicians who are prepared to contribute to (and lead) health care quality initiatives in their future practices.
Ironically, health care system experts and QI analysts have noted that physicians are often perceived as a central barrier to improvements in health care systems13,14; this may in part be attributable to physicians’ lack of expertise in systems-based performance and error analysis, lack of awareness of systems-based outcomes, and attitudinal biases that may result in generalized resistance to systems-based change. The Institutes of Medicine has called for training of physicians and physician leaders in health care improvement to help overcome some of these obstacles.15,16
Thus, in addition to training our residents how to deliver outstanding care to individual patients, how to effectively teach colleagues and students, and how to conduct and analyze scholarly investigation, through the EIP we now aspire to train our residents to become health care system leaders, armed with the requisite knowledge base, skill set, and proactive attitudes to become agents of effective change in whatever health care system they enter on graduation.
Beth Israel Deaconess Medical Center (BIDMC) is a 600-bed tertiary medical center in Boston and is one of the three major teaching affiliates of Harvard Medical School. Both before and after the EIP innovations, the internal medicine residency program has been led by a program director and five associate program directors; each of the 158 residents is assigned to one of four medical firms. The firms have traditionally been educational and advising units; one quarter of the residents are assigned to each firm to work longitudinally with top faculty both clinically and for advising. Throughout three years of training, residents spend approximately 12 months rotating on the general medical service.
Our program’s EIP combines two major interventions designed to synergistically enhance residents’ knowledge, skills, attitudes, and participation in health care system design and QI:
- A large-scale reorganization of the clinical care delivery model and inpatient medical teaching service into geographically localized clinical microsystems to effectively integrate QI and patient safety concepts into the daily clinical practice of all residents.
- A focused educational intervention designed to impart a foundational understanding of QI and patient safety to all residents.
Creation of geographic unit-based microsystems.
Before our EIP project, patients were assigned to resident teams on the basis of a traditional rotating call cycle, without regard to geographic bed assignment or firm. As a result, teams on the general medical service routinely cared for patients admitted to any of up to eight inpatient floors in addition to patients in the emergency department awaiting beds. This delocalized strategy of patient–doctor assignment created significant barriers to residents’ ability to effectively communicate and collaborate with nursing staff, case managers, therapists, and patients because residents could not be considered a reliable presence on any single nursing unit. Longitudinal relationships between physicians and unit-based multidisciplinary staff were difficult to maintain. Moreover, the diffuse clinical experience frustrated any attempt to provide residents feedback regarding practice-related performance measures because available data for outcomes were not correlated with any specific subgroups of residents.
For the EIP, we reconfigured our inpatient medical service to function around geographically based units, or microsystems of care, in which residents, nurses, and staff could function in integrated teams physically colocated around shared patients.17 In this new model, residents are assigned to specific nursing units and care for any patients admitted to their unit. This is a patient-centered approach to house officer assignment (rather than a house officer call-cycle-centered approach to patient assignment), as the physician now becomes an integral component of the multidisciplinary team that forms around the patient on arrival to the inpatient unit. Bed availability determines patient assignment; each day, there is one long-call and one short-call team on each unit.
In our new model, each firm is now permanently assigned to a specific bed population (unit), and each time a resident rotates on the inpatient general medicine service, he or she works on the same geographic unit. Teams are formed between the unit’s multidisciplinary staff and the residents, allowing meaningful participation of residents in the ownership of care on the unit and in the governance of the unit. In addition to these firm-based units, we also have created geographically distinct homes for teams caring for solid organ transplant patients, bone marrow transplant patients, cardiology patients, and oncology patients.
We have worked with the medical center’s advanced information systems group to facilitate assessment of process and outcomes data at the unit level. Each unit now has its own “dashboards” of process and outcome measures, reflecting data only about those patients cared for on that unit and by that firm’s residents. For example, rather than reflecting what the hospital’s central line infection rate is (which can result in the “not true for my patients” response), our residents are able to reflect on the relevant quality indicators (central line infection rate, in this example) for their patients. Examples of metrics of outcomes include patient satisfaction rates, cardiac arrest rate by unit, number of falls, and number of decubitus ulcers. We anticipated that this approach would improve residents’ sense of ownership of quality data and commitment to systems improvement.
Colocalizing residents and staff around patients on a given unit has allowed us to augment teamwork on the unit. In keeping with principles of team training, we instituted unit-based daily “board rounds” with participation of staff from all disciplines. At board rounds, the floor’s residents and nurses “huddle” in a discussion in which the focus is mainly on quality metrics and care plans for every patient, every day, in addition to raising situational awareness of critically ill or at-risk patients on the unit. Corresponding development of real-time, unit-based electronic dashboards has allowed the residents to review their own patient care and make changes at the point of service (see Chart 1).
Finally, each unit formed a governance board with representation from all disciplines (including but not limited to nursing, case management, social work, physical therapy, resident and faculty physician staff, and patients). This board reviews unit-specific data and helps to set and maintain the longitudinal QI agenda for the unit.
Creation of experiential QI curriculum.
At BIDMC, we had previously instituted a successful elective in QI; during that three-week elective block, residents actively participated in the real-time evaluation of medical errors, performed root-cause analyses, and worked through a structured curriculum with a health-care-quality nurse and a faculty QI expert.18 When restructuring the residency program to empower residents to participate in and improve care on their units, we felt it necessary that all residents achieve the basic skills offered in the QI elective. We therefore chose to mandate and expand the rotation; each of our second-year residents (46–49 residents annually) now completes the QI rotation and generates a patient safety portfolio consisting of an adverse event review and a detailed QI project (previously, 8–14 residents rotated per year). To prepare residents for this work, in their first year, they complete an online module reviewing the basics of system theory and adverse event review.
The expansion of the educational program and increase in number of performance- improvement projects initiated by residents required more expert faculty time for mentoring. We therefore created a funded 1.0 FTE of patient safety core faculty (PSCF) time; rather than fund one faculty member, we chose to partially fund seven faculty members. Representing various medical specialties and levels of training, each PSCF member mentors three groups of residents annually during block rotations, participates in regular monthly organizational meetings, and attends and directs conferences for the residents. Because half of the core faculty had no prior training in QI, we initiated a training program delivered at the start of each year. The faculty with QI experience led workshops on root-cause analysis, performance improvement, and our institutional approach to QI; to standardize the teaching approach, one of us (A.V.T.) mentored all new faculty for their first teaching block.
Residents choose QI projects for focus during the elective, with their own unit’s priorities in mind; they often seek the input of their unit’s medical director or governance board. They work closely with their assigned monthly PSCF member and with up to three other residents to design and implement their chosen project.
Residents’ work is integrated with the institution’s health care quality department in two important ways. When possible, residents’ projects are aligned with ongoing initiatives and existing data sets. In addition, all adverse event review is done with mentorship from the hospital’s case reviewers. Residents complete the required documentation and present their analyses to the departmental peer review committee.
To create a forum for residents to share their projects and ideas and to learn from the work of their colleagues, we instituted a monthly quality and outcomes conference. At this conference, PSCF teach basic QI principles, and residents present their projects and interesting adverse event review findings to their peers. Nurses are invited to attend. Residents present their work, and medical directors share unit-based outcomes including core measure and patient satisfaction data and generate discussion of key themes.
We made these changes to our program in July 2006 at the start of the EIP launch. We speculated that these changes would
- increase total numbers of residents participating actively in QI projects,
- improve house officer engagement in the QI work of the hospital,
- enhance the culture of safety the residents perceive in their training environment,
- improve overall work flow on the general medicine ward rotations, and
- improve the overall experience for the residents on ward rotations.
Survey of residents’ attitudes.
Before our EIP changes, our residents were already working in a system in which patient safety was a priority. Even so, many of our residents were initially skeptical of the proposed work-flow changes necessary for our EIP; thus, one resident planned and undertook a survey to assess residents’ attitudes before and after the start of the EIP. The 23-question survey tool (see Appendix 1) was designed after reviewing existing resources and the ACGME competencies. It was modified after input from a trial group of residents, to assess residents’ overall satisfaction (with education, clinical care, and ACGME competency assessment). Residents were asked to show agreement or disagreement using a four-point Likert-like scale. The anonymous survey, which was deemed IRB exempt by the BIDMC review board, was delivered to all residents one month before the geographic admitting began (May 2006) and again one year later (May 2007). We compared responses before and after intervention across all four points of the Likert scale using the Wilcoxon rank sum test.
After each rotation, residents complete a confidential online evaluation assessing various aspects of each rotation. As required by the ACGME and to continuously improve our program, the residency and department leadership routinely review evaluations from all internal medicine rotations. Questions are in a four-point Likert-like scale anchored as 1 = poor, 2 = fair, 3 = good, and 4 = excellent. We compared these responses for the inpatient general medicine services from the 2005–2006 academic year (before the intervention) with those of the 2006–2007 academic year (after intervention) using the Wilcoxon rank sum test.
Outcomes of the Program
Patient safety education
In the first 18 months of our EIP project, 74 residents completed the QI rotation, representing a 300% increase in enrollment. All completed an adverse event review. Residents now review more than two thirds of all cases presented at the monthly departmental QI committee meeting compared with 10% before the EIP project. Near-misses and adverse events reviewed include procedural complications, medication errors, communication failures, delays in diagnosis, and handoff issues.
During the first 18 months, 24 groups of three residents worked on QI projects of their own design. Residents are taught the Plan–Do–Study–Act cycle and asked to pick specific quality measures for assessment during their three-week rotation. Some groups are able to complete only the Plan and Do portions; then, the project is handed to a subsequent rotation group to complete. Many projects align with institutional goals. See List 1 for a sample of current projects.
Project results and adverse-event review findings are presented by residents at the monthly quality and outcomes conference. See List 2 for an example of topics discussed.
Survey of residents’ attitudes
We collected anonymous responses from 124/158 (78%) of the residents before the intervention (55, 35, and 34 residents from the first, second, and third years, respectively) and from 79% of the residents one year later (52, 34, and 39 residents). A summary of the responses is shown in Table 1. Tests of significance were calculated on the distribution of responses across the four-point Likert-like scale; we show only change in the combined categories of “strongly agree” and “agree” for simplicity.
Culture of safety in residency.
The significant increase in the number of residents who responded “agree” or “strongly agree” on the culture of safety items (see Table 1) reflects a stronger culture of safety: role in designing and implementing change, opinions being sought, and level of comfort in reporting error. In addition, residents felt the admitting system no longer interfered with effective communication. Patient safety was a priority for residents before the intervention, with high rates of agreement with its value as an educational topic and its relevance to the care delivered by residents. These high ratings did not change after the intervention; residents remained interested in learning even more about the area.
Residents’ perception of educational goals.
In addition to assessing the safety culture, we asked residents to show their agreement or disagreement with whether our program prepares residents to meet ACGME competency descriptions. We noted little change in the numbers of residents that agreed or strongly agreed with ACGME competency language to describe their medical knowledge, patient care, practice-based learning and improvement, interpersonal skills, professionalism, and systems-based practice. However, all of these questions had high rates of agreement at baseline.
Residents’ assessment of inpatient general medicine rotation
In academic year 2005–2006, residents completed 188 of a possible 238 confidential, blinded online evaluations of the inpatient general medicine service (79% completion rate); in academic year 2006–2007, the completion rate was 74% (177 of 238). As reflected in Table 2, after the initiation of the EIP changes outlined above, our residents reported significant improvement in the quantity of teaching, patient load, and level of autonomy as well as in the overall value of their inpatient general medicine rotations on their monthly ward rotation evaluations.
Reflections on the First 18 Months
On a short timeline, we have brought about tremendous change on our inpatient general medical teaching service and achieved many of our goals. We trained and funded an expert QI faculty; we changed our schedule to allow a rotation centered on active QI work. We completely changed our admitting system, creating geographical multidisciplinary teams with the patient at the center. New clinical microsystems exist with shared rounds, unit-specific quality dashboards, and a charge to the teams to assess and improve care around the unit’s own priorities.
We have been able to increase total numbers of residents participating actively in QI projects and improve house officer engagement in the QI work of the hospital. We have learned that, given a structured opportunity, residents effectively (and enthusiastically) engage in QI work and choose projects that align with institutional priorities. We have not yet saturated them with QI learning, either; residents continued to note in the resident survey that they would like to learn more. Survey results show enhancement of the culture of safety in the training environment and improvement in the overall experience for the residents on ward rotations.
We were gratified to see that despite initial resistance to change, residents perceived the education on the inpatient general medical services to have improved. Residents perceived both an increase in teaching quantity and a decrease in patient load despite no change in the teaching schedule and no significant change in total team census. It is quite likely that many of the improvements the residents perceived were attributable to geographic admitting (less walking from floor to floor, fewer pages to answer) and to a more balanced admission schedule (fewer excessive peaks and fewer valleys in team census than in the traditional call cycle system). Though not directly measured, these changes likely allowed for more efficient rounds and facilitated an efficient work flow for physicians. This efficiency then allowed better attendance at bedside rounds and formal teaching rounds and greater opportunities for discussion of patient care. These improvements in teaching could have generated the greater sense of direct patient involvement and autonomy reported in the monthly evaluations.
Though we speculate that resident work flow improved, it is apparent that nursing and physician work remains mostly “in parallel” rather than improved via systematic coordination of multidisciplinary processes. To achieve our initial goal of overall improved work flow on the unit, a fundamental and coordinated redesign of all work flow may now be necessary.
We accomplished these outcomes with few resources; our work should be transportable to other programs across a spectrum of medical centers. We required 1.0 FTE support for the PSCF, our information system enhancements were completed within existing staff capacity, and the EIP requirement for increased program director salary support was not necessary because our program director was already funded at the required level. Substantial unmeasured resources required were the attention and energy needed to produce such global change; our group of program leaders has diverted much time and energy to this project, and other initiatives that may be equally or even more valuable have been deferred.
Importantly, BIDMC’s preexisting and now public commitment to QI and patient safety served as a crucial platform on which to construct our proposed changes, ensuring the endorsement and active engagement of the institution’s clinical, administrative, and health care quality leadership throughout this initiative.19
We learned many lessons during our project implementation. Although we have successfully established primary medical units on which the geographic microsystem structure is built, there continues to be a minority of patients assigned as “boarder” medical inpatients on other units because of census overflow. Based on internal tracking of census listings by resident teams, it is estimated that approximately 20% to 25% of medical inpatients on the teaching service were assigned as boarders in 2006–2007. In addition, preservation of geographic integrity leads to discontinuity for the few patients who are moved to the other primary unit because of the need for isolation or other clinical issues. In an effort to reduce the proportion of boarder patients and patients who are moved, one of the primary units was relocated to a geographic floor with a larger number of inpatient and private beds in January 2008. Corresponding nursing staffs were relocated along with residents to preserve the established unit-based multidisciplinary teams. Minimizing the proportion of boarder patients to preserve geographic localization of our multidisciplinary teams remains an ongoing operational effort in times of high admitting volume and high inpatient occupancy rates.
To care for emergency room patients who have long waits for inpatient beds, we created the Medicine–Emergency Room In Transition (MERIT) rotation. The resident assigned to this rotation is responsible for managing acute issues that may arise until a bed is assigned. To preserve the majority of primary work-ups for the admitting teams, the MERIT resident assumes care only when a patient has been waiting three hours and then writes only a minimal note and admitting orders, leaving the majority of the admission work-up to the team that eventually will be involved. Though we did not measure direct impact on learning, our survey results suggest that the introduction of a MERIT resident has not decreased the residents’ perception of their level of autonomy.
It has been challenging to support longitudinal improvement initiatives in the context of a relatively short (three-week) rotation-based resident experience in QI and patient safety. Many of the projects initiated during the first year of this program remain in early phases of the PDSA cycle; maintaining project continuity as resident workgroups turn over has been difficult and requires faculty to claim ownership of specific QI initiatives to keep partially completed project streams active.
Creation of a QI-focused curriculum did require changes in our core curriculum including substitution of core ambulatory lectures with the quality and outcomes conferences. We were, however, able to allow residents to participate in QI projects without sacrificing time allotted for traditional research projects.
Not unexpectedly, a significant hurdle (especially in the early phases of the EIP intervention) was resistance to change amongst residents. Strategies employed to mitigate this resistance included (1) presentation of a compelling need for change by demonstrating existing educational and clinical care inefficiencies/deficiencies with the pre-EIP model, (2) early and explicit engagement of visible skeptics to become part of the change process and evaluation, and (3) addressing of unanticipated operational hurdles using real-time problem solving with residents, especially in the early phases of implementation. As reflected in the reported resident survey data, residents were able to transition beyond early resistance to ultimately embrace both the goals and the operational structure of the EIP-related interventions.
This project shows that fundamental structural change at a large academic center can be undertaken with few resources on a short timeline and can result in substantial improvements in work flow, QI work by residents, and teaching and education. Although our work has been in a single, large academic institution with a proven commitment to QI, we believe this work is transportable to other, different institutions because of the simplicity of its design and the limited resources we have used to date.
The ultimate tests of our EIP system might be whether patient care outcomes have improved and whether we are creating better-prepared doctors at the end of training. Now that we believe our interventions improve residency education, our next steps involve the further collection of outcome data to assess the impact of the interventions on residency outcomes after graduation and on patient-related outcomes like morbidity, length of stay, readmission rates, and compliance with Health Care Effectiveness Data and Information Set measures, and expansion of our work on microsystems to the outpatient setting.
The authors thank Shruti Mulki for her assistance with data collection and analysis and Roger B. Davis, ScD, for statistical analysis. They also thank Dr. Mark Zeidel, chairman of medicine, Dr. Gordon Strewler, vice chairman for education, and Dr. Mark Aronson, vice chairman for quality, for their support of the EIP project. The authors also thank their medical center’s information systems programmers for their work on the electronic dashboard, and the patient safety core faculty for their commitment to teaching. Finally, the authors thank the internal medicine residents at BIDMC for their commitment to patient care, their flexibility, and their pursuit of excellence.
Survey data and findings given in this article were presented in poster format at the Association of Program Directors in Internal Medicine Annual Spring Meetings in 2006, 2007, and 2008.