Purpose: This study assesses the effects of a resident teaching service restructuring on resident, student, and patient outcomes.
Method: Interventions included eliminating a “day float” admitting team, converting one-resident:one-intern teams to one-resident:two-intern teams, reducing patient caps from 11 to 7 patients per intern, and increasing pairing between resident teams and attendings. Resident end-of-rotation evaluations and time spent in categorized activities; student end-of-clerkship evaluations, patient logs, and subject exam scores; and hospital-collected patient outcome data were compared before (2007–2008) versus after (2008–2009) the changes.
Results: Interns covered fewer patients per day post intervention (9.9 apiece to 6.3 apiece), whereas the total number of patients covered increased (2,501 to 2,916). Enjoyment of the rotation was higher post intervention for interns and senior residents. Residents' time in direct patient care activities and with interns increased post intervention, but residents spent less time with medical students.
Students' ratings of several aspects of the clerkship were significantly higher in the postintervention year. Students evaluated more previously unevaluated patients post intervention (32.6% to 45.8%, P < .001), but subject exam scores were unchanged.
The median length of stay decreased post intervention (5.0 to 4.0 days, P = .02), and fewer patients required ICU care (11.2% to 7.9%, P < .001). These differences persisted after adjusting for multiple covariates.
Conclusions: An intervention that reduced handoffs and intern patient census and that increased hospitalist pairing was associated with improved resident and student experiences, a favorable impact on patient outcomes, and probable cost savings.
Dr. O'Connor is associate professor and associate director of the internal medicine residency program, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Lang is associate professor and director of the internal medicine clerkship and subinternship, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Bordley is professor, director of the internal medicine residency program, and associate chair for education, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Correspondence should be addressed to Dr. O'Connor, Box MED/HMD, Strong Memorial Hospital, 601 Elmwood Ave., Rochester, NY 14642; telephone: (585) 275-4912; fax: (585) 276-2144; e-mail: firstname.lastname@example.org.
First published online October 25, 2011
Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A66.
The pace, complexity, and intensity of inpatient care have increased over the past two decades.1–5 Teaching hospitals have struggled to adapt inpatient teaching services to emphasize education within this environment.1,5 Without a reduction in workload (e.g., number of patients), duty hours limits for trainees may result in “compression” of too much work into too little time, which has been associated with increased stress, fatigue, burnout, suboptimal patient care, and decreased attendance at educational conferences.4–15 Duty hours limits, intended to decrease trainee fatigue and increase patient safety, have also increased the number of patient handoffs, which compromise education and, potentially, patient outcomes.4,9,10,12,16,17
A recent study demonstrated improvement in residents' satisfaction and time spent in educational activities by reducing the number of patients per resident, increasing the availability and quality of teaching attendings, and improving other aspects of the learning environment (e.g., call frequency),18 but these interventions required substantial funding to accomplish.15 Cost-neutral methods to improve residents' learning environment are needed.
To try to address these concerns, we restructured the internal medicine service at Strong Memorial Hospital (SMH) and tracked the effects of the restructuring on trainees and patients. We hypothesized that restructuring the teaching service would improve residents' and students' experiences and could improve patient outcomes.
Rationale for service structure changes
Feedback from our residents and students suggested that they were accepting many patients after the bulk of the diagnostic and treatment decisions had been made. Interns' workload interfered with their ability to focus on learning and to reflect on patient interactions; many complained that educational conferences interfered with their ability to complete their work. Interns struggled to complete traditional intern tasks, such as writing notes and initial orders, completing discharge paperwork, and scheduling follow-up appointments; senior residents performed these tasks with increasing frequency to help their interns. In addition, multiple attendings often shared the teams' patients, and many considered the attending rounds' structure disruptive to work and of relatively low educational yield.
Changes to the teaching service structure for the 2008–2009 academic year
During the 2007–2008 academic year, the medicine resident teaching service structure consisted of five teams of one resident and one intern, and an emergency department (ED)-based day float “transition team,” which consisted of two interns and two residents with overlapping day and evening shifts (Figure 1). The transition team admitted medicine patients in the ED to be handed off to resident or midlevel-provider teams when the patients were transferred to inpatient units. The intention of this structure was to maximize the clustering of a team's patients onto a single patient care unit within the hospital. Unfortunately, resident team patients continued to be spread over multiple patient units because of hospital overcrowding, and long ED boarding times meant that floor teams routinely accepted patients well into their diagnostic and treatment trajectories.
For 2008–2009, we eliminated the transition team to increase the number of fresh admissions evaluated by the teams and reduce the number of handoffs within the service (Figure 2). We changed four of the one-intern teams to two-intern teams (we did not have enough interns to make all five of the teams two-intern teams), as shown in Figure 1. The interns' patient cap was reduced from 11 to 7 patients per intern for the two-intern teams and to 10 patients for the remaining one-intern team. The call cycle, in which teams admit patients most days of the week if they remain below their patient cap, remained unchanged.
In the preintervention year, assignment of patients to resident versus nonteaching teams was not based on the identity of the attending physician, so each resident team typically consisted of a mixture of patients assigned to hospitalists, community-based primary care providers, and subspecialty attendings. Teams could also have patients managed by more than one hospitalist at the same time.
In 2008–2009, we began emphasizing the assignment of hospitalist patients to the resident teams. In addition, whenever a hospitalist service patient was assigned to a resident team, the team's specific hospitalist became the patient's attending in order to rigidly pair each team with a single hospitalist. These changes were intended to maximize the number of patients a team shared with a single attending to improve team efficiency and to increase the opportunity for patient-based teaching.
In the preintervention year, attending rounds consisted of (1) one-hour work rounds with the team's usual hospitalist three days per week and (2) twice-weekly, 75- to 90-minute case conferences with two to three resident teams and an attending not otherwise linked with the teams or patients. In 2008–2009, attending rounds became daily, 90-minute combined work and teaching rounds with the team's hospitalist, emphasizing bedside teaching whenever possible.
Two minor changes to the medicine clerkship occurred during the study period. First, more teams were assigned two students (from 18.3% of teams before the intervention to 30% post intervention). Second, a series of online virtual patient cases19 based on the national clerkship curriculum20 were gradually introduced during the postintervention year, with a concomitant reduction in the resident-provided lectures on the corresponding topics.
The nonteaching service structure, consisting of nurse practitioners and physician assistants working with a variety of attendings as described elsewhere,21 was unchanged.
This study was conducted within SMH, the 750-bed, tertiary care teaching hospital for the University of Rochester. During the study period of academic years 2007–2008 and 2008–2009, there were 70 medicine and 32 medicine–pediatrics residents per year. This study was approved by the University of Rochester research subjects review board.
Resident end-of-rotation evaluations.
We collected data from the normal end-of-rotation evaluations for the two academic years included in the study. An automated evaluation system (www.e-value.net) managed the end-of-rotation evaluations, which consisted of 7 to 15 questions (depending on the year) scored on a Likert scale (1 = strongly disagree, 5 = strongly agree). We analyzed all questions that overlapped between the two academic years (2007–2008 versus 2008–2009). The evaluation questions for the transition team differed from those for the floor teams, so the transition team evaluation responses were not included in the analysis.
Quantifying how residents spend their time.
We used the previously described “random paging” method22 to estimate the proportion of time that interns and residents spent in prespecified activities and with specific people. During a two-month block in each study year (during spring of 2008 and spring of 2009), the program office paged residents at random times with a code page prompting them to complete a sheet in a random paging log created for this study. The log sheets required residents to check boxes that best described (1) their activity at the time of the page (e.g., in educational conference, rounding with team, writing notes, completing discharge paperwork) and (2) each person they were with at the time of the page (e.g., with patient, resident, student, attending). Random times for pages were determined with a random number generator and occurred four to six times a day, between 7 am and 5 pm, Monday through Friday.
For analysis, we grouped activities into categories: direct patient care (e.g., examining a patient or discussing a patient on work rounds), indirect patient care (e.g., writing notes or completing discharge paperwork), or teaching/ learning (e.g., teaching on wards or attending an educational conference).18,22 We excluded teaching conference data when analyzing who residents were with at the time of the pages.
Students who completed a four-week rotation (of their eight-week internal medicine clerkship) on the general medical floors at SMH were included in the analysis. All students are required to complete a 33-question end-of-clerkship electronic evaluation; we compared responses from academic years 2007–2008 and 2008–2009 to the questions that were most likely to be affected by the new resident team structure, such as evaluations of rounds and the teaching they received on the wards (as opposed to other teaching sessions that were unchanged). We reviewed required logs summarizing the number of inpatients cared for, including whether the patients were received in handoff (from night float, transition team, ICU transfer, or another member of the team) or were “fresh” (previously unevaluated by an admitting or night float team). We also compared students' National Board of Medical Examiners (NBME) medicine subject examination scores from the two study years. We were unable to adjust the subject exam scores for individual students' United States Medical Licensing Exam (USMLE) scores,23 but the mean USMLE scores did not differ between the years.
Patients' team assignment (defined as the team managing a patient at discharge) is not collected by the hospital, but it is stored in a locally developed sign-out program. We linked team assignment in the sign-out program to existing hospital data collection so we could compare patient outcomes for different teams.21 Patients discharged from the transition team were combined with those from the floor teams to study the “resident patients” of the preintervention year. We used hospital-coded, diagnosis-related groups and a publicly available macro24 to calculate Charlson comorbidity index (CCI) scores, which estimate patient morbidity burden and mortality rate.25
We performed all statistical analyses using SAS version 9.2 (Cary, North Carolina). We considered P < .05 to be significant for those outcomes we hypothesized to be improved by the intervention, including overall rotation evaluation responses and random paging data assessing time spent in learning versus direct versus indirect patient care; other unadjusted P values between .01 and .05 should be treated with greater skepticism. Two-sided Wilcoxon rank-sum test and Fisher exact tests were used for the hypothesis testing of medians and proportions, respectively. Unpaired, two-sided t tests were used to compare the means of numeric evaluation summaries.
Regression analyses were performed using the PROC LOGISTIC function in SAS. We included the following variables in the regression models: age, sex, Medicare status, Medicaid status, attending, and CCI. We classified attendings into one of four categories: hospitalist, primary care physician, subspecialist, and a single, high-volume hospitalist who does not serve as a team teaching attending but who specializes in institutionalized, complex patients (these patients tended to have very long stays and high mortality rates, so they were separated from the other groups). For the regression analyses of length of stay (LOS), we stratified LOS into quartiles. We also included a disposition classification (home versus skilled nursing facility versus deceased versus other) in the LOS models. In a sensitivity analysis, we explored replacing the CCI scores with the hospital-generated severity of illness score (a 1–4 scale) in the models and found similar results.
The total number of patients discharged from the resident service (including the transition team) increased after the intervention (2,501 to 2,916—a 16.6% increase). The mean number of patients seen by interns (not including the transition team) fell from 9.9 per day before the intervention to 6.3 per day in year two (36.4% decrease); the number of patients supervising residents saw daily increased from 9.9 to 11.4 in year two (15.2% increase). With the expansion of the teaching service capacity, the nonteaching service staffing decreased from an average of nine nurse practitioners or physician assistants to eight per day after the intervention.
One hundred sixty-nine of 181 (93%) end-of-rotation evaluations were completed in the 2007–2008 year, and 224 of 265 (85%) were completed in 2008–2009. We found similar numeric ratings by the floor teams except that, post intervention, there were higher scores rating overall rotation enjoyment by interns (3.91 to 4.26, P < .001) and senior residents (3.85 to 4.04, P = .04) (Table 1). In year one, 15 of 21 (71.4% [95% CI 49.8%–86.4%]) random paging logs were returned, whereas in year two, 22 of 42 (52.4% [95% CI 37.7%–66.6%]) were returned. Residents spent substantially more time in direct patient care activities post intervention (from 34 of 149 pages [22.8%] to 93 of 244 pages [38.1%], P = .002), and there was a nonsignificant increase in time in teaching and learning activities for interns (from 33 of 324 pages [10.2%] to 48 of 340 pages [14.1%], P = .13) (Table 1). Residents spent more time with interns (from 42 of 123 pages [34.2%] to 92 of 203 pages [45.3%], P = .049) and, possibly, with attendings post intervention (from 10 of 123 pages [8.1%] to 31 of 203 pages [15.3%], P = .08). Interns and residents both spent very little time with their patients in both years (range 4.9%–8.4%) and less time with medical students post intervention (Table 1).
All students completed their required end-of-clerkship evaluations and patient logs (66 of 66 in 2007–2008 and 71 of 71 in 2008–2009), and all but one took the medicine subject exam. Students rated several aspects of the portion of their clerkship spent at SMH more highly post intervention, including rounds with the ward attending (3.11 to 3.49, P < .001, on a four-point scale) and the accessibility of teachers/faculty (4.15 to 4.44, P = .02, on a five-point scale); ratings of the clerkship as a whole improved (4.05 to 4.30 on a five-point scale), but this finding did not achieve statistical significance (P = .052). The number of patients who were not previously evaluated by medicine residents increased from 573 of 1,756 (32.6%) to 1,206 of 2,632 (45.8%) post intervention (P < .001). Students' medicine subject exam scores were similar from year one to year two (77.5 to 76.1, P = .33). Student evaluation and patient data are detailed in Supplemental Digital Table 1, available at http://links.lww.com/ACADMED/A66.
The number of hospitalist-attended resident patients increased from 1,296 of 2,501 (51.8%) in year one to 1,938 of 2,916 (66.5%) in year two (P < .001). Resident patients were less likely to have an ICU component to their hospitalization after the intervention (from 281 of 2,501 [11.2%] to 231 of 2,685 [7.9%], P < .001). Resident patients' unadjusted LOS decreased from a median of 5.0 (interquartile range [IQR] 2.0–9.0) to 4.0 (IQR 2.0–8.0) days (P = .02), and this difference persisted after adjusting for demographic and clinical variables (Table 2). There were no differences in the unadjusted or adjusted rates of readmission or mortality.
Entire medical service.
We analyzed patient outcomes for the entire medical service (combined teaching and nonteaching services) to exclude the possibility that differences in resident patient outcomes were due to changes in the allocation of patients between the teaching and nonteaching services. The number of patients with a hospitalist attending increased from 3,652 of 7,084 (51.6%) to 4,771 of 7,532 (63.3%) (P < .001), but CCI was unchanged (Table 2). Fewer patients spent time in the ICU during year two, even after adjusting for demographic and clinical differences, and the adjusted LOS decreased slightly (Table 2).
The postdischarge patient satisfaction survey response rate was poor (1,102 of 7,084 [15.6%] in year one; 987 of 5,649 [17.5%] in year two), but the survey results were similar from year one to year two (Table 2).
We redesigned the structure of our medical teaching service to improve the educational experience of our residents. Our intervention was associated with improvements in resident satisfaction, allocation of resident time, student satisfaction, the evaluation of previously unevaluated patients, ICU admission, and patient LOS, and it did not require external funding.
Changing systems of care delivery is challenging, often requiring multiple simultaneous, synergistic interventions,26 and demonstrating improvements in care is difficult.27 Our intervention was associated with several positive effects, which might be attributable to some combination of the following factors: reducing handoffs between providers, which have been shown to adversely affect patient care28–30 and education;31 improving pairing of resident teams with hospitalist attendings, which may improve residents' educational experience;32,33 and reducing interns' patient volume. Some have called for a reduction in the number of patients that residents carry,4,5 and the Residency Review Committee for Internal Medicine has recently reduced interns' inpatient caps from 12 to 10, but there is little evidence to guide estimates of the optimal number of patients per intern.15 There is likely to be a “sweet spot” in patient volume, where experiential learning opportunities and clinical challenges are maximized but residents avoid becoming overworked and burned out.2,4,7,15,34 It is also likely that the optimal number of patients per intern depends on the specifics of the patients (e.g., acuity and heterogeneity), the hospital, the supportiveness of the educational environment, and, likely, the intern.15,35 The use of a “resident work formula” to determine the appropriate cap in specific situations has been advocated.5,15
Our findings build on those of McMahon and colleagues,18 who recently demonstrated improved resident satisfaction and time for learning with a multifaceted intervention that reduced resident workload and optimized attending interactions. In addition, we found improved student experiences and shortened LOS.
A critical aspect of our intervention is that it did not require increased expenditures for implementation, though we did shift two interns into the new structure from other rotations. Although costs were not directly measured, we found several sources of probable savings to the hospital. The expansion of the total resident service capacity allowed for reduced staffing of the nonteaching service (one less nurse practitioner or physician assistant on average per day) despite an increase in total patient volume; some of this cost savings to the hospital has been applied to hiring a resident assistant, whose job is to perform indirect patient care tasks, such as scheduling discharge follow-up appointments.36 Moreover, our redesign was associated with a reduction in LOS and ICU admissions, which suggests additional hospital savings.
An important benefit for medical students (and likely also for residents, though not measured directly) was the increased opportunity to evaluate fresh patients. In a previous study, we found a small but significant correlation between the number of fresh patients evaluated during our medicine clerkship and performance on the NBME subject exam.31 However, in the current study we were only able to account for students' exposure to fresh patients for half of the clerkship because students also rotated on other services and in five different hospitals. We suspect that students' increased contact with faculty and the increased satisfaction with attending rounds were likely related to the alignment of teams with a single teaching attending who was also caring for the same patients.
This study has a number of limitations. Research attempting to assess the effects of an educational intervention in a real-world setting is inherently challenging. We attempted to address potential confounders in our analyses, but we cannot conclude definitively that postintervention differences are attributable to the intervention. The sample of residents from year one to year two changed as old residents graduated and new ones started, so cohort effects might account for some differences. We were unable to directly measure possible effects of the intervention on our residents' learning, and resident enjoyment of a rotation may not correlate with learning. We had insufficient random paging data to detect moderate-sized differences in time spent in different activities or with different people, and the return rate for the random paging logs, particularly in year two, was suboptimal.
The medical service at our hospital is very complex, with many simultaneous factors contributing to the overall experience of patients and residents. Although we attempted to account for as many of these factors as possible, additional important factors, such as patient geography within the hospital (which we were unable to account for in our analysis), may have played a significant role in residents', students', and patients' experiences. Differences in diagnoses or other patient characteristics not accounted for in our regression models may also have contributed to the differences in outcomes.
In summary, our resident teaching service redesign seemed to be beneficial to our residents. It was also associated with improvements in the medicine clerkship experience, a favorable impact on patient outcomes, and probably substantial savings for the hospital. Additional research is needed to establish the generalizability of our findings and to determine the most critical features of our intervention.
The authors thank Bojia Li and Kasia Sipp for their help with data collection; Alice Gordon, Mary Lou Beagan, and Maryann Countryman for their help with the random paging; Steve Lurie and Chris Mooney for their help assessing student data; and Ron Epstein, Denham Ward, and the Dean's Teaching Fellows for supporting this project and providing helpful comments and feedback about the manuscript.
The University of Rochester Dean's Teaching Fellow Program provided salary support to Dr. O'Connor.
This study was reviewed and approved by the University of Rochester research subjects review board.
An abstract summarizing these data was presented at the Association of Program Directors in Internal Medicine Annual Meeting, April 2010, Baltimore, Maryland.
1 Smith LG, Humphrey H, Bordley DR. The future of residents' education in internal medicine. Am J Med. 2004;116:648–650.
2 Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC. Redesigning residency education in internal medicine: A position paper from the Association of Program Directors in Medicine. Ann Intern Med. 2006;144:920–926.
3 O'Malley PG, Khandekar JD, Phillips RA. Residency training in the modern era: The pipe dream of less time to learn more, care better, and be more professional. Arch Intern Med. 2005;165:2561–2562.
4 Ludmerer KM, Johns MM. Reforming graduate medical education. JAMA. 2005;294:1083–1087.
6 Friesen LD, Vidyarthi AR, Baron RB, Katz PP. Factors associated with intern fatigue. J Gen Intern Med. 2008;23:1981–1986.
7 Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burnout and self-reported patient care in an internal medicine residency program. Ann Intern Med. 2002;135:358–367.
9 Vidyarthi AR, Arora V, Schnipper JL, Wachter RM. Managing discontinuity in academic medical centers: Strategies for a safe and effective resident sign-out. J Hosp Med. 2006;1:257–266.
10 Vidyarthi AR, Auerbach AD, Wachter RM, Katz PP. The impact of duty hours on resident self report of errors. J Gen Intern Med. 2007;22:205–209.
11 Logio LS, Djuricich AM. Handoffs in teaching hospitals: Situation, background, assessment, and recommendation. Am J Med. 2010;123:563–567.
12 Wayne DB, Arora V. Resident duty hours and the delicate balance between education and patient care. J Gen Intern Med. 2008;23: 1120–1121.
13 West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;23:1294–1300.
14 Gopal R, Glasheen JJ, Miyoshi TJ, Prochazka AV. Burnout and internal medicine resident work-hour restrictions. Arch Intern Med. 2005;165:2595–2600.
15 Fletcher KE, Reed DA, Arora VM. Doing the dirty work: Measuring and optimizing resident workload. J Gen Intern Med. 2011;26:8–9.
16 Goiten L, Shanafelt TD, Wipf JE, Slatore CG, Back AL. The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. Arch Intern Med. 2005;165:2601–2606.
17 Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF, Saint S. Systematic review: Effects of resident work hours on patient safety. Ann Intern Med. 2004;141:851–857.
18 McMahon GT, Katz JT, Thorndike ME, Levy BD, Lascalzo J. Evaluation of a redesign initiative in an internal-medicine residency. N Engl J Med. 2010;362:1304–1311.
25 Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619.
26 Nutting PA, Crabtree BF, Miller WL, Stewart EE, Stange KC, Jaén CR. Journey to the patient-centered medical home: A qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med. 2010;8(suppl 1):s45–s56.
27 Zwarenstein M, Bryant W. Interventions to promote collaboration between nurses and doctors. Cochrane Database Syst Rev. 2000;(2):CD000072.
28 Petersen LA, Brennan TA, O'Neil AC, Cook EF, Lee TH. Does housestaff discontinuity of care increase the risk of preventable adverse events? Ann Intern Med. 1994;121:866–872.
29 Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign-out and suggestions for improvement: A critical incident analysis. Qual Saf Health Care. 2005;14:401–407.
30 Horwitz LI, Moin T, Krumholz HM, Wang L, Bradley EH. Consequences of inadequate sign-out for patient care. Arch Intern Med. 2008;168:1755–1760.
31 Lang VJ, Mooney CJ, O'Connor AB, Bordley DR, Lurie SJ. Association between hand-off patients and subject exam performance in medicine clerkship students. J Gen Intern Med. 2009;24:1018–1022.
32 Hauer KE, Wachter RM, McCulloch CE, Woo GA, Auerbach AD. Effects of hospitalist attending physicians on trainee satisfaction with teaching and with internal medicine rotations. Arch Intern Med. 2004;164:1866–1871.
33 Kulaga ME, Charney P, O'Mahony SP, et al. The positive impact of initiation of hospitalist clinician educators. J Gen Intern Med. 2004;19:293–301.
34 Thomas NK. Resident burnout. JAMA. 2004;292:2880–2889.
35 Haney EM, Nicolaidis C, Hunter A, Chan BK, Cooney TG, Bowen JL. Relationship between resident workload and self-perceived learning on inpatient medicine wards: A longitudinal study. BMC Med Educ. 2006;6:35.
36 Finn KM, Carty M, Laing M, Harris L, Rachelefsy L, Katz JT. A program for reducing noneducational resident activities at an academic medical center. J Clin Outcomes Manag. 2008;15:25–30.
Supplemental Digital Content
© 2011 Association of American Medical Colleges