Residents can make a significant contribution to medical student education. Research has demonstrated that the ability and willingness of residents to teach medical students influences students’ subsequent career choice,1 professional growth, clerkship performance, perceptions of clerkship quality, and clerkship satisfaction.2–7 In a survey by Byrne and Cohen,8 medical students indicated that, of all clinical team members, residents contributed the most to the students’ learning in the clinical clerkship.
In 2003, the Accreditation Council for Graduate Medical Education (ACGME) implemented restrictions on resident duty hours, imposing an 80-hour-per-week limit.9 Studies on the effects of duty hours regulations on resident teaching have yielded mixed results. Some have found no negative influence on teaching or that residents teach more10,11; others suggest that residents are less involved in student teaching.12–15 Kogan and colleagues14 reported that a majority of internal medicine clerkship directors felt that residents had less time to teach after the duty hours changes. A retrospective study comparing students’ perceptions of resident teaching in a surgery clerkship before and after implementation found a consistent negative change in students’ perceptions of the quality and quantity of resident teaching after the regulations.15
Although the ACGME actively regulates residents’ duty hours, changes in medical student duty schedules have been slower to follow. Medical students frequently take calls during their clinical clerkships that can last up to 36 hours. A study on the effects of a week of 12-hour night-float shifts compared with every-fourth-night call found that students preferred the night-float experience.16
On July 1, 2009, to accommodate increased class size at the University of Missouri–Kansas City School of Medicine (UMKC SOM), we implemented a schedule change in our pediatric clerkship’s four-week inpatient rotation. The change eliminated the previous schedule of daytime work hours plus an evening call once a week from 6:00 AM to 10:30 PM and replaced it with a schedule of three weeks of daytime work hours and one week of five consecutive 12-hour overnight shifts. The overnight hours paired students with residents on our night-float team. We hypothesized that incorporating medical students on the resident night-float team for five days would provide more time for one-on-one student–resident teaching. We also anticipated that the new schedule would allow us to effectively handle the increased number of students per clerkship without detrimental effects on students’ satisfaction with the clerkship or performance on the National Board of Medical Examiners (NBME) pediatric shelf exam.
Therefore, the objectives of our study were to measure the effect of the change in scheduling on (1) students’ pediatric NBME cognitive performance, (2) number of admissions the students performed, (3) students’ overall satisfaction with the clerkship, and (4) students’ perceptions about the amount and quality of resident teaching.
Study concept and design
Our study had two independent parts. Part 1 consisted of a retrospective historical controls study; we included only preexisting, deidentified participant data in the Part 1 retrospective study. Part 2 was a prospective survey. The social sciences institutional review board at the UMKC SOM approved both parts of the study protocol. Part 1 of the study was exempt from informed consent. We obtained informed consent from all participants in the Part 2 prospective study.
Participants were medical students from the UMKC SOM completing the four-week inpatient portion of the pediatric clerkship at Children’s Mercy Hospitals and Clinics in Kansas City, Missouri. UMKC SOM is a combined six-year bachelor of arts/medical degree program. Students complete the eight-week pediatric clerkship in year five.
Observations and measurements
There were two groups of students in Part 1 of the study—the historical control group and the comparison group. The historical control group consisted of students who completed the inpatient portion of the pediatric clerkship between November 2008 and April 2009. Students in the historical control group had inpatient work hours that included daytime hours plus an evening call (DT+C) five times per month. The comparison group consisted of students who completed the inpatient portion of the pediatric clerkship between November 2009 and April 2010, following the schedule change. These students had inpatient work hours that consisted of three weeks of daytime shifts and one week of five consecutive overnight shifts (DT+OS).
For Part 1, we used deidentified aggregate data provided by the medical school and the hospital for all participants. We used scores on the pediatric NBME to assess cognitive performance. We assessed student involvement in patient care by counting the number of admission history and physical examinations (HPEs) completed in the electronic health record per student during the four-week inpatient rotation. We measured student satisfaction using the standard UMKC SOM end-of-clerkship evaluation. Specifically, we used students’ responses to the statement “Overall, I felt this was an effective clinical experience” on a five-point Likert scale (1 = Strongly Disagree; 5 = Strongly Agree).
The Part 2 study surveyed students using a two-item questionnaire with a field for open-ended comments, detailed below. We administered the online questionnaire to students by e-mailing one invitation each week for four weeks during the inpatient portion of the clerkship between November 1, 2009, and June 30, 2010. Thus, all students had a chance to complete the same questionnaire three times on the basis of their perceptions of their daytime experience, and once on the basis of their perceptions for the week of overnight work hours. Completing the survey was optional each time. We only included data from students who completed at least one daytime survey and the overnight survey as a paired sample in the analysis of the two items from the survey. For students who completed more than one daytime survey, we used the average of the daytime responses to create a single paired sample for each student. Students received no incentive for completing the survey.
We designed the survey instrument to study students’ perceptions of the time residents spent teaching (RTT) and the quality of resident teaching (QRT). First, we asked students to estimate the average number of hours residents spent teaching throughout the week and respond to the question “During the inpatient rotation you had in THIS PAST WEEK how many hours were spent EACH DAY by the residents in activities related to teaching medical students?” The survey defined teaching activities through the following examples: “demonstrating history and physical exam, critiquing your presentation, clarifying your notes, interpretation of labs/X-rays, discussion of differential diagnosis, management of patients, addressing patient problems and concerns, and communication with parents and health care workers such as nurses/respiratory therapists.” The second item on the survey asked students to indicate their level of agreement with the statement “I am satisfied with the quality of teaching by the pediatric residents” on a five-point Likert scale (1 = Strongly Disagree; 5 = Strongly Agree). We asked students to provide open-ended comments with the following prompt: “Please use the text box below to type any comments you might have regarding the above questions.”
We included all comments from all completed surveys in the qualitative analysis. We used a modified version of grounded theory to analyze the qualitative data.17,18 This was an iterative process that identified common themes from the open-ended comments collected on the weekly surveys. During this process, three of the investigators (N.T., S.T., and K.M.) independently analyzed the qualitative data, searched for emerging themes, and marked the key points with a series of codes, which they then extracted from the text. They grouped the codes into similar concepts and formed the themes. The final iteration of the analysis included group consensus of specific themes. Researchers then compared the similarities and differences between the identified themes representing the open-ended comments from the medical students on the daytime and overnight surveys.
We used an online anonymous survey tool (SurveyMonkey, Palo Alto, California) to distribute the survey and collect data from the medical students.
We used the independent t test to compare the NMBE scores, number of HPEs, and the overall satisfaction with the pediatric clerkship between the historical control and comparison groups for Part 1 of the study. To assess the RTT and QRT during the daytime and overnight work hours, we used paired samples t tests. All t tests were conducted using PASW Statistics 18, release version 18.0.0 (SPSS, Inc., Chicago, Illinois). We also examined the effect sizes to evaluate the practical significance of the statistically significant differences.
The DT+C group had 46 medical students, of which 27 (59%) were female; the DT+OS group had 54 medical students, of which 28 (52%) were female. Medical students in the DT+OS group had higher overall mean scores on the NBME than the DT+C group (73.94% versus 71.93%). However, this difference did not reach statistical significance (t = 1.79; P > .05). There was also no statistically significant difference between the DT+C and DT+OS groups in terms of overall satisfaction with the clerkship (t = 1.63; P > .05). The majority of students in both groups agreed or strongly agreed with the statement “Overall, I felt this was an effective clinical experience,” indicating a high level of satisfaction with their overall experience.
Medical records documented an increased number of HPEs per student per inpatient rotation in the DT+OS group compared with the DT+C group (mean = 7.49, standard deviation [SD] = 3.34 in DT+OS versus mean = 6.11, SD = 2.95 in DT+C), and this difference was statistically significant (t = 2.17; P = .03). In addition, we examined the effect size as an indication of practical significance and found a medium effect size for this difference (Cohen d = 0.44). This means that the average student in the DT+OS group had a higher number of HPEs than 67% of the students in the DT+C group.
Of the 101 students eligible for participation in the survey, 4 declined, leaving a total of 97 possible participants, of which 53 (55%) were female. Sixty-three (65%) of the students completed at least one daytime work hours questionnaire and the overnight work hours questionnaire. We only included data from the paired samples from these students in the analyses. The paired samples t test showed that the students’ perception of RTT did not differ significantly between the daytime and overnight surveys (t = 1.56; P = .12). However, the perception of QRT was significantly higher during the overnight shifts week (t = 2.47; P = .02) (see Table 1). We also found that the average student on overnight shifts rated the QRT higher than 68% of the students on daytime work hours (Cohen d = 0.48).
Tables 2 and 3 summarize the findings of the qualitative analysis from comments students provided on the overnight and daytime shifts surveys. We collected 49 comments from daytime surveys and 25 from overnight surveys. Five themes emerged for each from the qualitative analysis of students’ comments. Three of the five themes were the same for both daytime and overnight weeks: residents’ teaching practices, time to teach, and willingness to teach. The majority of comments (14; 56%) for overnight shifts mentioned residents’ use of student-centered teaching and allowing students to practice hands-on patient care skills, whereas only 18 (37%) of the total comments from the daytime shifts involved these themes. Although 9 (18%) of the daytime comments mentioned the fact that residents did not have much time to teach because of busy service, only 2 (8%) of the comments for overnights were related to residents’ limited time to teach. Students completing both daytime and overnight weekly surveys thought that residents were willing to teach despite the busy service and heavy patient load.
Themes exclusive to surveys completed during the daytime weeks of the rotation included recommendations on how to improve the clinical experience on the inpatient portion of the clerkship (7; 14%) and comments on the amount of time faculty attendings spent teaching (8; 16%). Only overnight student surveys had comments related to their degree of satisfaction with the week’s experience (4; 16%).
In our study, the incorporation of overnight hours into the inpatient schedule significantly improved the students’ experience without negative effects on their clerkship satisfaction or NBME performance. From our retrospective comparison of HPEs as a measure of the students’ clinical experience, we learned that students on the DT+OS schedule saw significantly more new-admission patients. This result, when viewed with the prospective study finding of improved perceptions of QRT on the overnight schedule, agrees with the findings of Dolmans and colleagues.19 That study of 1,208 medical students found that the perceived effectiveness of clinical rotations depended on the number and variety of patients and the quality of supervision. Students also perceived that fewer opportunities to examine patients, lack of time for supervision, and the presence of too many students at one time were factors that inhibited learning.20 We believe that although the participants in our study did not perceive a difference in RTT between the two clerkship structures, the increased ability of the night-float team residents to provide one-on-one supervisory behaviors in the form of feedback, direct observation, and role modeling, coupled with students’ increased participation in admissions and hands-on patient care skills, accounted for the significant difference in QRT.
Our study differs in important ways from other studies that have examined the effect of duty hours regulations on resident teaching. Part 2 studied students’ perceptions of both RTT and QRT rather than the perceptions of clerkship directors or residents. In addition, we conducted our study more than five years after the implementation of duty hours regulations, well after residents and programs made adjustments to the new rules, such as the implementation of night-float teams. As a result, our findings may be more relevant than previous studies to current resident schedules. For medical educators, our findings would support that resident night-float teams represent opportunities for both higher-yield student experiences and improved resident teaching.
Restructuring our pediatric clerkship to include one week of overnight shifts allowed us to add 4 students per eight block, or 24 students per year, without negatively affecting the quality of the clinical experience, students’ performance on the NBME subject exam, or amount of resident teaching. This is relevant to clerkship directors who face increased class sizes in response to recommendations by the Association of American Medical Colleges.21 Between the 2002 and 2010 academic years, 80% of the schools accredited in 2002 increased their enrollment.22 Combined first-year MD-granting and DO-granting medical school enrollment in 2015 is projected to be 35% above 2002 levels. With enrollment on the rise, medical schools must find ways to expand capacity while preserving the quality of clinical training.
Our study has several limitations. It was conducted at a single site for a single clerkship in pediatrics and may not be generalizable to other sites, specialties, or clerkships. In addition, because the night-float team does not have a faculty attending in house, students work exclusively with residents during their week of overnight shifts. It is possible that the presence of faculty attendings during students’ daytime shifts superseded the daytime residents’ role in QRT. We did not, however, find any qualitative themes related to this in our surveys.
We used a group of historical controls for Part 1 of the study, comparing groups from two different academic years. The groups may not have had the same characteristics, and faculty and resident exposure varied from rotation to rotation. Patient volume could have varied also, although we could find no evidence from hospital records that it differed substantially from month to month between the two years. It is possible that these factors may have contributed to the differences we found. We were, however, able to compare students from the same school, using the same evaluation system, rotating at the same children’s hospital, with the same team structure of faculty and residents during corresponding time periods. Additionally, we made no other changes to the clerkship during the periods of comparison.
In Part 2, we only included data from students who completed at least one daytime survey and the overnight survey as a paired sample in the analysis of the two survey items. Furthermore, we averaged the responses from students who completed more than one daytime survey. We do not know what effect, if any, this may have had on the analysis. Because students participating in the Part 2 prospective study frequently referred to the time of day they worked in their comments, we were unable to blind the reviewers regarding daytime versus overnight comments for the qualitative portion of the study.
Finally, data suggest that schedules with both daytime and overnight shifts will disrupt sleep and circadian rhythm.16 Our study was limited in scope, so we did not ask students about fatigue, sleep schedule disruption, or other quality-of-life measures. These potential negatives associated with overnight scheduling should be addressed in future research.
Implementing overnight work hours for the medical students in our pediatric clerkship allowed us to increase clerkship capacity while maintaining student satisfaction and cognitive performance. Students also enjoyed the added benefits of increased clinical experience and improved quality of resident teaching.
Acknowledgments: The authors would like to thank the student participants who made this work possible.
Other disclosures: None.
Ethical approval: The social sciences institutional review board at the University of Missouri–Kansas City School of Medicine approved both parts of the study protocol.
Previous presentations: This work has been presented in abstract form at the Pediatric Academic Societies Annual Meeting, Denver, Colorado, April 2011; at the Council on Medical Student Education in Pediatrics Annual Meeting, San Diego, California, March 2011; and at the International Association of Medical Science Educators Annual Meeting, New Orleans, Louisiana, July 2010.
1. Wright S, Wong A, Newill C. The impact of role models on medical students. J Gen Intern Med. 1997;12:53–56
2. Bing-You R, Edwards JDistlehorst LH, Dunnington GL, Folse JR eds. Residents as teachers. Teaching and Learning in Medical and Surgical Education. 2000 Mahwah, NJ Lawrence Erlbaum Associates, Inc.
3. Sheets KJ, Schwenk TL. Teaching skill improvement for graduate medical trainees. An agenda for research and development. Eval Health Prof. 1988;11:3–19
4. Kernan WN, Fagan MJ, Michael Elnicki DCDIM Council. . The Clerkship Directors in Internal Medicine’s agenda for advancing medical student education in internal medicine. Am J Med. 2004;117:713–717
5. Griffith CH 3rd, Georgesen JC, Wilson JF. Specialty choices of students who actually have choices: The influence of excellent clinical teachers. Acad Med. 2000;75:278–282
6. Roop SA, Pangaro L. Effect of clinical teaching on student performance during a medicine clerkship. Am J Med. 2001;110:205–209
7. Gerbase MW, Germond M, Nendaz MR, Vu NV. When the evaluated becomes evaluator: What can we learn from students’ experiences during clerkships? Acad Med. 2009;84:877–885
8. Byrne N, Cohen R. Observational study of clinical clerkship activities. J Med Educ.. 1973;46:93–95
9. Accreditation Council for Graduate Medical Education. . Common Program Requirements. http://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramResources/Common_Program_Requirements_07012011.pdf
. Accessed November 15, 2012
10. Nixon LJ, Benson BJ, Rogers TB, Sick BT, Miller WJ. Effects of Accreditation Council for Graduate Medical Education work hour restrictions on medical student experience. J Gen Intern Med. 2007;22:937–941
11. Jagsi R, Shapiro J, Weinstein DF. Perceived impact of resident work hour limitations on medical student clerkships: A survey study. Acad Med. 2005;80:752–757
12. Zahn CM, Dunlow SG, Alvero R, Parker JD, Nace C, Armstrong AY. Too little time to teach? Medical student education and the resident work-hour restriction. Mil Med. 2007;172:1053–1057
13. Lin GA, Beck DC, Garbutt JM. Residents’ perceptions of the effects of work hour limitations at a large teaching hospital. Acad Med. 2006;81:63–67
14. Kogan JR, Pinto-Powell R, Brown LA, Hemmer P, Bellini LM, Peltier D. The impact of resident duty hours reform on the internal medicine core clerkship: Results from the clerkship directors in internal medicine survey. Acad Med. 2006;81:1038–1044
15. Brasher AE, Chowdhry S, Hauge LS, Prinz RA. Medical students’ perceptions of resident teaching: Have duty hours regulations had an impact? Ann Surg. 2005;242:548–553
16. Matthews ML, Gross P, Herbert WN. Post-call cognitive function and satisfaction in medical students on different call schedules: A prospective observational pilot study. Am J Obstet Gynecol. 2006;195:1484–1488
17. Denzin NK, LincolnY S eds The Handbook of Qualitative Research. 20053rd ed Thousand Oaks, Calif Sage Publications
18. Merrian SB ed Qualitative Research in Practice: Examples for Discussion and Analysis. 2002 San Francisco, Calif Jossey-Bass Publishers
19. Dolmans DH, Wolfhagen IH, Essed GG, Scherpbier AJ, van der Vleuten CP. The impacts of supervision, patient mix, and numbers of students on the effectiveness of clinical rotations. Acad Med. 2002;77:332–335
20. Dolmans DH, Wolfhagen IH, Heineman E, Scherpbier AJ. Factors adversely affecting student learning in the clinical learning environment: A student perspective. Educ Health (Abingdon). 2008;21:32
21. Center for Workforce Studies. Recent Studies and Reports on Physician Shortages in the US. https://www.aamc.org/download/100598/data/recentworkforcestudies.pdf
. Accessed November 21, 2012
22. Center for Workforce Studies. Results of the 2010 Medical School Enrollment Survey. https://www.aamc.org/download/251636/data/enrollment2011.pdf
. Accessed November 16, 2012