Kogan, Jennifer R.; Bellini, Lisa M.; Shea, Judy A.
Problem Statement. Residency programs have changed to comply with resident duty hour regulations. This study's purpose was to determine if there are differences in medicine clerkship students’ activities, associations, and perceptions of educational value before and after duty hour reform.
Method. Medicine clerkship students, both before (n = 36) and after (n = 33) duty hour reform, wore random reminder pagers for one week and completed time allocation surveys with each signal. Event proportions were calculated and analysis of variance assessed group differences.
Results. A total of 804 and 912 surveys were completed before and after reform, respectively. No differences existed in proportion of time for direct patient care (.13 versus .14, p = .35), indirect patient care (.35 versus .32, p = .21), and education (.38 versus .37, p = .69) activities, students’ associations, educational value, and time in the hospital before and after reform.
Conclusions. Residency program changes had minimal impact on medicine clerkship students.
In July 2003, residency programs across the United States were required to comply with the Accreditation Council of Graduate Medical Education's resident duty hour regulations: when averaged over four weeks, residents can work no more than 80 hours per week and must have one day in seven free; they must not work more than 24 continuous hours with an additional six hours for education and transfer of care; in-house call must occur no more frequently than every third night; and there must be at least ten hours of rest between duty periods.1 While the impact of duty hour reform most affects residents, it is plausible that medical students on core clinical rotations also would be affected.2–4 During inpatient clerkships, medical students typically are assigned to a team of interns, residents, and faculty. Given changes in team structure and admitting patterns,2–4 there is concern that duty hour reform might impact student education, specifically the continuity of students’ experience with their team, the amount of didactic and bedside teaching, and the perceived educational value of daily activities (personal communication, Clerkship Directors in Internal Medicine meeting, 2003).
Random work sampling, which requires individuals to record their activities instantaneously at randomly selected moments during the workday, previously has been used to quantify the daily and on-call activities of residents.5,6 The perceived educational value of an activity can be simultaneously rated.5,6 This technique allows educators to examine the structure and content of their programs and use the information to rationally redesign curricula and programs.5,7
This study was designed to determine, using random work sampling, if there are differences in medicine core clerkship students’ daily activities, whom they interact with, and the perceived educational value of their experiences before and after implementation of resident duty hour regulations at our institution. We hypothesized that students would spend less time in the hospital, perceive their educational experiences less favorably, and would be less likely to interact with their team after duty hour reform. Additionally, we sought to describe the time allocation of students during their medicine clerkship and understand which activities they perceive to be most educationally rewarding. We expected that students would most value those experiences involving direct patient care and contact with housestaff and faculty.
All students enrolled in the six-week inpatient internal medicine core clerkship at our institution between March 31 and June 20, 2003 (n = 41) (pre-reform period) and between September 29 and December 19, 2003 (n = 39) (post-reform period) were asked to participate. Although student participation was voluntary, a raffle for a university bookstore gift certificate was used as an incentive. Students were not enrolled immediately after duty hour reform implementation (July through September) since we expected that residency program changes would still be in flux. During their core clerkship, students are assigned to one of four inpatient hospital sites. All sites were partially in compliance with duty hour regulations before July 2003 in that residents typically had one day in seven free, always had more than eight hours off between duty periods, and usually worked fewer than 80 hours per week on average. All sites implemented programmatic changes and were fully compliant with duty hour regulations by July 2003. The major structural change was the addition of day-float nurse practitioners to each team to support postcall workload, enabling the postcall team to leave within the required 30 hours. Before and after reform, students took call approximately every fourth night until 10 pm. After reform, we did not specify that students should or should not follow resident duty hours.
Participants were randomized by week and by hospital site to wear a random reminder pager (manufactured by Divilbliss Electronics, Champaign, Illinois). For any given week, approximately four or five students wore a random time pager. The pager was set to signal randomly approximately every 90 minutes for one week. This time interval was chosen as one that would be minimally invasive in the students’ day and is the time interval used in prior work sampling studies.5 Additionally, this sampling frequency would provide enough data points to compare pre- and post-reform groups. Assuming a minimum of four students per week for eight weeks would record five events per day over five days (eliminating the days pagers were received and returned), we had power of .80 with an alpha of .05 to detect differences in proportions of .10 between groups with sample sizes of 407 at mid-distribution and 160 at the ends of the distribution. Students were instructed to turn the pager on when arriving at the hospital and turn it off when leaving for the day. Students did not wear the pager if they were not in the hospital (i.e., evenings and noncall weekends). Students were given a booklet of surveys that could fit in their white coats and were instructed to complete a survey immediately after the pager signaled.
The survey was adapted from a time allocation survey used with residents in which activities were characterized using the dimensions: who (association), where (location), when (time), what (activity type), and why (value of task).5 When items included day of the week, time, and call cycle day. The where item referred to activity location defined by five discrete sites (patient's room, conference room, hallway, ward, other). The activity (what) item used a branching format in which questions progressed from lesser to greater specificity (see Table 1). Major activities included direct patient care, indirect patient care, education, personal, walking, or other. Students were instructed to choose only one activity. Who was defined by a list of 13 people students might be with (students selected all that applied) (see Table 2). Students rated the activity's educational value on a five-point rating scale (1 = low; 5 = high). Finally, students recorded the times they arrived to and left the hospital daily. Instructions for completing the survey were reviewed with students; booklets were returned at the end of each week. The study was approved by the Institutional Review Board.
The proportion of events spent in each activity (event proportion) was calculated and then averaged across all students within the pre- and post-reform groups. Analysis of variance was used to assess group differences in mean proportions. Each event was treated as an independent event based on the random sampling of students within week and events.
Thirty-eight students (93%) agreed to participate before duty hour reform and 37 (95%) after duty hour reform. Sixty-nine students (92%) returned surveys (n = 36 before reform and n = 33 after reform). A total of 804 and 912 surveys were completed before and after reform, respectively. The mean number of surveys completed per student per week was 24.9 (SD = 7.6); the mean number per day was 4 (SD = 2.5). There were no differences in the amount of time students spent in the hospital per day before and after reform (nine hours, 53 minutes versus nine hours, 12 minutes; p > .05). Additionally, there were no differences between groups in the amount of time spent in the hospital during on-call days (13 hours, five minutes versus 13 hours, nine minutes per day; p > .05) or postcall days (seven hours, 36 minutes versus seven hours, 43 minutes per day; p > .05).
With respect to location, there were no significant differences before and after reform in the likelihood that students were in a patient's room (.11 versus .13, p = .26), conference room (.27 versus .30, p = .17), or hallway (.17 versus .14, p = .07). Before reform, students were more likely to be on the ward (.28 versus .21, p = .0007). With respect to students’ activities, there were no significant differences between groups in event proportions for direct patient care, indirect patient care, and education (see Table 1). Although there were no differences between groups in the perceived educational value of direct patient care or education activities, indirect patient care activities were more valued before reform. Events most valued included doing the initial history/physical, observing the history/physical, observing education/counseling, and most education activities (see Table 1).
Students were equally likely before or after reform to be alone, with their resident, their attending, another resident and another intern, but were more likely to be with their own intern after reform (see Table 2). Students valued interacting with attendings though means were relatively high for all interactions except being alone.
Many have expressed concern that resident duty hour reform might have negative consequences on graduate and undergraduate education.2–4 Residency programs have implemented or expanded day-float programs, night-float programs, and “uncovered services” that create frequent hand-offs that compromise continuity of care and might hinder clinical skill and professional development.2 We utilized a work-sampling method to evaluate the impact of duty hour reform on medical students in our medicine core clerkship. This technique is inexpensive and reduces the bias imposed by observers, is less subject to recall bias compared to activity diaries and can produce quantifiable results.5,6
Similar to resident time and motion studies, we found that students spent minimal time at the bedside and significant time doing indirect patient care.5,7 However, students did spend a significant amount of time engaged in highly valued education activities. Importantly, despite changes in our residency program in response to duty hour reform, there were few differences in the time students spent in the hospital, who they interacted with, the proportion of events spent in various activities, and their perceived educational value, thus reassuring us that our students have been minimally affected by the residency changes implemented. The total number of students engaged in the specific activities shown in Table 1 was small, however, and further conclusions about what students value cannot be drawn from this study.
Our findings were unanticipated. Why was there no difference when we expected there to be one? First, the study occurred during two different times in the academic year. March through June is the end of the residency year, a time when housestaff are efficient in their patient care and educational responsibilities. As such, resident–student teams before duty hour reform may have been able to leave the hospital earlier each day thereby minimizing any difference in postcall day length or activities compared with the post–duty hour reform period. Second, students attend didactic conferences two afternoons per week. Presuming that the impact of duty hour reform is in the afternoon, students, by virtue of the didactic curriculum, would be less affected by the change. However, our findings did not change when we excluded from the analyses the days students attend didactic conferences (data not shown). Third, it is conceivable that while the allocation of time did not differ, qualitative experiences might. Since clerkship students either did their medicine clerkship before or after duty hour reform, students were not able to make comparative judgments of the effect. Finally, our program adopted expanded day-float programs staffed by nurse practitioners to meet the duty hour regulations. This initiative may have preserved enough traditional team structure and interactions that the change for students was minimal.
There are several limitations. First, this is a single-institution study. Generalizability of these findings to other medical schools, disciplines, and residencies adopting alternative solutions to duty hour regulations requires further investigation. Second, although we instructed students to complete surveys immediately after being paged, we do not know definitively that they instantly completed surveys since the pagers did not record the time of each page. The lower-than-expected mean number of pages per day indicates that surveys may not have been completed for every page. Third, this study only assessed the brief period of time after reform and is unable to address the long-term impact of duty hour reform on student's clinical skill and professional development. Fourth, given the random sampling of students within weeks and events within students, each data point was treated as an independent event. Arguably, there was potentially clustering at the level of the student's team that influenced the likelihood of participating in various events. Whatever impact this may have had was not apparently different before and after reform.
Despite these limitations, we believe this is one of the first studies to assess the impact of duty hour reform on medical students. Continued efforts exploring the long-term impact of duty hours on students are needed so that educators can be assured that students’ clinical experiences are conducive to achieving curricular goals.