The Accreditation Council for Graduate Medical Education (ACGME) work-hour guidelines were established in 2002. Section 405 of the New York Public Health Code, publication of the Institute of Medicine report “To Err Is Human”, and policies of the ACGME led to implementation of resident work-hour restrictions [12, 14, 16]. These restrictions limit the number of work hours to 80 hours weekly, overnight call frequency to no more than one every third day, 30-hour maximum continuous shifts, and a minimum of 10 hours off between shifts. Although these limits are voluntary, adherence has been mandated for the purposes of accreditation.
Since the inception of work-hour regulations, there has been considerable concern regarding the impact on graduate medical education. More specifically, there has been concern in orthopaedic surgery that residents may experience diminished opportunity to gain experience in the operating room and clinic. Baskies et al. reported the implementation of work-hour regulations did not considerably alter operative case logs of residents before and after restrictions at one academic, university-based residency program . A survey study by Kusuma et al. found that although orthopaedic residents report an improved quality of life as a result of these new rules, residents had concerns regarding a decrease in the quality of training . Respondents cited deficits in learning and training opportunities (24%), decreased continuity of care (16%), and a “shift worker” mentality (10%), as problems with their clinical program following work hour restrictions . Their study revealed that although almost 50% of residents thought that work hour restrictions had benefited them personally, only 40% thought these restrictions had a positive impact on the residency program .
Although operative and clinical experience are critical factors in determining successful resident education, exposure and opportunity for academic productivity during training remains a focus of many residency programs. Academic productivity can be measured by number of publications, and these publications often take substantial time and work to complete. However, it is uncertain whether implementation of work-hour restrictions has influenced resident academic performance as defined by peer-reviewed publications. Based on reduced clinical hours, we presumed residents would produce more publications per resident-year after the implementation of work-hour rules.
Our primary study questions were: (1) Has the implementation of work-hour rules increased the number of published resident clinical studies, laboratory research, case reports, and review articles? (2) Does the postgraduate year (PGY) influence the number of publications before and after implementation of work-hour rules. (3) Do selected demographic variables influence the number of resident-authored publications?
Materials and Methods
We reviewed all 139 residents in the orthopaedic surgery residency program at the University of Pennsylvania from 1995 to 2009 (total of 789 resident-years). This is an academic-based orthopaedic surgery residency program that is 5 years in length with an elective resident research year option for two residents per year between the second and third years. This study met the University of Pennsylvania Institutional Review Board's exemption criteria given that all information already was published, publically available, or blinded to the investigators.
A power analysis revealed that for our primary outcomes of interest, number of publications, we would require 50 resident-years per group in each of two groups (before and after implementation of the work-hour rules) to detect a medium effect size between groups (standardized delta 0.50) with an alpha of 0.05 and a power of 0.80 . Because number of peer-reviewed publications is sensitive to outliers, we performed a power calculation for probability of publishing an article in any given resident-year. Each resident-year is considered either 1 or 0, so this method of assessment is not affected by outliers. With a similar tolerance to Type I and Type II errors (alpha of 0.05 and power of 0.80),we needed a total of 184 resident-years to detect a 15% difference in publication rate between groups before and after the work-hour rules were in place .
Names of current and past residents of the University of Pennsylvania orthopaedic residency program from academic years 1995-1996 to 2008-2009 were obtained from the current residency coordinator. PubMed was searched for peer-reviewed publications for each resident. We chose to restrict our study to publications in peer-reviewed journals listed in Index Medicus. Each search involved the last name with the first initial. Each PubMed search had a time limitation of the subject's years in residency plus an additional 2 postgraduate years to accommodate for publication lag time after graduation. Citations were analyzed individually to verify the appropriate resident's authorship for a publication. Verification was based on the publishing institution information and crossreferencing senior authors as faculty members at the University of Pennsylvania. To ensure all publications that were completed during residency were included, we included publications if they could be linked to the resident's medical school through the same verification protocol. The number of indexed publications for each resident for each year was recorded along with type of publication (clinical series, laboratory research, review article, or case report) and if the resident was the first author. Individual resident demographics regarding gender, PGY in residency, medical school publications, and advanced degrees held by the subject (in addition to a medical degree) were assessed. Advanced degrees were defined as any Master's or Doctorate degree beyond a medical degree. To determine total number of publications, type of publication, and whether the resident was the first author, the 2003-2004 academic year was the first year considered “postimplementation” of work-hour changes. This allowed for the time needed to ensure improved compliance with the work-hour rules and to account for publication lag time. Demographics were similar between populations of residents before and after work hour restrictions. However, there were fewer females before work hour rules compared with after work hour rules (Table 1).
Student's t-tests for independent samples with equal variances not assumed were used to assess differences between the pre-80-hour work week group and the post-80-hour work week group in terms of total publications and type of publication. To assess the probability that any given resident would publish in any given year, the chi square test with Yates' correction for continuity was used. To compare within PGY when there were less than five contacts per cell, a Fisher's exact test was used instead of the chi square test. Linear regression was used to determine the relationship between date of publication and publication rate. Results of publications and publication rate were graphed against time. Data analysis also was conducted after eliminating subjects who undertook a dedicated research year. Multiple binary logistic regression was used to eliminate confounding effects on publication productivity as a result of improved resident recruitment efforts, the presence of research-oriented residents, and gender.
After postwork-hour restrictions were implemented, residents were 1.53 times more likely (95% CI, 1.08, 2.02; p = 0.006) to publish in a given year when compared with pre-80-hour work week, adjusted for gender, advanced degree, and whether the resident had articles published during medical school. The average number of publications per resident year increased for total articles (from 0.53 per resident-year to 0.95 per resident year; p < 0.001), clinical series (from 0.28 to 0.50 per resident-year; p = 0.001), case reports (from 0.07 to 0.18 per resident-year; p = 0.002), and reviews (from 0.07 to 0.19 per resident-year; p < 0.001) after implementation of work-hour limitations (Table 2). There was no difference in rate of publications per resident-year for basic science manuscripts. There was an increasing trend with time for total publications after work-hour restrictions (Fig. 1), and an increasing trend for total publications per resident-year after work-hour restrictions (Fig. 2). When analyzing all subjects, there was a greater probability (odds ratio, 1.54; p = 0.005) of publication in any given resident-year after work-hour rules than before. There was an increased rate of publications in which the resident was the first author (from 0.23 to 0.39 per resident-year; p = 0.006). Linear regression showed a linear trend for total number of publications or publications per resident/year with time (p < 0.001 and 0.001, respectively). The linear trend between total publications and publications per resident remained consistent after eliminating patients who undertook the mandatory research year from the data analysis.
The number of publications increased in each PGY after work-hour restrictions (Fig. 3). However, when stratified by PGY, only PGY 1 and PGY 2 residents published peer-reviewed articles at a higher rate (p = 0.009 and 0.013, respectively) after work-hour restrictions than before work-hour restrictions (Table 3).
When demographics were considered, a resident was more likely to publish in any given year if he or she held an advanced degree (odds ratio [OR], 2.3; p = 0.001) or had a medical school publication (OR, 1.14; p = 0.005). Gender was not a factor (p = 0.380) in odds of publication.
Orthopaedic surgery training programs have gone through several transitions, including adaptation of new work-hour rules, increasing demands of clinical activity, and decreasing reimbursements. Increasing educational debt also may be an important factor that negatively influences a resident's interest in an academic career, whereas early exposure to academic endeavors may be a positive influence. As resident work-hours have been reduced, we assume residents have more personal time available for clinical and laboratory research and/or writing review articles or case reports. In this study, we defined resident academic performance by number of publications and sought to determine whether implementation of work-hour rules increased the numbers of published clinical studies, laboratory research, case reports, and review articles by residents. Secondarily, we attempted to examine whether PGY or demographic variables also influenced publications before and after implementation of work-hour restrictions.
There are inherent limitations in this study. First, we cannot account for actual time spent on academic pursuits by residents. Second, we do not know how many residents were involved in research studies or with papers that never were published. Third, we did not search the literature outside that indexed by PubMed and may be underestimating by 78% to 100% the total publications for each resident [17, 18]. Searches limited to PubMed have been used in other studies [5-10], and more importantly, we have no reason to suspect this method would selectively bias the findings toward one or the other groups in the study. Additionally, increased publications may have been influenced by other factors such as implementation of online submissions by many journals. Fourth, some residents complete their residency training at the same institution as where they attended medical school and often will start completing research projects that then are published after they start their residency. If a resident published a paper during the residency years and it could be traced to either their medical school or their residency, we included it so as not to miss any papers that were completed during residency; we included such papers in both groups for consistency. Fifth, some publications may have more than one resident as a coauthor, and we assumed this practice did not change during the study period. We found the number of first-author resident papers has increased since implementation of work-hour restrictions. Sixth, there have been changes in our institution's orthopaedic surgery program during the study period. There was a change in chairperson in 1997; however, the same chairperson was present for 12 of the 14 study years and the program director has remained unchanged during the past 12 years. There also have been changes in faculty members; however, every subspecialty of orthopaedics has remained staffed throughout the research period and there have not been changes in resident research requirements during the study period. Seventh, it is possible that an increased interest in resident publication has occurred during the past 6 years that is unrelated to work-hour regulations; however, we had no measure of interest for assessing this factor. Eighth, we defined academic performance in this study as number of publications; however, one could define academic performance by other measures, including in-training scores or the quality of research published as opposed to just numbers. Perhaps the quality of peer-reviewed publications has diminished; however, we are unable to assess quality based on level of evidence. It is our belief that some Level I or II studies may have less impact and be less “quality” than some Level III or even IV studies and this was not the specific goal of this study. Finally, this is an initial study that evaluates resident education at one urban, academic program. Previous studies of this kind also have been single-institution investigations [3-5, 9, 11], and additional investigation is necessary to determine whether these changes are consistent across other programs.
We found that since implementation of work-hour restrictions, total resident publications and publications per resident-year have increased. To our knowledge, only one other study has specifically evaluated resident publication rates. Elliott and Lee trended general surgical resident research productivity during 16 years at one institution using methods similar to ours . They did not specifically evaluate the effects of the 80-hour work week, but separated residents into an early group (1990-1996) and a late group (1999-2005). They showed productivity at their single university-based program with elective research time did not appear to deteriorate with time and noted a nonsignificant trend toward increased productivity during the final few years of their study. In other studies of single-institution general surgery programs, investigators have observed improved scores on in-training examinations after implementation of work-hour restrictions and no substantial change in the number of procedures performed [3, 9]; however, none of these studies examined publication rates.
When stratified by PGY, we observed a trend toward increased publications during the first and second PGYs after work-hour restrictions. Other studies have investigated the influence of work-hour restrictions on other facets of residency education and stratified data by PGY. Baskies et al. analyzed the operative logs of 109 consecutive orthopaedic surgery residents from 2000 through 2006 at New York University's orthopaedic resident program . They determined that after implementation of work-hour policies, there was no difference in the operative volume for PGY 2, 3, or 4 residents, and there was an increase in the operative volume for PGY 5 residents. We propose that work-hour restrictions may grant more academic opportunity for junior residents who likely had limited time for such pursuits before the 80-hour work week. We observed no difference in publications when analyzed by postgraduate year and therefore believe residents further in their training (PGY 3 and higher) were less affected by work-hour restrictions.
Our data suggest residents who conducted research during medical school and have advanced degrees are more likely to publish manuscripts. After adjusting for these variables, we still found increased odds of a resident publishing a paper in any given year after implementation of work-hour regulations. Although a couple studies indicate that MD/PhD students have a strong interest in becoming clinician-scientists [1, 2], to our knowledge, no study specifically evaluates medical school publications or advanced degrees as predictors of publications during residency.
Work-hour restrictions have generated considerable concern regarding orthopaedic resident education. We showed that work-hour restrictions may provide residents with additional time and/or opportunity to increase productivity in academic pursuits, specifically in generating peer-reviewed publications. Although our study indicates that since implementation of work-hour restrictions, resident publications have increased, more long-term evaluation of publications during the 80-hour work week is necessary to determine whether these improvements in publication rates will be sustained.
1. Ahn, J., Man, LX., Wanderer, J., Bernstein, J. and Iannotti, JP. The future of the orthopaedic clinician-scientist. Part I: The potential role of MD-PhD students considering orthopaedic surgery. J Bone Joint Surg Am.
2008; 90: 1794-1799. 10.2106/JBJS.G.00460
2. Ahn, J., Watt, CD., Greeley, SA. and Bernstein, J. MD-PhD students in a major training program show strong interest in becoming surgeon-scientists. Clin Orthop Relat Res.
2004; 425: 258-263. 10.1097/00003086-200408000-00037
3. Barden, CB., Specht, MC., McCarter, MD., Daly, JM. and Fahey, TJ III. Effects of limited work hours on surgical training. J Am Coll Surg.
2002; 195: 531-538. 10.1016/S1072-7515(02)01242-5
4. Baskies, MA., Ruchelsman, DE., Capeci, CM., Zuckerman, JD. and Egol, KA. Operative experience in an orthopaedic surgery residency program: the effect of work-hour restrictions. J Bone Joint Surg Am.
2008; 90: 924-927. 10.2106/JBJS.G.00918
5. Bhandari, M., Devereaux, PJ., Guyatt, GH., Cook, DJ., Swiontkowski, MF., Sprague, S. and Schemitsch, EH. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am.
2002; 84: 615-621. 10.1302/0301-620X.84B7.12532
6. DeMola, PM., Hill, DL., Rogers, K. and Abboud, JA. Publication rate of abstracts presented at the shoulder and elbow session of the American Academy of Orthopaedic Surgery. Clin Orthop Relat Res.
2009; 467: 1629-1633. 10.1007/s11999-008-0474-2
7. Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res
. 2009 Nov 24. [Epub ahead of print].
8. Elder, NC. and Blake, RL Jr. Publication patterns of presentations at the Society of Teachers of Family Medicine and North American Primary Care Research Group annual meetings. Fam Med.
1994; 26: 352-355.
9. Elliott, ST. and Lee, ES. Surgical resident research productivity over 16 years. J Surg Res.
2009; 153: 148-151. 10.1016/j.jss.2008.03.029
10. Gavazza, JB., Foulkes, GD. and Meals, RA. Publication pattern of papers presented at the American Society for Surgery of the Hand annual meeting. J Hand Surg Am.
1996; 21: 742-745. 10.1016/S0363-5023(96)80185-7
11. Hassett, JM., Nawotniak, R., Cummiskey, D., Berger, R., Posner, A., Seibel, R. and Hoover, E. Maintaining outcomes in a surgical residency while complying with resident working hour regulations. Surgery.
2002; 132: 635-639. 10.1067/msy.2002.127542
12. Kohn, KT., Corrigan, JM. and Donaldson, MS. To Err Is Human: Building a Safer Health System
Washington, DC: National Academy Press; 1999.
13. Kusuma, SK., Mehta, S., Sirkin, M., Yates, AJ., Miclau, T., Templeton, KJ. and Friedlaender, GE. Measuring the attitudes and impact of the eighty-hour workweek rules on orthopaedic surgery residents. J Bone Joint Surg Am.
2007; 89: 679-685. 10.2106/JBJS.F.00526
14. NY Comp. Codes R & Regs. Section 405.4.2002.
15. Portney, LG. and Watkins, MP. Foundations of Clinical Research Applications to Practice: Appendix C Power Analysis and Determination of Sample Size
Stamford, CT: Appleton and Lange; 1993: 651-661.
16. Report of the New York State Department of Health Ad Hoc Advisory Committee on Emergency Services. Supervision and Residents' Working Conditions.
Albany, NY: New York State Department of Health; 1987 Oct 7.
17. Suarez-Almazor, ME., Belseck, E., Homik, J., Dorgan, M. and Ramos-Remus, C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials.
2000; 21: 476-487. 10.1016/S0197-2456(00)00067-2
© 2010 Lippincott Williams & Wilkins, Inc.
18. Wilkins, T., Gillies, RA. and Davies, K. EMBASE versus MEDLINE for family medicine searches: can MEDLINE searches find the forest or a tree? Can Fam Physician.
2005; 51: 848-849.