Journal Logo

AOA Critical Issues in Education

Restructuring of an Orthopaedic Surgery Residency Research Rotation Correlates with Increased Academic Productivity in Teaching Faculty

Granger, Caroline J. BS1; Rothy, Alexander MD2; Nigh, Evan MD3; Hernandez, Victor H. MD2; Baraga, Michael MD, FAOA2; Conway, Sheila Ann MD, FAOA2,a

Author Information
doi: 10.2106/JBJS.OA.20.00012
  • Open
  • Disclosures

Abstract

Academic productivity and research contribution are integral components of residency education1-3. Participation in scholarly activity is a core requirement of all residency programs accredited by the Accreditation Council for Graduate Medical Education (ACGME)1,2 and the ACGME stipulates, “the sponsoring institution and program should allocate adequate educational resources to facilitate resident involvement in research.” These accreditation requirements highlight the importance of training orthopaedic surgeons to possess a fundamental understanding of the scientific method to critically evaluate basic science discoveries and translate them into clinical practice6,7. However, the ACGME does not specifically define the scholarly expectation of residents at graduation nor does it define “adequate educational resources.” As such, significant variability exists among residency programs regarding resource allocation, time, structure, expectation, and resident scholarly activity. Orthopaedic surgery residency programs' individual research requirements range from an additional year of dedicated research to little or no dedicated research time3. Regardless, the research experience during residency should not only be structured to fulfill accreditation requirements but should also focus on fostering critical thinking skills and intellectual curiosity of residents9.

Fig. 1
Fig. 1:
Graph depicting the mean impact factor sum per faculty member by year. One hundred three curriculum vitae were reviewed on 23 faculty members. A notable and statistically significant increase was observed across this academic parameter over 5 years. X-axis designates the year, and Y-axis designates impact factor as an annual sum.
Fig. 2
Fig. 2:
Graph depicting the mean number of podiums per faculty member by year. One hundred three curriculum vitae were reviewed on 23 faculty members. A notable increase was observed across this academic parameter, reaching a peak in 2016. X-axis designates the year, and Y-axis designates the number of podium presentations by year.
Fig. 3
Fig. 3:
Graph depicting the mean number of posters per faculty member by year. One hundred three curriculum vitae were reviewed on 23 faculty members. A notable increase was observed across this academic parameter over 5 years. X-axis designates the year, and Y-axis designates the number of posters by year.
Fig. 4
Fig. 4:
Graph depicting the mean number of publications per faculty member by year. One hundred three curriculum vitae were reviewed on 23 faculty members. A notable and statistically significant increase was observed across this academic parameter over 5 years. X-axis designates the year, and Y-axis designates the number of publications by year.

From 1987 through 2014, a single institution orthopaedic surgery residency research experience included 2 dedicated research rotations of 6 to 7 weeks duration, for a total of 3 months over 5 years. Despite this allocated research time, a minority of resident projects were completed to academic presentation or publication, contradicting data displaying that dedicated research time is correlated with increased research output10. In 2015, resident feedback and literature review were used to restructure the research curriculum with a goal of increasing academic productivity among graduating trainees11-14. This restructuring included the appointment of 2 clinical orthopaedic faculty to serve as codirectors of the rotation, implementation of strict rules regarding protected time, revised rotation curricula, use of multilevel resident research teams, bimonthly meetings, and creation of a centralized accessible research database.

Analysis of the curriculum vitae (CVs) of the graduating residents from 2015 to 2018 demonstrated a significant increase in all indicators of academic productivity within 2 years of implementation4,5. Although it is understood that these changes improved resident research productivity, the concomitant impact of this program on the productivity of clinical orthopaedic faculty is undetermined and may

provide an unanticipated benefit of this curriculum modification. The aim of this study was to investigate the impact of implementation of a restructured orthopaedic residency research curriculum on faculty academic productivity with a goal of providing a model for other programs to use for the purpose of enriching institutional research efficiency and caliber.

Methods

The CVs of all full-time orthopaedic teaching faculty at a single institution were collected and retrospectively reviewed from the start of 2014 through the end of 2018 to determine the number of indicators of academic productivity. These full-time faculty were core faculty for an ACGME-accredited orthopaedic surgery residency program of 7 residents per year. Of 27 full-time faculty members, 23 responded to our request for CVs with an 85% response rate. CVs were requested up to 10 times via email through the department of orthopaedic surgery. CVs were included per year of faculty status. Parameters of academic productivity were defined as peer-reviewed publications, impact factor as an annual sum, podium presentations, and poster presentations. There was no change in faculty research resources, and no specific incentives were put in place to encourage faculty research productivity over the study period. The faculty group had a minimal turnover during the study period. One professor retired in 2018, one associate professor left the institution 2017, and 2 assistant professors were on boarded in 2018.

Source of Funding

No funding external to the department was sought out or required for the completion of this study. No funding source played any role in this investigation.

Statistics

JMP 14 software (SAS Institute) was used for statistical analysis. One CV is defined as all events reported in 1 year to allow for standardization across data points. All CVs were recorded into a data matrix. Data points were categorized both by year and faculty member. To evaluate the effect of the research rotation on research productivity over yearly intervals, a repeated measures analysis of variance was completed. Sphericity was calculated for all measures of academic productivity, and the appropriate correction was applied to all sphericity violations as determined by Muller and Barton6,7. The results were statistically significant when the p value was less than <0.05.

Results

One hundred three CVs were reviewed on 23 faculty members. A notable steady increase was observed in all academic parameters over 5 years, with the exception of podium presentations, that reached a peak in 2016 (Table I, Figures 1–4). Sphericity was violated for all measures, and the Greenhouse-Geisser correction was applied. There was a significant effect of time after introduction of the research rotation on faculty summative impact factor (F(2.4, 31.5) = 3.67, p = 0.03) and journal publications (F(2.5,32.7) = 5.68, p = 0.005).

TABLE I - Mean Publication Type Per Faculty Member By Year*
Publication Type 2014 2015 2016 2017 2018 p Value
Impact factor 2.36 5.18 3.70 7.08 6.91 0.030
Publications 1.36 3.11 2.30 4.33 3.71 0.005
Podiums 0.43 0.67 1.45 0.57 0.90 0.152
Posters 1.14 1.11 2.00 1.90 2.52 0.215
*n = 103 curriculum vitae; 23 faculty members.
Denotes p < 0.05.

Discussion

This 5-year single institution analysis displayed an increase in all measured indicators of faculty productivity after the restructuring of an orthopaedic surgery residency program research rotation. An increase in peer-reviewed journal publications, journal impact factor, and poster and podium presentations occurred over the study period, indicating an improvement in both qualitative and quantitative indicators of productivity. No variables significantly decreased over the investigated time period. Podium presentations, peer-reviewed publications, and impact factor rose steadily throughout the study period, although poster presentations peaking in 2016. The lack of increase in poster presentation after 2016, with ongoing, parallel increase in podium and peer-reviewed publications, could indicate an increase in the quality of the research. Because most meetings offer podium or poster invites for abstracts, with podium reserved for the highest quality abstracts, podium and poster presentations could understandably have an inverse relationship.

Our group has previously reported that the restructuring of an established resident research rotation significantly increased the research productivity of graduating orthopaedic surgery residents4,5. The changes included the formation of trainee research teams, improved rotation structure and individual accountability, and enhanced research mentorship. Although an increase in resident research was the goal and the anticipated outcome from these curricular changes, the parallel increase in faculty academic productivity was unexpected. Certainly, the increased number of resident projects coauthored with teaching faculty over the study time period indicates that the research curriculum changes likely influenced the parallel rise in faculty academic productivity.

Resident research teams successfully increased resident engagement, coauthorship with trainees, and coauthorship with core faculty. As these research teams were built with trainees, not faculty, we do not believe that the rise in faculty productivity is explained by increased faculty coauthorship on projects. Furthermore, the increase in original, peer-reviewed publications and the rise in impact factor both indicate an increase in both the quality and quantity of faculty academic productivity. Although difficult to measure, we hypothesize that the success of the research restructuring is also attributable to the positive change in academic culture and renewed energy surrounding resident research restructuring.

Although protected time for resident research already existed before the aforementioned modifications, the structure of and mentorship during the rotation transformed dramatically after implementation. Konstantakos et al. previously reported that after the establishment of structured research teams, the academic productivity of orthopaedic residents increased, although their impact on faculty productivity was not mentioned8. The research team approach was successful in our program for several reasons. Consisting of residents at multiple postgraduate levels, these research teams were optimized to allow projects to move forward steadily and ameliorated the likelihood that projects would be seen to completion after chief resident graduation. Furthermore, the strict enforcement of dedicated research meetings ensured the accessibility of the codirectors to assist with project progress on a predictable, regular basis. Attendance at regular meetings and clearly communicated expectations also increase resident accountability for their respective projects and probably contributes to boosted faculty engagement by extension.

The time demands of faculty members' multiple responsibilities are often cited as the primary deterrent to perform research, leading to a lack of potential research mentors for residents1,9-11. The modern academic physician has 3 main responsibilities: delivering high-quality patient care, providing comprehensive resident and student education, and contributing to scholarship. The current financial climate presents these as competing forces with administrative pressure to generate more clinical revenue, increased regulatory requirements to spend time supervising residents, and decreased funding to pursue research endeavors12. As a result, academic physicians are placed in a difficult situation with seemingly little incentive and often few resources to engage in meaningful mentoring or research with residents.

At the same time, although academic institutions stress the importance of education and patient care, they operate in a competitive environment where prestige and access to funding is based on research productivity13. Similarly, the consideration for academic appointment, promotion, and tenure of faculty members is predominantly measured by research productivity and scholarly reputation13-15. The observed increase in scholarly achievement of our orthopaedic faculty, particularly the significant rise in impact factor and peer-reviewed publications, may have a positive impact on faculty promotion and institutional funding16,17. Increasing the quantity and quality of peer-reviewed publications in higher impact journals bolsters the reputation of the department and institution as a whole, paving the way for additional research funding16. Protected research time for residents allows faculty to serve as invaluable mentors to residents having a profound impact on resident satisfaction, personal development, career guidance, and research productivity13,15,18-21. Most importantly, the increase in scholarly productivity and discovery will advance the field of orthopaedics and improve the care of patients22.

This study has limitations. Because matching into orthopaedic residency has become increasingly more competitive, it may be surmised that orthopaedic residents of more recent classes had more experience and interest in pursuing research than in previous years; however, we performed demographics in this study to control for this potential confounder and address the possibility of this limitation1,9,23-25. To additionally address this limitation, one may compare our analysis of academic indicators of both coresidents and faculty from our program4,5. As demographic analysis of the resident groups did not change over the years regarding United States Medical Licensing Exam Step 1 or Alpha Omega Alpha status, we do not believe there was a significant shift in the resident candidates. Faculty turnover could likewise impact faculty productivity, but we observed minimal faculty transitions over the study time period. With only 3 years of postntervention data, it is unclear whether these changes are sustainable and ongoing study is underway to study the long-term impact of this educational intervention. This is of particular concern because the codirectors of the research rotation receive no incentive, specific support, or protected time for this time-consuming responsibility. Finally, this was a single institution study and may not be applicable to all orthopaedic residency programs because of the high variability of program structure, size, and resources.

Conclusion

Increased mentorship and improved structuring of an established orthopaedic surgery resident research rotation has resulted in a significant increase in academic productivity for the teaching faculty of the department at our teaching hospital. This increase parallels the expansion seen in orthopaedic resident productivity over the same time period, indicating that this residency program curricular advancement has positively impacted faculty academic development and overall research productivity of the department. Ideas for restructure and relevant findings from this study may be applied to various orthopaedics programs with a goal of enhancing institutional research output.

References

1. ACGME program requirements for graduate medical education in orthopaedic surgery. 2016. Available at: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/260_orthopaedic_surgery_2016.pdf. Accessed March 18, 2019.
2. 2017 ACGME general program requirements. Available at: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed November 8, 2019.
3. Bernstein J, Ahn J, Iannotti JP, Brighton CT. The required research rotation in residency: the University of Pennsylvania experience, 1978-1993. Clin Orthop Relat Res. 2006;449:95-9.
4. Conway S. Increased academic productivity in graduating orthopaedic surgery residents following restructuring of a research rotation. 2019. Paper presented at: 2019 Annual Meeting of the Florida Orthopaedic Society; June 23, 2019; University of Miami Miller School of Medicine, Miami, FL.
5. Sabeh KFC, Spivey V, Hernandez V, Conway SA. Orthopaedic surgery resident research rotation: the impact of a structured rotation and faculty leadership on scholarly activity. 2018. Paper presented at: American Orthopaedic Association Annual Leadership Meeting; June 26–30, 2018; Boston, MA.
6. Gribbin MJ, Chi YY, Stewart PW, Muller KE. Confidence regions for repeated measures ANOVA power curves based on estimated covariance. BMC Med Res Methodol. 2013;13:57.
7. Muller KE, Barton CN. Approximate power for repeated-measures ANOVA lacking sphericity. J Am Stat Assoc. 1986;84(406):549-55.
8. Konstantakos EK, Laughlin RT, Markert RJ, Crosby LA. Assuring the research competence of orthopedic graduates. J Surg Educ. 2010;67(3):129-34.
9. Ahn J, Donegan DJ, Lawrence JTR, Halpern SD, Mehta S. The future of the orthopaedic clinician-scientist: part II: identification of factors that may influence orthopaedic residents' intent to perform research. J Bone Joint Surg Am. 2010;92(4):1041-6.
10. Brand RA, Hannafin JA. The environment of the successful clinician-scientist. Clin Orthop Relat Res. 2006;449:67-71.
11. Rosier RN. Institutional barriers to the orthopaedic clinician-scientist. Clin Orthop Relat Res. 2006;449:159-64.
12. Steinmann AF, Dy NM, Kane GC, Kennedy JI Jr, Silbiger S, Sharma N, Rifkin W. The modern teaching physician—responsibilities and challenges: an APDIM white paper. Am J Med. 2009;122(7):692-7.
13. Gofton W, Regehr G. Factors in optimizing the learning environment for surgical training. Clin Orthop Relat Res. 2006;449:100-7.
14. Boyer Commission. Reinventing Undergraduate Education: A Blueprint for America's Research Universities. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching; 2008.
15. Flint JH, Jahangir AA, Browner BD, Mehta S. The value of mentorship in orthopaedic surgery resident education: the residents' perspective. J Bone Joint Surg Am. 2009;91(4):1017-22.
16. Zhu EY, Shemesh S, Moucha CS. The association between scholarly impact and National Institutes of Health funding in orthopaedic surgery. Bull NYU Hosp Joint Dis. 2017;75(4):257-63.
17. Ence AK, Cope SR, Holliday EB, Somerson JS. Publication productivity and experience: factors associated with academic rank among orthopaedic surgery faculty in the United States. J Bone Joint Surg Am. 2016;98(10):e41.
18. Choi AMK, Moon JE, Steinecke A, Prescott JE. Developing a culture of mentorship to strengthen academic medical centers. Acad Med. 2019;94(5):630-3.
19. Fan B, Raman T, Benevenia J, Berberian W. Predictors of career choice in academic medicine among orthopaedic surgery residents. J Bone Joint Surg Am. 2014;96(1):e6.
20. Levine WN, Braman JP, Gelberman RH, Black KP. Mentorship in orthopaedic surgery—road map to success for the mentor and the mentee: AOA critical issues. J Bone Joint Surg Am. 2013;95(9):e59.
21. Sambunjak D, Straus SE, Marušić A. Mentoring in academic medicine: a systematic review. JAMA. 2006;296(9):1103-15.
22. Einhorn TA. Funding the mandate for the orthopaedic clinician scientist. Clin Orthop Relat Res. 2006;449:76-80.
23. DePasse JM, Palumbo MA, Eberson CP, Daniels AH. Academic characteristics of orthopaedic surgery residency applicants from 2007 to 2014. J Bone Joint Surg Am. 2016;98(9):788-95.
24. Schrock JB, Kraeutler MJ, Dayton MR, McCarty EC. A comparison of matched and unmatched orthopaedic surgery residency applicants from 2006 to 2014: data from the National Resident Matching Program. J Bone Joint Surg Am. 2017;99(1):e1.
25. The National Resident Matching Program. Charting outcomes in the match: U.S. allopathic seniors. Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2018 main residency match. 2018. Available at: http://www.nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-in-the-Match-2018-Seniors.pdf. Accessed November 8, 2019.

Supplemental Digital Content

Copyright © 2020 The Authors. Published by The Journal of Bone and Joint Surgery, Incorporated. All rights reserved.