Secondary Logo

An Elective Research Year in Orthopaedic Residency: How Does One Measure Its Outcome and Define Its Success?

Segal, Lee, S; Black, Kevin, P; Schwentker, Edwards, P; Pellegrini, Vincent, D

Clinical Orthopaedics and Related Research: August 2006 - Volume 449 - Issue - p 89-94
doi: 10.1097/01.blo.0000224059.17120.cb
SECTION I: SYMPOSIUM I: C. T. Brighton/ABJS Workshop on Orthopaedic Education
Free
SDC

The concept of an elective research year during orthopaedic residency training is attractive to residents and faculty involved in graduate medical education. Measuring the success and outcomes of a research residency year remains poorly defined. The goal of this study is to evaluate the role and effect of the elective resident research year. How does one determine or define the “success” of a resident research year? Does the research residency year encourage residents to become clinician-scientists? A previously published questionnaire was mailed out to the 93 residents who completed their orthopaedic residency training between 1976 and 2005. The response rate was 70%. The majority of residents went into private practice (91.2%). In comparing residents with a research year to those without, no difference was noted in residents entering private or academic practice, or completing a fellowship. The research residents had a greater mean number of publications cited in PubMed. The mean number of publications after residency was similar. Exposure to an elective year of research did not appear to positively influence residents to enter a career in academic medicine.

From the *Department of Orthopaedics and Rehabilitation, The Milton S. Hershey Medical Center and The Pennsylvania State University College of Medicine, Hershey, Pennsylvania, and the †Department of Orthopaedic Surgery, University of Maryland School of Medicine, Baltimore, Maryland. Each author certifies that he has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article. Each author certifies that his institution has waived approval for the human protocol for this investigation and that all investigations were conducted in conformity with ethical principles of research.

Correspondence to: Lee S. Segal, MD, Department of Orthopaedics and Rehabilitation, Box 850, The Milton S. Hershey Medical Center, The Pennsylvania State University College of Medicine, Hershey, PA 17033. Phone: 717-531-4800; Fax: 717-531-0126; E-mail: lsegal@psu.edu.

The role of the resident research year in graduate medical training remains poorly defined. On the one hand, the research year provides an invaluable opportunity for the resident to take time off from the busy clinical years of training to learn the principles and often complex methodologies and study design in both basic and clinical research.10 But the resident research year may also function as a means of training future clinician-scientists, an “endangered species” in academic medicine.12 The increased presence of managed care in academic medical centers has resulted in marked reductions of institutional funding and time available for faculty research.1,4,7,10,11 There is growing pressure to increase clinical activities in academic medicine, reflected by the statistic that presently more than half of academic surgeons spend less than 20% of their efforts in research.1 An increasing concern is the lack of modeling and mentoring within academic medical centers to foster the development of future clinician-scientists.

What is the motivation for residents to participate in an elective research year, spending time away from their clinical residency training, and ultimately extending their residency? Some residents may be interested in pursuing a career in academic medicine. For other residents it may be a great learning experience, helping them determine if they wish to participate in future research. It may give them time to be academically productive, enabling them to be more competitive for a highly sought after subspecialty fellowship or academic appointment.2 Disingenuously for some, it may represent a “back door” entrance into a competitive residency training program.

We evaluated the role and effect of the elective resident research year on the residents in our residency training program and on the department collectively. The residents who chose to participate in the elective research year requested spending an additional year to better understand the scientific process and design during their residency, or had matched into the research/clinical position when it was offered by the department. The primary questions addressed in this study were the following: (1) How does one determine or define the “success” of a resident research year? (2) Does it have consequences on the subsequent career choices for the resident? (3) Does it encourage residents to become clinician-scientists?

Back to Top | Article Outline

MATERIALS AND METHODS

The 93 residents who completed their orthopaedic residency training between 1976 and 2005 at the Penn State University College of Medicine and the Milton S. Hershey Medical Center received a questionnaire5 requesting information on: (1) demographic data which including where the residents established their practice in relationship to their training program; (2) distribution of professional time spent on different activities such as clinical or academic practice, clinical or basic science research, teaching, and administrative duties; (3) fellowships; (4) ranking of interest in clinical practice, academic practice, clinical or basic research at various times of their medical training; and (5) ranking of factors affecting their participation in medical research. Phone calls and repeat faxed transmission of the questionnaires were sent out to non-responders 2 weeks after the initial mailing to increase the response rate. Other information regarding the residents who completed the residency program was obtained from the department's alumni records and personal knowledge of the graduated residents.

The residents were asked to provide data on the amount of time devoted to various professional activities. The percentages were subdivided to compare residents before and after the resident research year was initiated, to compare the research residents and the clinical residents, and to compare those residents in academic versus private practices. We asked the question, “At different times during your training and career, how would you have described your career goals?”5 To assess how these changed over time, a scale from 1 to 4 was used to determine relative interests in clinical practice versus academic practice and clinical versus basic research at different stages of their professional orthopaedic career.

There were 15 research residents since 1986 (including the three most recent residents not yet having completed the program), and 81 clinical residents. Three research residents elected to lengthen their research experience to 2 years. Only one of the residents was female. Ninety-one of the residents had a doctorate of medicine degree only, and two had additional degrees (engineering, physical therapy). The mean age at time of questionnaire was 44.7 years (range, 31-59 years). The mean age at completion of the residency was 32.2 years (range, 29-41 years). The resident research questionnaires were returned from 65 of 93 (70%) former orthopaedic residents who graduated from the program.

We determined the academic productivity of the residents by performing PUBMED searches for all of the residents' citations, both clinical and research residents. Publication data only was also obtained from three research residents who have not completed the residency training program (one is scheduled to graduate in 2007 and two transferred to a different residency training program in the institution). Data from the questionnaires were not collected from these three residents. Publications both during and after residency training were identified. We elected to not collect other measures of academic productivity such as book chapters, non-peer reviewed articles, and studies presented but not published.

Comparisons were made between the residents who competed the resident research year and the clinical residents, between research residents spending 1 and 2 years for their research experience, between residents who responded to the questionnaire and residents who did not respond to the questionnaire, and between residents graduating from the orthopaedic residency training program before and after the establishment of the resident research year. Statistical analysis was provided by our health evaluation sciences department, and performed using Fisher's exact test analysis for categorical data, Mann-Whitney test for nonparametric data, and the Wilcoxon Rank test for nonpaired variables, with significance defined at the p < 0.05 level.

Back to Top | Article Outline

RESULTS

Thirty-four percent of the 93 residents remained in the Commonwealth of Pennsylvania after their residency, and another 18% in bordering states. A large majority (91.2%) of residents went into private practice after their residency and/or fellowship training; only eight have entered academic practices, and one is in the military. Sixty percent of the residents completed a fellowship following their residency training in the following subspecialties: adult reconstruction (1), spine (14), pediatrics (4), trauma (7), hand (3), sports medicine (11), foot and ankle (2), musculo-skeletal oncology (2), and pathokinesiology (1). This data was obtained from both the residents who responded to the questionnaire, as well as department alumni records.

In comparing clinical residents versus research residents, no statistical difference was noted in residents entering academic practice or private practice, or those residents obtaining additional fellowship training (Table 1, Fig 1). Of the two residents with advanced degrees, one entered academic practice.

TABLE 1

TABLE 1

Fig 1

Fig 1

The research residents had a greater (p = 0.0005) total mean number of publications (5.4) compared with clinical residents (2.3) (Table 2). The average number of publications cited in PUBMED was also greater (p = 0.002) for research residents during residency (3.3 versus 0.98). The average number of publications after residency was similar (research residents = 1.83; clinical residents 1.26). The mean number of publications by clinical residents before and after the initiation of the resident research year was similar (before = 0.89 ± 1.48; after = 1.05 ± 1.39; p = 0.52).

TABLE 2

TABLE 2

Three of the research residents extended their RRY to 2 years. The average number of publications for the three residents having 2 research years (11.9) was higher (p = 0.056) than for nine having only one resident research year (4.5). Two of the nine residents with one research year went into academics, compared to one of three residents having two research years.

We found no difference between the residents who responded to the questionnaire and the nonresponders when considering residents entering academic or private practice, or the total number of publications. Residents who graduated before the resident research year was started had a lower response rate than those residents graduating after (before 17/37, 46%; after 48/56, 86%; p = 0.0001) (Table 3).

TABLE 3

TABLE 3

The research residents devoted more of their professional time to academic practice (mean research residents = 23.5%; clinical residents = 7.1%) and more time for teaching (mean research residents = 7.0%; clinical residents = 4.1%) (Table 4).

TABLE 4

TABLE 4

In response to the question posed, “At different times during your training and career, how would you have described your career goals?”5 the entire cohort of residents (evaluating the fraction of respondents) noted an increase in clinical practice, a decrease in interest in academic practice as individuals begin their clinical training (residency and fellowship), and the relative static degree of interest in clinical and basic research over time (Fig 2).

Fig 2

Fig 2

Back to Top | Article Outline

DISCUSSION

The concept and intent of an elective research year during orthopaedic residency training is attractive to residents and faculty involved in graduate medical education. But what is the outcome of the research residency year, and how is it measured? In this study, we attempted to answer some of the questions surrounding the elective resident research year in orthopaedics. The two major questions posed in this retrospective study were: (1) How does one determine or define the “success” of a resident research year? (2) Does it have consequences on the subsequent career choices for the resident and does it encourage residents to become clinician-scientists? We focused our efforts on the residents, as opposed to the faculty and their perceptions of the success of the resident research year.

The study limitations include the reliability and validity of the questionnaire5 distributed to the orthopaedic residents and the small number of research residents compared to the clinical research residents which may have impacted on our statistical findings. A large number of residents graduating before the resident research year was started did not respond to the questionnaire. This may have impacted the results of this study. The statistical analysis only indicates associations between variables rather than causal relationships. The data and inferences regarding causality reported in this study must be cautiously interpreted.8 The data presented in this report originate from a clinically oriented orthopaedic residency training program. Future research to obtain and interpret findings from an academic or research-focused residency training program should be encouraged.

Our department initiated an elective resident research year in 1986 to provide the residents dedicated time to pursue either basic science or clinical research. Has our experience with a resident research year been successful? The research residents published more papers during their residency, which likely increased the number of national presentations and publications for faculty and the department. This has helped to increase the exposure and improve the department's reputation on a national level. The resident research year, however, did not facilitate more published papers by the clinical residents as was hypothesized before the study, when the number of publications by clinical residents was analyzed before and after the resident research year was started. In a general surgery survey, only 28% of general surgical residents and 20% of chairman thought the number of articles published should be the most important way to determine success of a resident research experience.10 From a departmental and an individual perspective, it is possible that the research residents secured highly competitive fellowships by having more publications and presentations, and thus more impressive curriculum vita. Interestingly, the elective resident research year did not increase the number of residents entering postgraduate fellowships when the research residents were compared to the clinical residents. The success of the resident research year could also be defined by the percentage of residents continuing to contribute to the body of orthopaedic literature The research residents published more papers during their residency, but not after their residency training compared to the clinical residents.

Academic medicine is in an “embattled state,” as increased penetration of managed care into academic medical centers has resulted in decreased institutional funding and corresponding pressure to increase clinical activities.1,4,7 There is growing concern clinician-scientists are an “endangered species” and that the quality of clinical research training is highly questionable.7,12 The success of the resident research year may be defined by the percentage of residents entering academic practice. The results of this study suggest that our elective resident research year has not been successful based on these criteria. The resident research year did not statistically increase the number of residents directing their professional careers toward academic medicine. However, a positive trend was noted in our research residents toward academic careers (25% research residents versus 6% clinical residents). The factors influencing career choices are highly complex, and likely involve more than just spending one or two years devoted to clinical or basic research. These include contemporary pressures such as financial, societal, and generational (lifestyle) issues.5,9

Does the amount of time or number of years doing research correlate with the likelihood of choosing an academic career? Three of the fifteen research residents elected to spend two years for their research experience. Only one of these residents went into an academic practice, compared with two residents who spent one year doing research. This finding differs from a previous study from the general surgery literature, which reported residents performing at least 2 years of research were more likely to become academicians.11 The mean total of publications (both during and after residency) was higher for this group of residents. An earlier study also noted that residents who spend 2 or more years in the lab were much more productive (had more publications), and the desire to seek research opportunities was more important than the length of time spent doing research in predicting future academic activity.2

The questionnaire distributed to the former orthopaedic residents was one previously used to evaluate those factors influencing career decisions and research activities in a cohort of pediatric residents from an academically oriented training program.5 This enabled an interesting comparison to the current study, focusing on orthopaedic residents from a clinically oriented residency training program. Our response rate was 70%, which compares favorably to other studies.2,5,11 The majority of the pediatric residents went into academic medicine (73%). Their professional time was distributed in 47% clinical care, 25% administration and teaching, and 28% in basic or clinical research. The orthopaedic residents from our program predominantly pursued careers in private practice (91%). Greco et al3 evaluated career development in general surgery, and found private practice continues to be the goal of most residents (65%) in both university and independent training programs. Their professional time was divided into 88% clinical care, 10% administration and teaching, and 2% basic and clinical research. When comparing development of interests over time, the most noticeable difference between each group was the marked increased interest in private practice during residency and fellowship for the orthopaedic cohort. Neacy et al8 also noted interest in an academic career in emergency medicine decreases as the residency progresses. There is also a much greater financial disincentive for an orthopaedic surgeon to pursue a career in academic medicine or as a clinician-scientist. Relative to non-surgical fields, the difference between salary as an orthopaedic clinician (private practice) and salary that can be obtained from grant funding is much larger. This may have assumed an increasingly important role in the past 15 years as the debt load of medical school graduates has become larger on a yearly basis. It may also be that the salary of an orthopaedic surgeon results in a selection bias in those that pursue this as a career, and income is a primary factor for many medical students in pursuing a career as an orthopaedic surgeon.

The financial constraints facing all academic health centers make it critical that all our academic missions, including the resident research year, be closely evaluated. This includes the efficient use of resources for training academic surgeons.2 Should research be mandatory for all residents? Dunn et al2 suggest that surgical departments must be highly selective in choosing residents committed to research and an academic career.

Several studies have evaluated specific predictors of resident research productivity and future research endeavors.1,2,6,8,12 The strongest predictor of research in residency was having one or more additional degrees, such as master of science, master of public health, or doctorate of philosophy, and having a publication before completing residency correlated with subsequent research productivity.12 Participation in research projects during medical school, length of training program, current year of postgraduate training, and perceived level of overall academic skill preparation were factors independently associated with an interest in an academic career.8 Lessin et al6 suggested research was more common among those general surgery residents pursuing subspecialty training into highly competitive fields such as pediatric surgery. Dunn et al2 observed residents with a strong desire to conduct research (ie, had requested research experience) were more productive in the lab and were three times more likely to continue their research efforts in academic surgery after residency. Collins et al1 found residents who train at one of the 20 academic departments that receive the most National Institutes of Health (NIH) research funding more frequently choose careers in research than those that train and matriculate elsewhere.

It has been our impression that fostering the development of academic orthopaedic surgeons was not achieved by our elective resident research year (although a positive trend was noted). It may be that our research year was not a rewarding year for the residents or was constructed in such a manner that they were not stimulated to pursue a career in academic medicine or as a clinician-scientist. There has been little structure to the year itself. In the year prior to going in to the lab, residents were expected to identify either a clinical or basic science faculty member to mentor them and with whom to do projects with. During the time of this study, our entire clinical faculty spent at least 80% of their time with clinical activities. Although each of the faculty members are expected to be academically productive, none were clinician-scientists and able or willing to commit 50% or more of their time to research. It may be that this lack of a clinician-scientist role model in the department resulted in a negative stimulus for a career in academic medicine. One other consideration is the timing of the resident research year, often done early during their residency training (following their PGY 1 or PGY II year of training). After completing the research year, the residents may lose interest in academic endeavors as they become immersed in their final years of clinical training during orthopaedic residency.

Similar to the recommendations proposed by the NIH,7 we have developed a structured post-baccalaureate certificate program in academic medicine, to provide the residents with the tools necessary to succeed after residency in an academic environment. This is scheduled to begin in the 2006-2007 academic year, and has been supported by a one time Orthopaedic Research Education Foundation grant. The residents will be given formal training and hands-on experience in grant and manuscript writing, basic science techniques (cellular/molecular), methodology to conduct clinical research trials, and leadership development. During this year, the resident will complete 16 hours (500 level) of approved coursework in clinical and basic science research in the departments of cellular and molecular biology, bioengineering, and health evaluation science at the Pennsylvania State University and complete an independent research project intended for presentation at a national conference. In addition, the resident will be given the opportunity to shadow an academic clinician-scientist at an outside institution for 2 weeks to provide additional mentorship and begin developing a network to promote a future career in academic medicine.

Acknowledgments

The authors would like to thank the former and future residents of the orthopaedic residency training program at the Pennsylvania State University College of Medicine; Marnie M. Saunders, PhD, for her efforts with the post baccalaureate certificate program in academic medicine; and acknowledge the contributions of Robert B. Greer III, MD, the first chairman of the Division of Orthopaedics whose vision and efforts developed the residency training program and the research residency year.

Back to Top | Article Outline

References

1. Collins J, Meyer R, Dawson JR. Getting residents involved in research: a challenge in the era of managed care. Acad Med. 1999; 74:1155-1157.
2. Dunn JC, Lai EC, Brooks CM, Stabile BE, Fonkalsrud EW. The outcome of research training during surgical residency. J Pediatr Surg. 1998;33:363-364.
3. Greco RS, Donetz AP, Mackenzie JW, Brolin RE, Trooskin SZ. Career development of residents in university and independent training programs: the influence of special training fellowships, type of practice, specialization, and research. Surgery. 1986;100: 312-320.
4. Ko CY, Whang EE, Longmire WP Jr. McFadden DW. Improving the surgeon's participation in research: is it a problem of training or priority? J Surg Res. 2000;91:5-8.
5. Ledley FD, Lovejoy FH Jr. Factors influencing the interests, career paths, and research activities of recent graduates from an academic, pediatric residency program. Pediatrics. 1993;92:436-441.
6. Lessin MS, Klein MD. Does research during general surgery residency correlate with academic pursuits after pediatric surgery residency? J Pediatr Surg. 1995;30:1310-1313.
7. Nathan DG. Clinical Research. Perceptions, reality, and proposed solutions. JAMA. 1998;280:1427-1431.
8. Neacy K, Stern SA, Kim HM, Dronen SC. Resident perception of academic skills training and impact on academic career choice. Acad Emerg Med. 2000;7:1408-1415.
9. Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical student research in the development of physician-scientists. J Investig Med. 2003;51:149-156.
10. Souba WW, Tanabe KK, Gadd MA, Smith BL, Bushman MS. Attitudes and opinions toward surgical research. A survey of surgical residents and their chairpersons. Ann Surg. 1996;223:377-383.
11. Thakur A, Thakur V, Fonkalsrud EW, Singh S, Buchmiller TL. The outcome of research training during surgical residency. J Surg Res. 2000;90:10-12.
12. Ullrich N, Botelho CA, Hibberd P, Bernstein HH. Research during pediatric residency: Predictors and resident-determined influences. Acad Med. 2003;78:1253-1258.
© 2006 Lippincott Williams & Wilkins, Inc.