Factors Associated with the Success and Timing of Orthopaedic Surgery Resident Research Thesis Project Publication

Abstract

Introduction: 

Scholarship and research are important aspects of orthopaedic surgery training. Many orthopaedic surgery residency programs have developed dedicated research curricula, often culminating in a capstone thesis project with the intended goal of peer-reviewed publication. However, data on the success of these programs are scarce. The purpose of the current study was to determine the success rate and time to publication of resident research thesis projects at our own institution while evaluating factors associated with these outcomes.

Methods: 

Resident research thesis projects performed over the past 15 years were aggregated and reviewed. Additional data regarding the projects and former trainees who performed them were collected using public resources which included measures of current academic aptitude (i.e., H-index and number of publications) as well as project and publication characteristics. Cox and linear regression analyses were conducted to assess the relation between numerous predictor variables and the success and time to publication. All analyses were conducted at the 95% confidence interval (CI) level.

Results: 

Sixty-eight (n = 68; 83%) resident research thesis projects were published an average of 2,582.8 days, or roughly 7 years from the start of their residency training. Graduate adjusted H-index was associated with increased success and decreased time to publication (hazard ratio 1.183 [95% CI: 1.059-1.322], p = 0.003). A lower journal impact factor was associated with taking significantly shorter time to reach publication (F(1,66) = 7.839, p = 0.007; Β₁ = 146.45, p = 0.007). Study type (clinical vs. laboratory), posttraining practice setting (academic vs. private), and whether the research topic was within the same area of the trainee's matched fellowship(s) did not predict publication success.

Discussion/Conclusion: 

Over the past 15 years, 83% of orthopaedic resident research thesis projects at our institution were published. A higher adjusted H-index was associated with greater completion and faster timing to publication. A lower journal impact factor was also associated with quicker publication. These data highlight the publication metrics of a formalized resident research program and identify factors associated with success and timing of publication.

Introduction

The Accreditation Council for Graduate Medical Education (ACGME) requires all orthopaedic surgery trainees to demonstrate academic scholarship through participation in sponsored research¹. However, other resident responsibilities such as balancing clinical duties often precludes time for consistent, meaningful engagement in research efforts². As a result, some programs have established dedicated structured research programs and rotations, which have generally resulted in increased academic productivity for trainees^3-6.

The Department of Orthopaedic Surgery at Washington University in St. Louis developed and continues to refine an Orthopaedic Residency Research Program⁷. The program requires a thesis research project to be completed by each trainee as a condition of graduation. The goal of the current program is 2-fold: (1) for residents to develop and apply critical thinking and analysis skills to become better consumers of scientific research and (2) for residents to make a meaningful contribution to the scientific literature. Protected, dedicated research rotations and semiannual lectures on relevant research topics are now a part of the overall curriculum. Specific deliverables are expected at defined points throughout residency training to ensure sufficient time to complete a meaningful research project. However, inevitably not all projects make it to completion and/or publication. In a recent study at another institution, 63% of orthopaedic resident thesis projects were ultimately published⁸. Of that cohort, about half (46.7%) undertook a thesis project in the same subspecialty they matched in for fellowship⁸.

The purpose of the current study was to evaluate the success and time to publication of research thesis projects performed by orthopaedic surgery trainees at our institution over the past 15 years since the inception of the program. A secondary aim was to identify factors associated with publication success and timing. We hypothesized that the overall success rate would be high (i.e., >80%) for publication of thesis projects and that timing to publication would be shorter if the thesis project was in the same subspecialty area as the trainees' matched fellowship.

Methods

This was a cross-sectional study of publicly available data. Institutional review board approval was granted with exempt status (IRB #202206103).

Subject Inclusion

Subjects included all graduated orthopaedic surgery residents at Washington University in St. Louis over the past 15 years (2007-2022). An email was sent to former trainees with queries regarding their thesis presentation title, whether the thesis project was ever published, and the reference for the publication (if published). The response rate was 100% (90/90).

Data Elements

For each former trainee and their thesis project, additional data were collected using public resources and compiled in a Microsoft Excel Spreadsheet (Microsoft). These elements are delineated below in Table I.

TABLE I - Resident and Publication Characteristics

Resident/Publication Characteristic	Levels/Information
Resident ID	ID
Year of graduation	Year
Number of years in practice	Number of years practicing after fellowship(s)
Current H-Index Adjusted H-Index	H-Index (Scopus) H-Index/number of years in practice
Current number of publications Adjusted number of publications	Number of publications (Scopus) Number of publications/number of years in practice
Current practice	Academic Private
Subspecialty	General practice Foot and ankle Hand Adult reconstruction Oncology Pediatrics Shoulder and elbow Spine Sports medicine Trauma
Thesis project resulted in publication	Yes No
Publication date	MM/DD/YYYY
Publication latency (days)	Publication date (MM/DD/YY) – Start of residency (07/01/YY)
Publication journal	Journal
Journal impact factor adjusted for year	Year adjusted impact factor (SCI journal)
Project subspecialty topic(s)	General practice Basic science Foot and ankle Hand Adult reconstruction Oncology Pediatrics Shoulder and elbow Spine Sports medicine Trauma
Study type	Basic science (cadaver/laboratory) Clinical research
Thesis project area matches subspecialty	Yes No

Current H-index⁹, number of publications, and practice setting were collected in an attempt to correlate “academic aptitude” with the success and timing to publication of their resident research thesis projects. The H-index and publication count were derived in June 2022 using the database, Scopus¹⁰. Some individuals without a claimed profile had multiple automated profiles, and the one with the highest H-index/publication count was used for purposes of this study. These measures of academic productivity were adjusted by dividing each by the number of years in practice, formulating an “adjusted number of publications” and an “adjusted H-index” for purposes of comparing these metrics across classes. Google¹¹ was queried to find each former trainee's current practice setting. If they were practicing within a university health system and/or held a teaching appointment, they were considered in an academic practice. Each publication was readily found on PubMed¹² and/or the publishers or journal's website. The year adjusted impact factor of the journal published in was derived using SCI Journal¹³. If any of these variables were not abundantly clear or readily available for certain trainees and/or their research thesis projects, additional publicly available resources were used to capture or confirm these data points.

Statistical Analyses

Data were synthesized and summary statistics were reported including counts (%), means (SD), and medians (interquartile range; IQR) where appropriate. Univariate cox regression analyses were conducted to assess the relation between predictor variables and the success/latency of publication. The time dependent variable was the elapsed time it took to reach publication in number of days since starting residency. The event variable was successful publication and end date was that of successful publication. Censored subjects were those whose project never reached publication. For censored subjects, the end date function was the day that the statistical analyses were conducted in June 2022. If more than 1 univariate analysis reached a conventional level of statistical significance of p < 0.05, they were combined into a multivariate cox regression model. A separate linear regression was conducted to assess the relationship between year adjusted journal impact factor and publication latency (given that censored subjects never published their projects). All analyses were conducted at the 95% confidence interval (CI) level with the statistical software program IBM SPSS version 28.0 (IBM).

Results

Descriptive Statistics

Of the 90 subjects, 8 (8.9%) were still finishing their fellowship and were excluded from the analysis. The remaining 82 had been in practice for a median of 7 years (IQR: 4-11). Half (n = 41; 50%) held academic positions at the time of this study. The median H-index and adjusted H-index were 6.50 (IQR 4.0-10.25) and 1.40 (IQR 0.50-2.0), respectively. Median number of publications and adjusted number of publications were 10.0 (IQR 6.0-23.25) and 2.31 (0.88-5.44), respectively. Most (n = 65; 79.3%) performed clinical research for their research thesis projects. Half (n = 41; 50%) performed studies within an area related to their matched fellowship(s). Sixty-eight (n = 68; 82.9%) projects resulted in a publication and it took an average of 2,582.8 days, or roughly 7 years, to have their project published in print since the start of their residency training. These were published most frequently in the Journal of Bone and Joint Surgery (Am) (n = 13; 15.9%), Journal of Hand Surgery (n = 11; 13.2%), Clinical Orthopaedics and Related Research (n = 7; 8.5%), Journal of Orthopaedic Trauma (n = 7; 8.5%), and American Journal of Sports Medicine (n = 5; 6.1%). A full list of journals can be found in the Appendix (Table A1, https://links.lww.com/JBJSOA/A458). The mean year adjusted impact factor for journals published in was 2.97 (±1.44).

Predictors of Publication Success and Latency

Given the very strong correlative nature between adjusted H-index and adjusted number of publications (r = 0.827, p < 0.001, variance inflation factor 3.17), it was decided to proceed only with adjusted H-index in further regression modeling to avoid potential issues with multicollinearity. Although not a perfect metric, the H-index, in theory, takes into account both scientific productivity and impact. In univariate Cox regression analyses, a higher adjusted H-index was associated with increased success of and decreased time to publication (hazard ratio [HR] 1.183 [95% CI: 1.059-1.322], p = 0.003). For projects that made it to publication, a lower journal impact factor was associated with taking significantly shorter time to reach publication (F(1,66) = 7.839, p = 0.007; Β₁ = 146.45, p = 0.007). Current practice setting (HR 0.747 [95% 0.463-1.203], p = 0.230), having the project related to matched fellowship(s) (HR 0.775 [95% CI 0.481-1.249], p = 0.295), and study type (HR 1.188 [95% CI 0.669-2.112], p = 0.556) were not significantly associated with success or timing to publication.

Discussion

Scholarship and research are important aspects of orthopaedic surgical training. Research is not only required by the ACGME¹ but also highly valued by fellowship directors¹⁴. Many orthopaedic surgery residency programs have developed dedicated research curricula where residents undertake a thesis project, but not all projects make it to publication. Over the past 15 years, more than 80% of our resident thesis projects were published. Data regarding success rates and timing to publication for resident research thesis projects are scarce not only in the orthopaedic surgery literature but also across other specialties. Pilche et al.⁸ found that 63% of orthopaedic surgery resident research thesis projects were published at their institution. They did not analyze factors associated with successful publication. In the dermatology literature, 1 residency program instituted a research program that involved assigned mentorship, financial support, statistical consultation, and a 2-week elective of protected time, and found that since the inception of the program, 10 of 16 (62.5%) residents had their research projects published in peer-reviewed journals¹⁵. Piche et al.⁸ found that their thesis projects were published in 4.04 years (range 2-7 years) from when their projects were started sometime during their second year of residency. Our time interval of 7 years to publication was calculated from the start of residency.

At our own institution, a dedicated research curriculum provides protected research time, longitudinal direction, training, mentorship, and numerous other forms of resources and support including biostatistician consulting which likely contributes to our high success rate of publication. Currently, during the postgraduate year (PGY)-1 year, interns identify a research mentor and work to come up with a potential project. Mentors are typically chosen based on previous resident research project involvement, success, and publication track record. Mentors' involvement is on a volunteer basis. During the PGY-2 year, residents have 6 weeks of dedicated time to design their research project and create a grant proposal to fund their project. It is now a requirement that all residents must apply for a research grant. Research mentors may also use some of their own discretionary funds to help in some capacities such as fronting publication fees. In the PGY-3 year, there is no dedicated research time, but most of the data collection is expected to be completed during this year. During the PGY-4 year, residents are given a second 6-week research block to finalize their research project and present their abstracts to the resident research committee. In the final PGY-5 year, manuscripts are written and submitted and a presentation is expected at graduation. Of note, residents remain in the junior and senior call pools during the research rotations to minimize impact on patient care. Moreover, the dedicated research blocks are placed into residents' schedules as a formal rotation so they do not affect the composition of residents available for clinical care.

A plethora of data support the added benefit of protected or dedicated research time for residents. For instance, 1 institution found that after implementing a dedicated research program, orthopaedic surgery residents who trained after implementation published more papers in higher impact factor journals during their residency than those who trained before the implementation³. Another study in 2017 examined 125 ACGME-accredited US orthopaedic residency programs and found that overall, the mean number of publications for residents in programs with protected time was almost double that of those in programs without protected time⁶.

To the best of our knowledge, this is the first study to assess factors associated with the completion and timing to publication of resident research thesis projects. We found that in our cohort, a higher graduate adjusted H-index was associated with greater completion rate and shorter time to publication. Individuals who are more academically inclined may be more likely to publish their projects and at an earlier time point. Interestingly, those currently in an academic practice did not demonstrate increased success when compared with their private practice counterparts. Half (50%) of our recent graduates currently practice in an academic setting, which is higher than reported AAOS general census data (16% academic, privademic [private practice while holding an academic appointment], or military)¹⁶. Although total number of publications during residency has been found to be greater for those who enter an academic career¹⁷, 1 investigation found that those who completed a research year had more publications but were not more likely to enter an academic practice¹⁸. The concordance between projects' subject area with matched fellowship(s) (50%) was similar to a previously reported rate (46.7%)⁸, but did not correlate with publication success or timing in our analysis. Year adjusted journal impact factor was correlated with timing to publication, such that projects in lower impact journals were published in a shorter amount of time.

Limitations

There are numerous limitations to our investigation. First, this is a single-institution effort, and our data may not be generalizable to other programs or residents. We did not delineate between resident publication metrics before and after residency, which likely introduces selection bias of the individuals who rank and match at programs with a stronger emphasis on research. In addition, programs may not have the same level of research infrastructure and resources to support a resident research program. Examining how scholarship varies by program and/or available resources would be an important avenue for future study. Second, online public sources were used to acquire data, and these data sources may not be completely accurate. For instance, although Scopus automatically compiles data on authors including publications, citation counts, and metrics such as the H-index, unless an author profile is “claimed” and continually updated, the numbers tallied are likely underestimations of total scientific output. The H-index itself has many advantages including its easy computability, lack of required data processing, and accessibility from most indexing databases, but it is certainly not a perfect measure of scientific productivity¹⁹. We did not collect data on project mentors which may also significantly affect the outcomes assessed in this study and would be interesting to investigate further⁴. Finally, “success” of our research program was operationally defined as resident theses resulting in peer-reviewed publication, but without any comparative analyses between programs or within our program itself before and after implementation of the research program, we cannot truly establish the effectiveness of our program. It could be that our publication rate is higher compared with other published rates due to other confounding factors.

Conclusions

In conclusion, over 80% of orthopaedic resident research thesis projects at our institution were published over the past 15 years. Success and timing to publication were correlated with graduates' current adjusted H-index such that a higher adjusted H-index was associated with a greater likelihood of completion and a faster timing to publication. A lower journal impact factor was also associated with quicker publication. These data highlight the publication metrics of a formalized resident research program and identify factors associated with successful publication and timing of research thesis projects.

Sources of Funding

No sources of funding were granted for the current study.

Appendix

Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (https://links.lww.com/JBJSOA/A458). This content was not copyedited or verified by JBJS.