Orthopaedic Resident Remediation

Frequency, Interventions, and Outcomes

Melton, William, MD1; Jackson, J. Benjamin III, MD1; Koon, David, MD1; Grabowski, Gregory, MD1

doi: 10.2106/JBJS.OA.18.00011
Scientific Articles
Open
SDC
Disclosures

Background: Despite multiple studies in other areas of surgical resident education, there is no current literature regarding orthopaedic resident remediation. The goal of the present study was to use a survey format to determine the frequency of remediation, the underlying etiology, the intervention strategies utilized, and the outcomes.

Methods: The program directors of all current 159 Accreditation Council for Graduate Medical Education-recognized orthopaedic residency programs in the United States were e-mailed a non-identifying digital survey. Follow-up e-mails were sent monthly for a 3-month period. The data remained anonymous, with no identifying information. The data are reported with use of descriptive statistics.

Results: Seventy (44%) of 159 program directors responded; most were from academic institutions with 4 to 5 residents per class. One hundred and fifty-eight residents were remediated, with the greatest number being remediated during the third postgraduate year (PGY-3). Professionalism, patient care, and communication were the most common deficiencies requiring remediation. Mentorship, feedback, and probation were the most common interventions. Of the 117 residents for which the outcome was reported, 58 graduated on time, 14 graduated from another program, 25 graduated from another specialty, 14 were terminated, 3 pursued litigation, and 3 chose a non-medical career. Rehabilitation, didactics, feedback, and mentorship were associated with the highest rates of on-time graduation.

Conclusions: Most remediated residents were PGY-3, suggesting increased scrutiny as residents moved from junior to senior responsibilities. The greatest number of deficiencies requiring remediation pertained to the affective domain, which highlights the importance of screening measures such as away rotations and interviews. The relationships formed during increased feedback sessions and mentorships can help problem residents to graduate on time.

1Palmetto Health USC Orthopaedic Center, Columbia, South Carolina

E-mail address for W. Melton: wmeltonn@gmail.com

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

The American Board of Internal Medicine defines a problem resident as “a learner who demonstrates problem behaviors significant enough to require intervention by program leadership….”1 The problem could present in various aspects of performance, including emotion management, knowledge base, social interaction, and surgical skills. This intervention could be in the form of remediation, defined as “intending to correct or improve one’s skill in a specific field.”2 In an effort to prevent the need for future remediation, researchers have attempted to identify selection criteria to predict a successful orthopaedic resident. Although no consensus exists, United States Medical Licensing Examination Step 1 score, honor clerkship grades, motor ability, and affective domain characteristics have been correlated with resident success3,4.

The Accreditation Council for Graduate Medical Education (ACGME) adopted 6 core competencies in 1999 to standardize the evaluations and expected performance of residents and faculty5. This framework for objective evaluation has led to an increase in publications regarding potential strategies to identify and help problem residents. In the Clinical Competency Committees: A Guidebook for Programs, the ACGME provides options for remediation such as intensive mentoring, additional readings, skill laboratories, added or repeated rotations, extended education, or counseling to consider another specialty or profession6.

Surveys sent to program directors have been utilized to gauge trends and strategies currently being implemented in other areas of medicine, including internal medicine, otolaryngology, and general surgery7-9. However, we are not aware of any current literature from organized committees or national surveys regarding orthopaedic surgery resident remediation. The goal of the present study was to utilize a national survey to determine the frequency of remediation, the underlying etiology for remediation, the intervention strategies utilized, and the outcome for orthopaedic residents.

Back to Top | Article Outline

Materials and Methods

The program directors of all current 159 ACGME-recognized orthopaedic residency programs in the United States were sent a non-identifying digital survey via e-mail. No programs were excluded. After the initial e-mail, follow-up e-mails were sent every month for a 3-month period to the program directors who had not responded. The survey had a check-box format that included the option to select all answers that applied or to include free text when indicated (see Appendix). The first question pertained to the total number of residents remediated at said program during the last 10 years. The remaining questions (questions 2 through 8) were asked for each resident reported in the first question. For example, if 3 residents had been remediated during the last 10 years, then the answer to Question 1 would be 3. Questions 2 to 8 would then be asked sequentially in 3 different sets to account for each of the 3 residents. No identifying data on individual residents were collected. The data were analyzed with use of descriptive statistics with the assistance of a trained biomedical statistician.

Back to Top | Article Outline

Results

Seventy of 159 program directors responded, for a response rate of 44%. The mean program size was 4.5 residents per year (range, 1 to 9 residents per year). Fifty-eight program directors reported that they had implemented remediation interventions, and 12 reported that no residents had been remediated. Sixty-three respondents were from academic institutions and 7 were community hospitals. Of the 58 directors who reported remediation, 53 were from academic institutions and 5 were from community hospitals.

The results of the survey indicated that, during the last 10 years, 158 residents had required remediation. The highest number of residents remediated from 1 program was 8, and the mean was 3. Of the 158 residents who had been remediated during the last 10 years, 91 had been accepted into the program via a traditional route and 13 had been accepted via a non-traditional route; the route was not specified for the remaining 54 residents. A traditional route is defined as an applicant matching into a first postgraduate year (PGY-1) position of an ACGME-accredited orthopaedic program. Most residents requiring remediation were identified during PGY-3 (Fig. 1).

Fig. 1

Fig. 1

Communication prompted the most instances of remediation followed by communication and professionalism (Fig. 2). Mentorship, increased feedback, and probation were the most commonly used interventions (Fig. 3). The outcome was reported for 117 residents, with 72 graduating from an orthopaedic program (Fig. 4). Most (58) graduated on time. Of the remaining residents, 25 graduated in another medical field, 14 graduated from another orthopaedic program, 14 were terminated, 3 pursued litigation, and 3 chose a non-medical career. The “select all that apply” aspect of the survey allowed for differences in the number of reported remediated residents, deficiencies, interventions, and outcomes.

Fig. 2

Fig. 2

Fig. 3

Fig. 3

Fig. 4

Fig. 4

The outcomes associated with each remediation strategy are shown in Table I. The rate of on-time graduation was 66.7% for didactic intervention, 59.5% for feedback, and 58.5% for mentorship. Rehabilitation was associated with the highest rate of on-time graduation (71.1%) but also was associated with the highest rate of termination (28.6%). Leave of absence predictably was associated with the lowest rate of on-time graduation (40.0%), followed by counseling (41.2%). Leave of absence also was associated with the second-highest rate of termination (20%).

TABLE I - Outcome Associated with Remediation Intervention
Graduated on Time Graduated from Another Orthopaedic Program Other Medical Field Termination Litigation Non-Medical Career
Increased feedback 59.5% 2.6% 20.3% 13.5% 2.7% 1.4%
Mentorship 58.5% 5.3% 20.2% 10.6% 3.2% 2.1%
Probation 47.7% 3.1% 27.7% 16.9% 3.1% 1.5%
Psychological counseling 41.2% 7.8% 25.5% 17.6% 3.9% 3.9%
Strict guidelines 47.2% 8.3% 19.4% 13.9% 5.6% 5.6%
Didactics 66.7% 3.7% 13.0% 9.3% 3.7% 3.7%
Leave of absence 40.0% 6.7% 13.3% 20.0% 6.7% 13.3%
Learning assessment 55.6% 0 44.4% 0 0 0
Substance abuse rehabilitation 71.4% 0 0 28.60% 0 0

Back to Top | Article Outline

Discussion

The response rate of 44% was higher than anticipated but lower than desired. The programs that responded were mostly at academic institutions and included 4 to 5 residents, but a large range of diverse programs was included. These programs averaged 3 residents remediated in the past 10 years. The majority of residents required remediation at PGY-3, indicating concern about graduating residents from junior to senior-level responsibilities. More PGY-5 residents than interns required remediation, suggesting that orthopaedic evaluation is difficult until responsibilities increase. The large jump in remediation from PGY-1 to PGY-2 supports this theory.

As responsibilities increase, the resident’s motor skill, affective domain, and cognitive ability are increasingly scrutinized. However, these characteristics can also be evaluated in medical students. In 2 separate studies regarding resident orthopaedic success, clinical clerkship grades were associated with better performance and better interpersonal skills3,10. These findings support the importance of clinical clerkships, away rotations and, to a lesser degree, interviews. Spending a month with an applicant allows evaluators to observe how the applicant handles stress, interacts with staff and peers, and presents himself or herself to others. Unfortunately, there is no panacea, but longer evaluations in varied environments offer the best chance for a program to evaluate the affective domain in the current system. Once an applicant has been accepted into a program, it becomes increasingly difficult to change his or her personality, values, and core beliefs.

However, improving medical knowledge or operative skill may be more feasible. The Orthopaedic In-Training Examination (OITE) provides a standardized, objective way to identify residents who may be candidates for remediation. In the present study, medical knowledge was cited as the reason for remediation in 60 cases, and more residents were remediated for this reason than were remediated for both operating room performance and clinical patient care. This difference may be due to the objective nature of identifying knowledge deficiencies and is consistent with the general surgery literature11. Although motor skills are often a focus of aspiring surgeons, the importance of a comprehensive understanding of pathophysiology and anatomy cannot be overlooked.

In their survey-based study of general surgery residency remediation, Torbeck and Canal reported that only 52% to 75% of programs had a specific policy for remediation8. Hauer et al.11 and Smith et al.12 proposed formal guidelines for remediation with 4 and 3 steps, respectively, applicable to residents in all specialties. Standardization of the process could make these strategies readily available and could decrease the national variation in current practices11-13.

Although there is a need for formal remediation strategies, preventing and identifying the deficiencies should not be overlooked. In an effort to prevent the need for remediation, the fourth year of medical school could be restructured with a more rigorous curriculum, which could include basic surgical skill laboratories, increased opportunity for patient care, and continued growth with a focus on the affective domain. Several strategies also could be implemented to improve the interview and away rotation processes. Ethical situation rooms could present an opportunity to further evaluate the affective domain. Increasing the number of away rotation positions could allow for better understanding of the quality of the applicant and could lead to fewer applicants on formal interview days as these rotations are essentially month-long interviews. These strategies have not been validated and may not be feasible. However, they could provide a change to the current system and could be investigated further in the future.

A large number of residents who required remediation graduated from an orthopaedic program, with most of them graduating on time. Others were able to transition to another orthopaedic program or another medical field and graduate as well. The low number of terminations and even lower number of reported litigation cases suggest that residents may not have left the field of orthopaedics or medicine on poor terms. Furthermore, increased use of remediation strategies in general surgery residencies has been associated with decreased attrition14. These findings may be due to communication and relationships established during the remediation process, specifically mentorship and feedback sessions.

Although the response rate in the present study was only 44%, we believe that this finding was not a limitation, for 2 reasons: (1) a large number of diverse programs were included in the present study and (2) this rate is higher than those in similar published studies in the general surgery and orthopaedic literature8,10. However, the present study had many other limitations. The self-reporting nature of surveys and the personal nature of these conflicts could lead to recall bias as well as selection bias. Recall bias may be stronger in cases of affective domain issues due to the personal and potentially egregious nature of these issues; learners who simply needed an additional reading plan may be less memorable to a program director 10 years later.

Another major limitation is the user error of the survey. For example, 58 program directors reported remediated residents, but only 52 program directors answered the follow-up questions specific to their remediated residents. Furthermore, some of the question sets were incomplete for remediated residents. For example, 7 of 10 questions may have been answered for 1 resident while 6 of 10 were answered for another. This limitation could have been mitigated if the survey had been piloted to program directors. Also, a survey format with a “select all that apply” option has inherent limitations such as the potential for multiple, overlapping reported outcomes. These limitations led to the discrepancy in the reported data as seen in the figures.

The survey described in the present report identified the number of residents who had been remediated, the reason for remediation, subsequent interventions, and outcomes. This information may allow program directors to understand the most commonly utilized and effective strategies for resident remediation to better serve residents in need.

Back to Top | Article Outline

Appendix

The full survey sent to program directors is available with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A80).

Investigation performed at the Palmetto Health USC Orthopaedic Center, Columbia, South Carolina

Disclosure: No external source of funding was received for this project. On the Disclosure of Potential Conflicts of Interest forms, which are provided with the online version of the article, one or more of the authors checked “yes” to indicate that the author had a relevant financial relationship in the biomedical arena outside the submitted work. The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJSOA/A79).

Back to Top | Article Outline

References

1. Yao DC, Wright SM. National Survey of Internal Medicine Residency Program Directors Regarding Problems Residents. JAMA. 2000;284(9):1099-1104. Apr 19; New Orleans, LA.
2. Dictionary.com. Remedial. http://www.dictionary.com/browse/remedial. Accessed 2018 Mar 23.
3. Spitzer AB, Gage MJ, Looze CA, Walsh M, Zuckerman JD, Egol KA. Factors associated with successful performance in an orthopaedic surgery residency. J Bone Joint Surg Am. 2009 Nov;91(11):2750-5.
4. Egol KA, Collins J, Zuckerman JD. Success in orthopaedic training: resident selection and predictors of quality performance. J Am Acad Orthop Surg. 2011 Feb;19(2):72-80.
5. American Board of Medical Specialties. Based on core competencies. http://www.abms.org/board-certification/a-trusted-credential/based-on-core-competencies/. Accessed 2016 Jun 15.
6. Andolsek K, Padmore J, Hauer KE, Edgar L, Holmboe E. Clinical competency committees: a guidebook for programs (2nd ed). Accreditation Council for Graduate Medical Education. 2017 Sep 23. https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf. Accessed 2018 May 4.
7. Bhatti NI, Ahmed A, Stewart MG, Miller RH, Choi SS. Remediation of problematic residents—a national survey. Laryngoscope. 2016 Apr;126(4):834-8. Epub 2015 Sep 22.
8. Torbeck L, Canal DF. Remediation practices for surgery residents. Am J Surg. 2009 Mar;197(3):397-402.
9. Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. JAMA. 2000 Sep 6;284(9):1099-104.
10. Dirschl DR, Dahners LE, Adams GL, Crouch JH, Wilson FC. Correlating selection criteria with subsequent performance as residents. Clin Orthop Relat Res. 2002 Jun;399:265-71.
11. Hauer KE, Ciccone A, Henzel TR, Katsufrakis P, Miller SH, Norcross WA, Papadakis MA, Irby DM. Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature. Acad Med. 2009 Dec;84(12):1822-32.
12. Smith JL, Lypson M, Silverberg M, Weizberg M, Murano T, Lukela M, Santen SA. Defining uniform processes for remediation, probation and termination in residency training. West J Emerg Med. 2017 Jan;18(1):110-3. Epub 2016 Nov 21.
13. Yaghoubian A, Galante J, Kaji A, Reeves M, Melcher M, Salim A, Dolich M, de Virgilio C. General surgery resident remediation and attrition: a multi-institutional study. Arch Surg. 2012 Sep;147(9):829-33.
14. Schwed AC, Lee SL, Salcedo ES, Reeves ME, Inaba K, Sidwell RA, Amersi F, Are C, Arnell TD, Damewood RB, Dent DL, Donahue T, Gauvin J, Hartranft T, Jacobsen GR, Jarman BT, Melcher ML, Mellinger JD, Morris JB, Nehler M, Smith BR, Wolfe M, Kaji AH, de Virgilio C. Association of general surgery resident remediation and program director attitudes with resident attrition. JAMA Surg. 2017 Dec 1;152(12):1134-40.

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2018 The Authors. Published by The Journal of Bone and Joint Surgery, Incorporated. All rights reserved.