Based on US Census data as of 2016, racial and ethnic minorities accounted for 38.7% of the US population, including 17.8% Hispanic or Latino, 13.3% Black or African American, and 5.7% Asian.1 According to the Council on Graduate Medical Education, the racial/ethnic composition of the US physician workforce does not reflect the general population of the United States. Importantly, it is thought that the racial and ethnic composition of the physician workforce improves healthcare access for underrepresented minorities (URMs).2 Increasing the diversity in the physician workforce is an important step toward improving patient communication and addressing healthcare disparities.3 Several medical specialties, such as internal medicine, family medicine, and pediatrics, are moving toward this goal with URMs making up an increasing percentage of residency programs (64.9%, 49.7%, and 44.3%, respectively).4 The 2008 Orthopaedic Practice in the US survey showed that minorities accounted for only 10.7% of the practicing orthopaedic surgeons (5.2% Asians; 1.9% Hispanics; 1.6% African Americans),5 lagging behind all other surgical subspecialties in the representation of Hispanics and African Americans.4 However, Asians, appear to be represented in the field of orthopaedic surgery (5.2%) at similar levels to their representation in the US population (5.7%). In addition, the minority composition of orthopaedic surgery residents (20.2% in total, made up of 11.7% Asian, 4.0% African-American, 3.8% Hispanic, 0.4% American Indians/Alaskan Natives, and 0.3% Native Hawaiians/Pacific Islanders) was lower than the minority representation in any of the other 16 training fields of medicine in 2008.5
The factors leading to lower representation of minorities in orthopaedic surgery training programs are not well understood. It is possible that such differences stem from lower admission rates of URM candidate applicants to orthopaedic residency programs. For example, representation of Black/African Americans and Hispanic/Latino in orthopaedic surgery residency programs remains lower than in US colleges or medical schools and other surgical subspecialties.5 Differences in the academic metrics of minority applicants may contribute to reduced admission rates relative to Caucasian applicants. To better investigate these factors, this study examined residency enrollment rates of first-time applicants to orthopaedic surgery residency by race/ethnicity over a 10-year period (2005 to 2014). The quantitative academic characteristics of first-time orthopaedic surgery residency applicants were also compared based on race. Applicants who successfully matriculated into orthopaedic surgery residency were also analyzed to further evaluate the possibility that differences in matriculation rates by race may be related to applicant pool academic metrics.
Orthopaedic surgery residency application data through the Electronic Residency Application System was obtained from the Association of American Medical Colleges (AAMC), and the National Board of Medical Examiners (NBME) granted access to the United States Medical Licensing Exam (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) scores for these applicants. Information on whether an applicant enrolled into an orthopaedic residency program was also provided by the AAMC through GME Track Resident Survey (part of the National GME Census) completed annually by residency program directors. The Electronic Residency Application System, NBME, and GME Track data were linked at the individual applicant level by the AAMC and were provided to the researchers in a de-identified manner.
Data on first-time applicants (n = 12,093) for orthopaedic surgery residency positions in the United States were reviewed from 2005 to 2014. The enrolled orthopaedic resident data are from GME Track, which includes all residents who entered into orthopaedic residency through the Match, Post Match Supplemental Offer and Acceptance Program, or some other agreement before, or after, the Match. Because of lower overall enrollment rates into orthopaedic residencies, applicants with medical school type of Foreign Medical Graduate (n = 1,008), International Medical Graduate (n = 1,341), Doctors of Osteopathic Medicine (n = 776), and those whose medical school type was not indicated (n = 2) were excluded from this analysis. The remaining data set consisted of 8,966 first-time applicants from US medical schools (MD), of which n = 6,218 had matriculated into orthopaedic surgery residency during the 10-year study period. Typically, quantitative factors used to assess applicant's academic indicators (USMLE Step 1 and Step 2 CK scores, number of publications, Alpha Omega Alpha [AOA] Honor Medical Society status, and volunteer experiences) were then broken down based on race/ethnic group.
Application metrics were compared for applicants that identified their race/ethnicity as White (group 1), Asian (group 2), Black or African American (group 3), Hispanic or Latino (group 4), or Other (group 5), which consisted of applicants who self-identified as American Indian or Alaskan Native, Hawaiian or other Pacific Islander, Other or Unknown. These factors were also examined for students who enrolled/matriculated into an orthopaedic surgery residency.
Regression analyses were performed, as described below, to evaluate the effects of the variables “year,” “race,” and their interactions. All analyses were performed separately on first-time applicants and enrolled candidates. For multiple group comparisons, an analysis of variance was performed, followed by Tukey Honest Significant Difference post hoc test. Findings of P < 0.05 were considered statistically significant. Statistical analyses were performed using IBM SPSS Statistics (Version 21). Data are reported as mean with standard error and range (minimum to maximum), unless otherwise indicated. In addition, the 95% confidence interval is provided in Tables 1–3.
Enrollment rate was computed for each ethnic/racial group and year as the ratio between the number of matriculated residents and the number of applicants (%). Effect of race/ethnicity and year were analyzed using multiple linear regression analysis. The interaction between year and race was not included.
USMLE Step 1 and Step 2 Clinical Knowledge Scores
Step 1 and Step 2 CK scores were compared using multiple linear regression. The main effects included in the model were “year” (ie, 2005 through 2014) and “race” for all applicants and enrolled candidates. The interaction between year and race was not significant, and so the main effects of “year” and “race” were examined alone in each group.
Alpha Omega Alpha Status
A multiple logistic regression model was used to determine if the proportion of the five race/ethnicity groups achieving AOA status was significantly different between groups and across time. The two main effects included in the model were “year” (ie, 2005 through 2014) and “race” in all applicants and enrolled applicants. The interaction between the two main effects was not significant, and the variables were examined alone.
Publication counts, as reported in the AAMC application, were compared using multiple linear regression. The effects included in the model were “year” (ie, 2005 through 2014) and “race,” as well as the interaction between time and race, because the interactive term was found to be a significant contributor.
Self-reported volunteer experiences, as reported in the AAMC application, were compared using multiple linear regression, with “year” (ie, 2005 through 2014) and “race.” The interaction between the two main effects was not significant, and the variables were examined alone.
Applicants and Enrolled Candidates
White applicants made up the most first-time applicants (71.0%), followed by applicants from Asian (11.8%), Other (8.9%), Black (5.7%), and Hispanic (2.5%) groups. The composition of enrolled residents followed a similar pattern, with White residents being the most represented group (74.7%), followed by Asian (11.6%), Other (8.0%), Black (3.8%), and Hispanic (1.8%) groups (Table 1).
Averaged for more than 10 years (2005 to 2014), applicants from minority groups (Asians, Blacks, Hispanics, and Other) comprised 28.9% (n = 2,596/8,966) of all applicants to orthopaedic surgery programs. Ultimately, the minority groups accounted for 25.3% (n = 1,571/6,218) of the total number of matriculated orthopaedic surgery residents (Table 1). Applicants from Asian, Black, Hispanic, and Other races/ethnicities matched at an average rate of 60.5% (n = 1,571/2,596), whereas White applicants had an average enrollment rate of 73% (n = 4,647/6,370; P < 0.0001). When comparing enrollment rates between the groups, differences in enrollment rates were observed over the 10-year study period (P < 0.0001; Figure 1). Enrollment rate of White and Asian applicants were similar (P = 0.453). Enrollment rate of Blacks and Hispanic applicants were significantly lower than White, Asian, and Other applicants (P < 0.0001); enrolment rate of Other applicants was lower than White applicants (P < 0.001). Enrollment rate of Black applicants was not significantly different from that of Hispanic applicants (P = 0.99) (Figure 1, B). The temporal dynamics of enrollment rates also varied based on the race/ethnic groups. Enrollment rate of White applicants was relatively steady (mean, 73.0%; SD, 2.3%; range, 69.6% to 76.8%) (Figure 1, A). Enrollment rate of applicants from oOther or Asian groups varied moderately (Other, 62.3%; SD, 4.4%; range, 55.9% to 70.0%; Asian, 69.1%; SD, 6.3%; range, 54.5 to 75.3%). Enrollment rates of Black and Hispanic applicants varied to the greatest level across the 10-year study period (Black, 47.0%; SD, 9.7%; range, 30.5% to 60.0%; Hispanic, 47.8%; SD, 8.5%; range, 33.3% to 60.0%).
USMLE Step 1 and Step 2 Clinical Knowledge Scores
There was an increase in Step 1 scores from 2005 to 2014 (P < 0.0001; Figure 2, A and B). Descriptive statistics of Step 1 scores by race/ethnic groups are reported in Table 2. In comparing race/ethnic groups, Step 1 scores from White (mean, 234.3; range, 182 to 280) and Asian (mean, 233.5; range, 184 to 274) applicants were not significantly different (P = 0.85; Figure 2, C). Similar Step 1 scores were also observed between White (mean, 238.2; range, 182 to 280) and Asian (mean, 237.8; range, 184 to 274) matriculated candidates (P = 0.96) (Figure 2, D). Step 1 scores by applicants from Black (mean, 218.1; range, 170 to 265) and Hispanic (mean, 223.0; range, 181 to 266) groups were significantly lower than those from White, Asian, and Other (mean, 231.4; range, 176 to 274) applicants (P < 0.001) (Figure 2, C). For matriculated candidates, Step 1 scores among White, Asian, and Other residents were similar (P > 0.21). Similar Step 1 scores were observed for Black and Hispanic matriculated candidates (P = 0.09; Figure 2D).
There was an increase in Step 2 CK scores from 2005 to 2014 (P < 0.0001; Figure 3, A and B). Descriptive statistics of Step 2 CK scores by race/ethnic groups are reported in Table 2. Step 2 CK scores from White applicants (mean, 237.4; range, 179 to 288) were significantly greater than those from all other race/ethnic groups (P < 0.0001). Step 2 CK scores from Asian (mean, 232.2; range, 182 to 282) and Other (mean, 233.7; range, 153 to 279) applicants were not significantly different from one another (P = 0.63; Figure 3, C). Higher Step 2 CK scores were observed in Asian applicants compared with Hispanic (mean, 226.1; range, 155 to 274) or Black (mean, 219.7; range, 176 to 268) applicants (P < 0.001). Hispanic applicants had higher Step 2 CK scores compared with Black applicants (P < 0.0001). In matriculated candidates, Step 2 CK score was significantly higher in Whites (mean, 241.0; range, 181 to 288) compared with all other groups (P < 0.004). No significant differences in Step 2 CK scores were observed between Asian (mean, 236.4; range, 182 to 282), Hispanic (mean, 234.4; range, 182 to 274), and Other (mean, 238.4; range, 186 to 279) matriculated candidates (P > 0.21) (Figure 3, D), although scores of Black (mean, 228.9; range, 184 to 268) candidates were lower than those of Asian or Hispanic candidates (P < 0.024) (Figure 3, D).
Alpha Omega Alpha Status
The likelihood of an applicant achieving AOA membership was not dependent on application year, for both applicants and matriculated candidates (P = 0.84; Figure 4). However, 22.9% (n = 2,056) of all applicants were AOA members over the 10-year study period (White: 18.8%, n = 1,682; Asian: 1.8%, n = 162; Black: 0.4%, n = 32; Hispanic: 0.3%, n = 29; Other: 1.7%, n = 151). White applicants had higher odds of having AOA membership compared with all other groups (P < 0.0001) (Figure 4, A). No significant differences were found between applicants from Blacks and Hispanics (P = 0.31). In addition, the odds of Asian applicants were similar to those of Other applicants (P = 0.33). Also, 29.6% (n = 1,840) of matriculated candidates were AOA members (White: 24.4%, n = 1,515; Asian: 2.4%, n = 147; Black: 0.4%, n = 26; Hispanic: 0.4%, n = 23; Other: 2.1%, n = 129). The same trends were observed when comparing matriculated candidates by race/ethnicity groups (Figure 4, B).
Number of Publications
The number of publications for both applicants and matriculated candidates increased significantly between 2005 and 2014 (P < 0.0001) (Figure 5, A and B). Descriptive statistics of publications over the 10-year time frame are presented in Table 3. Because of the statistical interaction between application year and race, we have focused statistical group comparisons on the most recent year in the data set (2014). Publication counts in the 2014 applicant pool were not statistically different when compared across multiple race/ethnicity groups (P > 0.06) (Figure 5, C). In 2014, publication counts by Hispanic matriculated candidates were significantly greater than those by all other race/ethnic groups (P < 0.04) (Figure 5, D). No significant differences were observed in the publication count between White, Asian, Black, and Other matriculated candidates (Figure 5, D).
An increase in the number of volunteer experiences was reported by all applicants and matriculated candidates (P < 0.0001) (Figure 6, C and D). Descriptive statistics of volunteer experience over the 10-year time frame are presented in Table 3. No significant difference was observed in volunteer experiences between White, Asian, Hispanic, and Other applicants (P < 0.77) (Figure 6, C). Black applicants had a significantly greater number of volunteer experiences compared with Asian applicants (P < 0.04; Figure 6, C). Black matriculated candidates reported greater number of volunteer experiences compared with all other race/ethnic groups, although these differences were not statistically significant (P > 0.17) (Figure 6, D). Volunteer experiences were comparable across all groups of matriculated candidates (Figure 6, D).
This study is the first to compare academic metrics of US medical student applicants to orthopaedic surgery residency and matriculated orthopaedic surgery residents by race/ethnicity. Academic metrics of the applicant pool and matriculated candidates have significantly improved over the 10-year study period for all racial/ethnic groups. A steady increase is observed in the USMLE test scores, publications, and volunteer experiences between 2005 and 2014, irrespective of race. Minorities made up 29% of applicants, and 25% of enrolled candidates, which is similar to the minority representation in the US population (38.7%). However, results point to a greater representation of Asians in orthopaedic residency (11.6%) compared with the US population (5.7%), whereas Hispanics and Blacks remain relatively underrepresented in orthopaedic residencies (1.8% and 3.8%) relative to the population (17.8% and 13.3%, respectively). Despite improving academic indicators by both White and minority groups, significant differences remain in the academic measures within the orthopaedic surgery applicant pool based on race/ethnicity.
White and Asian applicants and matriculated candidates had higher Step 1 scores compared with their counterparts. White applicants and matriculated candidates had higher Step 2 CK scores and were more likely to have AOA status than minorities. However, Hispanic enrolled candidates had a greater number of publications. Black applicants had more volunteer experiences than any other group.
From 2005 to 2014, minority applicants enrolled into orthopaedic residencies at a considerably lower rate than Whites. When rates are analyzed for individual race/ethnic groups, the differences are even more dramatic, with Black and Hispanic enrollment rates being significantly lower than those of Whites and Asians. These results suggest that standardized test scores and AOA status, two metrics in which White applicants outperformed URM, may play a greater role in evaluating applicants for orthopaedic residency programs than publications count or volunteer experiences, two other metrics in which URM generally outperformed Whites. Other selection factors, such as conscious or unconscious bias, may also contribute to the lower match rate of URM; however, such factors were beyond the scope of this study.
The finding that minority students scored lower on USMLE examinations is not limited to orthopaedic surgery applicants. Andriole and Jeffe6 found that more minority students (3.2%) failed Step 1 compared with White students (1.8%). Of the students who nevertheless proceeded to take Step 2 CK, more minority students failed than White students (73.9% versus 26.0%, respectively). Lerner and Nagai7 reported that the University of Maryland School of Medicine admitted Black applicants with significantly lower MCAT scores compared with other racial groups and observed differences in Step 1 and Step 2 scores in Black students compared with White students.
Potential reasons behind why minority students score lower on standardized tests compared with White students are multifactorial. It is unlikely that the tests are biased because test questions undergo bias and sensitivity review and examinees receive the same instructions, amount of testing time, and types of computer equipment.8 There are socioeconomic factors including educational environment and regional differences that can contribute to gaps in test scores among racial groups. In one study, Black and Latino medical school applicants were more likely than White applicants to have been raised in families with lower household incomes.8 Giordano et al9 found that medical students who had financial need did not perform as well on Step 1 as those who did not have financial need, despite having access to the same preparation tools and curriculum, including free tutoring. In addition, minority and White students differ with respect to the level of parents’ education. Black and Latino medical school applicants were less likely to have parents who received college degrees.8 Each medical student applying to residency comes from a unique environment with different financial, family, and educational backgrounds. This makes it difficult to attribute specific socioeconomic factors to differences seen in standardized test scores.
Although many factors are considered in evaluating candidates for orthopaedic surgery residency, USMLE Step 1 and Step 2 CK scores are often used as a screening tool to help determine which applicants will be invited to interview. In 2014, the National Resident Matching Program Director Survey found that 94% of the 1,793 residency program directors representing all medical specialties regarded Step 1 scores as one of the most important factors for evaluating an applicant, with a rating of 4.1 out of 5.9 USMLE Step 1 and Step 2 examinations are often used as a standardized method to evaluate applicants from different medical schools, which use different grading systems. Step 1 examination focuses on assessing a student's understanding and ability to apply preclinical concepts to the practice of medicine. Step 2 CK examination is intended to evaluate a student’s ability to apply medical knowledge and skills to coalesce into patient care under supervision, with an emphasis on health promotion and disease prevention. However, USMLE was not necessarily designed to be predictive of success in residency. No standardized test can measure qualities of professionalism or surgical aptitude that are inherent to success in an orthopaedic residency training. Sutton et al10 examined the correlation between USMLE Step 1 Scores, board score, and faculty evaluations of general surgery residents and found that good applicants with lower than average board scores could go on to be successful residents and pass their surgical boards. However, such applicants may be lost from the applicant pool with increasing threshold cutoffs of USMLE Step 1 scores used in screening applicants. Using USMLE cutoffs in the residency application process may be contributing to the loss of minority applicants with lower test scores for consideration of acceptance, with minimal contemplation of additional indicators or types of achievement. Interestingly, the National Association for College Admissions and Counseling has reported that removing standardized testing requirements for undergraduate admissions (ie, test optional admissions policies) has led to an increase in the representation of URM students (both numeric and proportionate) in the undergraduate applicant pool.11
Induction into AOA is made in each medical school for up to one sixth of students in each class. Selected students must be in the top quartile of their class and show leadership among their peers, professionalism, and a commitment to community service. AOA status has been found to be associated with successful performance on the American Board of Orthopaedic Surgery (ABOS).12 In 2008, Herndon et al12 looked at the predictors of success on the ABOS I examination, including the number of honors in four selected clerkships (medicine, surgery, pediatrics, and orthopaedics), membership in AOA, and whether the resident scored above or below the mean on the USMLE examination. Among the residents who achieved AOA status, 96% passed part I and 98% passed part II on the first attempt, compared with non-AOA members who had an 81% pass rate on part I and 82% pass rate on part II. The study also showed that White applicants to orthopaedic surgery residency programs were more likely to have AOA status compared with minority applicants, which is consistent with findings of the current study. AOA status is predictive of successful passage of ABOS, and thus it is useful as a criterion for residency selection. The lower incidence of minority applicants with AOA status may also contribute to the lower match rates of this group into orthopaedic surgery.
The “leaky pipeline theory” postulates that sequential losses of URM interested in medicine and orthopaedic surgery throughout the educational pipeline from high school to residency contributes to the lack of diversity in residency programs.13 Orthopaedic surgery remains with the lowest representation of URM among all medical specialties, despite a clear need for more minority physicians.4 It is important to maintain the availability of professional development programs for URM students because they have been successful at increasing the number of students entering a given profession. For example, Nth Dimensions created a target pipeline curriculum with an Orthopaedic Summer Internship Program for medical students to provide early exposure and mentorship within orthopaedics. For URMs, completion of the internship program was associated with increased odds of applying and matching into orthopaedic surgery residency programs compared with the national average.13 Medical schools also have a responsibility to expose students to orthopaedic surgery and give students the opportunity to interact with potential mentors. Okike et al5 reported that musculoskeletal medicine courses in medical school have been shown to increase the likelihood that students will apply to orthopaedic surgery residency programs, and the observed increases have been disproportionately large among URM and women. Increased exposure to orthopaedic surgery can lead to the development of mentoring relationships and help to dispel potential negative perceptions of orthopaedic surgery as a profession.14 By appealing to all qualified medical students, regardless of race or ethnicity, orthopaedic surgery residency training programs can ensure that they attract the best and brightest applicants.
This study is limited by the fact that the data presented are AAMC and NBME data from 2005 to 2014, and therefore, these may not be representative of current and future applicants. The enrolled candidate data come from the GME Track Resident Survey from 2005 through 2014. The estimated response rates were 79.0% to 94.5% of programs and may not be representative of all the orthopaedic residents who enrolled during the study period. Additionally, we were only able to analyze the variables available through the AAMC, which do not characterize the candidates as a whole. There are many other factors that contribute to the strength of each applicant that are not evaluated in the current study. Specifically, sex differences were not included in this analysis. Intersectional race and sex factors, which may have interactive effects, may alter findings of this study, which were based solely on race/ethnicity. Nevertheless, the regression analyses performed using large sample sizes in this study are sufficiently powered (≥80% power) to detect medium effect sizes with α = 0.05, and a stringent post hoc test (Tukey Honest Significant Difference) was used to make multiple group comparisons. Future analyses that encompass additional variables and more complex statistical models are warranted.
The academic characteristics of applicants for orthopaedic surgery residencies have increased significantly over the past 10 years for both White and minority students. Although the metrics analyzed in this study are necessary for assessing applicant characteristics, persistent differences in the USMLE Step 1 and Step 2 CK scores and AOA status exist on the basis of race/ethnicity. This difference may be contributing to lower enrollment by URMs. Although standardized test scores were lower for Black and Hispanic applicants compared with White and Asian applicants, the lower enrollment rate is out of proportion with the differences in academic achievements compared with Asian and White applicants. Although URM outperformed Whites and Asians in publications or volunteer experiences, this does not appear to be adding sufficient impact to improving the enrollment rates of minority applicants over time. This is the first study to quantify the difference in enrollment rates of minority candidates in orthopaedic surgery, which appear to be more variable year to year for URM compared with White and Asian applicants. The significant differences in both the enrollment rates and objective academic metrics of applications between URM and Whites require further consideration by residency selection committees to begin enhancing racial and ethnic diversity in orthopaedic surgery residency programs.