Increasing the size and diversity of the federally funded physician–scientist workforce are national priorities.1–3 Currently, investigators from some racial/ethnic groups are less likely than others to receive research funding as independent investigators.4 Mentored K (K01/K08/K23) career development awards are positively associated with physicians’ success in becoming federally funded independent investigators.1,5 An evaluation of the National Institutes of Health (NIH) K award program has shown that there have been fewer blacks, Hispanics, and Native Americans among mentored K award program applicants and awardees compared with their representation in PhD and graduating medical school classes.5 As candidate review criteria for mentored K awards include consideration of an applicant’s research experiences and academic record,6–8 we hypothesized that research experiences and academic performance would explain the association between race/ethnicity and mentored K award receipt. We conducted a retrospective, national cohort study of medical school graduates who planned to pursue research-related careers. Using mediation analysis, we measured the extent to which certain variables explained the relationship between race/ethnicity and mentored K award receipt.
Our database included individual, deidentified records for 129,867 matriculants in academic years 1993–1994 through 2000–2001 at U.S. Liaison Committee for Medical Education (LCME)-accredited medical schools. Using graduation date, we included in our analysis those individuals who graduated from 1997 through 2004, followed through August 2014 (allowing for ≥ 10 years, postgraduation follow-up). Washington University School of Medicine’s Institutional Review Board approved this study as non-human-subjects research.
We explored several potential mediators of the association between race/ethnicity and mentored K award receipt based on the literature.1,2,4,5,9–13 The 10 variables include 4 research activities (participating in a research elective during medical school, authorship of a research report submitted for publication, spending at least one year during residency doing research, and receipt of a federal research fellowship [F32] grant), 2 academic performance measures (scores on the United States Medical Licensing Examination [USMLE] Step 1 and Step 2 Clinical Knowledge [CK]), plus medical school research intensity, degree program, debt at graduation, and specialty. We describe each of these potential mediators and our data sources in detail below.
In 2014, the Association of American Medical Colleges (AAMC) provided us with updated deidentified data for this cohort of matriculants from the AAMC’s Student Records System (SRS), Graduation Questionnaire (GQ), and Graduate Medical Education (GME) Track and from the Information for Management, Planning, Analysis, and Coordination (IMPAC) II database (the internal NIH database of extramural federal applications and awards14). Each individual’s records were merged using unique AAMC-generated identification numbers.
The SRS contains enrollment and tracking information on the national population of medical students, from matriculation through graduation. The SRS includes information provided by medical school registrars, who regularly update their school’s student records.15 SRS variables included in our study are graduation year, sex, and race/ethnicity. The categories for race/ethnicity are as follows:
- Asian/Pacific Islander (Asian/PI, including Japanese, Filipino, Vietnamese, Korean, Chinese, Indian, Pakistani, other Asian, Hawaiian, and other Pacific Islander);
- Underrepresented minorities in medicine (URM, including black, Hispanic, and Native American/Alaska Native);
- other/multiple/unknown (including “other,” multiple races, or no race/ethnicity reported);
- and white.
We created a dichotomous race/ethnicity variable for analysis (non-URM [including Asian/PI and white] vs. URM), excluding graduates of other/multiple/unknown race/ethnicity.
We included six items from the GQ, a national questionnaire administered on a confidential and voluntary basis to all graduating medical students in the spring of their final year.16 GQ items addressed issues critical to the future of medical education and students’ well-being.16 Because we lacked grant application information, we limited our inclusion to only those graduates who indicated on the GQ that their career intentions included a research component: “Full-time University Faculty: Basic science teaching/research,” “Full-time University Faculty: Clinical teaching/research,” or “Other: Non-university research scientist.” We excluded graduates who indicated any other career intention on the GQ (including multiple options for “Full-time Clinical Practice” in various settings, “Other” non-research-related and non-practice-related options, and “Undecided”). Another of the GQ items we included asked graduates to “Indicate the activities you will have participated in during medical school on an elective or volunteer (not required) basis.” We examined yes/no responses to “Research project with faculty member” (hereafter “medical school research elective”) and “Authorship (sole or joint) of a research paper submitted for publication” (hereafter “authorship”). We created a four-category variable for total debt at graduation (no debt; $1–$49,999; $50,000–$99,999; and ≥ $100,000). Based on responses to two items about intended specialty for board certification, we created a seven-category specialty variable (internal medicine, family medicine, pediatrics, obstetrics–gynecology, surgery, no/undecided about board certification plans, and all other nongeneralist/nonsurgical specialties). We created a three-category variable for “degree program at graduation”: MD–PhD, MD-other-advanced (non-PhD [e.g., MPH]), and MD (including MD, BA–MD, and BS–MD).
The AAMC also provided scores from students’ first attempts on the USMLE Step l and Step 2 CK.17 These standardized examinations assess examinees’ knowledge of the basic sciences (Step 1) and of the clinical sciences (Step 2 CK) important to the practice of medicine.17 The AAMC also provided a variable indicating attendance (yes/no) at a research-intensive medical school (top 40 ranked for NIH funding).18 We included a variable from the AAMC GME Track indicating whether or not graduates completed at least one year of research during GME (yes/no) as reported by their program directors on the National GME Census, which is administered jointly by the AAMC and the American Medical Association.19,20 The National GME Census, voluntarily completed annually by residency program directors and institutional officials, includes a survey component that inquired about the training status and activities of each resident and fellow. The AAMC GME Track is the database that contains the National GME Census data.19
Using a set of multiple identifiers shared between the AAMC and the NIH (e.g., full name, sex, medical school name, graduation year), we obtained publicly available IMPAC II awards data from federal records of individual research grants awarded to graduates in our cohort. Net ESolutions Corporation (Bethesda, Maryland), contracted by the NIH and AAMC, conducted the record match and provided awards data to the AAMC on our behalf; the AAMC provided deidentified awards data to us. We created two binary variables: (1) for receipt (vs. no receipt) after medical school graduation of F32 postdoctoral fellowship award and (2) for the outcome, mentored K (K01/K08/K23) award (hereafter “K award”) receipt.
We used chi-square tests to describe associations among categorical variables, and analysis of variance (ANOVA) to describe between-group differences in continuous variables. We reported descriptive statistics for each variable examined, grouped by race/ethnicity and K award receipt.
We examined correlations among potential mediators as follows. We measured tetrachoric correlations between binary variables, biserial correlations between binary and continuous variables, polychoric correlations between binary and ordinal variables, polyserial correlations between continuous and ordinal variables, Cramer V for associations between nominal and/or ordinal categorical variables, ANOVA for associations between continuous and nominal categorical variables, and Pearson product–moment correlations between continuous variables.
We examined the potential mediating effect of each of the 10 variables on the relationship between race/ethnicity and K award receipt in models comparing non-URM versus URM graduates. We controlled for sex and graduation year in all models (neither sex nor graduation year was a manipulable variable and therefore not considered a potential mediator). Figure 1 illustrates the paths of our mediation model, in which race/ethnicity is examined in association with the outcome, K award receipt (Path C); race/ethnicity is examined in association with each mediating variable (Path A); and each mediating variable is examined in association with K award receipt (Path B).
Within the specified mediation framework, we followed approaches suggested by Baron and Kenny21 and by Judd and Kenny22 to empirically evaluate the 10 potential mediating variables we selected. First, we measured the association of race/ethnicity with K award receipt using a binary logistic regression model (Path C) adjusting for covariates, sex and graduation year. Then, we measured the association of race/ethnicity with each potential mediating variable using appropriate logistic or linear regression models, with each mediating variable as the outcome and race/ethnicity as a predictor (Path A), adjusting for sex and graduation year. Next, we measured the association of each potential mediating variable with K award receipt using binary logistic regression models (Path B), including race/ethnicity as a covariate in addition to sex and graduation year.
We selected potential mediating variables that were significantly associated with both race/ethnicity (Path A) in the hypothesized direction (e.g., non-URM graduates were more likely than URM graduates to have had particular research experiences) and K award receipt (Path B) for mediation analysis. To examine whether there were potential race/ethnicity–mediator interactions that might bias estimates of the proportion of the effect of race/ethnicity on K award receipt in mediation analysis, we used logistic regression models to test the interactions between race/ethnicity and each potential mediator on K award receipt, adjusting for sex and graduation year.
The mediation effect was quantified by the proportion of the effect of race/ethnicity on K award receipt that is explained by a mediator.23 The proportion of the effect of each mediator on the association between race/ethnicity and K award receipt was obtained by, first, estimating regression coefficients of race/ethnicity on K award receipt with and without the mediator, adjusting for sex and graduation year; and, then, dividing the difference between the two regression coefficients by the regression coefficient from the model without the mediator. We used the public SAS macro MEDIATE24 for estimation and statistical inference (confidence interval [CI] and test of significance) of the mediation effect for each mediator alone, for all significant research activities together as a block, and for all significant mediators together as a block. We performed analyses using SAS version 9.3 (SAS Institute, Inc., Cary, North Carolina), and we considered two-sided P values of < .05 to be statistically significant.
Of the 129,867 matriculants in U.S. LCME-accredited medical schools in academic years 1993–1994 through 2000–2001 in our database, 119,906 graduated in 1997–2004, including 28,690 graduates who had indicated research-related career intentions at graduation on the GQ and were thus eligible for inclusion in our study. We excluded 498 graduates of other/multiple/unknown race/ethnicity, 30 with missing Step 1 and/or Step 2 CK score data, and 641 with missing GQ data for one or more of the GQ items of interest. Our final sample included 27,521 graduates with complete data (95.9% of the 28,690 graduates eligible for study inclusion). Among the 27,521 graduates included in the final sample and the 1,169 graduates excluded from the final sample because of missing data, there were similar proportions of K award recipients (1,147/27,521 [4.2%] vs. 57/1,169 [4.9%], respectively; P = .24), research-intensive medical school graduates (12,411/27,521 [45.1%] vs. 496/1,169 [42.4%], respectively; P = .07), graduates who had participated in ≥ 1 GME research year (6,130/27,521 [22.3%] vs. 235/1,169 [20.1%], respectively; P = .08), and F32 award recipients (310/27,521 [1.1%] vs. 13/1,169 [1.1%], respectively; P = .96).
Table 1 shows descriptive statistics of the study sample for each covariate and potential mediator grouped by race/ethnicity and by K award receipt. A lower proportion of URM graduates (79 of 3,341 [2.4%]), compared with non-URM graduates (1,068 of 24,180 [4.4%]), received K awards. Consistent with the hypothesized direction of associations, higher proportions of non-URM than URM graduates reported each of the following: a medical school research elective, authorship, and MD–PhD program graduation. In addition, higher proportions of non-URM than URM graduates were reported to have had ≥ 1 GME research year, and to have received F32 awards. Mean Step l and Step 2 CK scores were higher among non-URM graduates compared with URM graduates.
We examined the relationship between race/ethnicity and K award receipt (Figure 1, Path C) in a logistic regression model that controlled for sex and graduation year. Non-URM graduates were more likely than URM graduates to be K award recipients (adjusted odds ratio, 1.90; 95% CI, 1.50–2.39).
Associations among potential mediators (Supplemental Digital Tables 1–5, http://links.lww.com/ACADMED/A471) were generally of low magnitude except the correlations between medical school research elective and authorship (tetrachoric correlation = 0.76; Supplemental Digital Table 1) and between GME research year and F32 award receipt (tetrachoric correlation = 0.45; Supplemental Digital Table 1), and the correlation between Step l and Step 2 CK scores (Pearson product–moment correlation = 0.75; Supplemental Digital Table 3).
Table 2 shows the association between race/ethnicity and each categorical potential mediator (Path A) in separate logistic regression models. As shown, non-URM (vs. URM) graduates were more likely to have attended a research-intensive medical school, reported a medical school research elective and authorship, graduated from an MD–PhD degree program, participated in ≥ 1 GME research year, and received an F32 award. Non-URM graduates were less likely to report any debt (each level vs. no debt) and choice of any other specialty category except pediatrics (each vs. internal medicine). In the ordinary least-squares linear regression models examining the associations between race/ethnicity and Step scores (Path A), non-URM (vs. URM) graduates were more likely to have higher Step l and Step 2 CK scores. Step 1 and Step 2 CK scores were each, on average, 17.0 points higher (standard error, 0.4) for non-URM than for URM graduates.
Table 3 shows the association between each potential mediator and K award receipt (Path B). As shown, graduates who attended research-intensive medical schools, reported a medical school research elective and authorship, were MD–PhD and MD-other-advanced-degree program graduates, had higher Step l and Step 2 CK scores, participated in ≥ 1 GME research year, and were F32 award recipients were more likely to be K award recipients. Graduates who reported debt of $50,000–$99,999 or ≥ $100,000 (each vs. no debt) and who chose every specialty category except pediatrics (each vs. internal medicine) were less likely to be K award recipients.
We did not observe any significant interactions between race/ethnicity and any potential mediator on K award receipt.
Table 4 shows the proportion of the effect of race/ethnicity on K award receipt explained by each mediator alone and by blocks of significant mediators, controlling for sex and graduation year. Significant single-mediator effects were observed for each potential mediator except debt. The largest single-mediator effect was observed for Step l score, which explained 80.3% of the effect of race/ethnicity on K award receipt. The block of all research activity mediators explained 81.5% of the effect of race/ethnicity on K award receipt. The block of all nine significant mediators explained 96.2% of the effect of race/ethnicity on K award receipt.
Through our mediation analysis, we identified nine variables that explained the association between race/ethnicity and K award receipt among U.S. LCME-accredited medical school graduates planning to pursue research-related careers. As we hypothesized, research activities (each alone and as a block) explained much of the observed racial/ethnic disparity in K award receipt. Thus, targeted efforts to promote greater participation of interested URM students in substantive and productive research activities during and after medical school could serve to mitigate racial/ethnic disparities in K award receipt.
We used Step 1 and Step 2 CK scores to test our hypothesis that academic performance would explain racial/ethnic disparities in K award receipt. The Step l single-mediator effect nearly equaled the effect of all four research activities together. Observations of lower Step l and Step 2 CK scores among racial/ethnic minority graduates were initially reported 20 years ago.25 Step l and Step 2 CK scores, which are only two of numerous measures of medical school academic performance, are not themselves part of the K award review process; however, these standardized test scores correlate with other preclinical and clinical medical school academic performance measures.26–28 Although Step scores have been noted to have limited ability to predict success in clinical medicine or biomedical research,29 these scores (particularly Step l) are extensively used in the GME resident selection process.29–31 Of 33 factors used by program directors to select applicants to interview for their programs, Step l score was the most frequently used factor, cited by 94% of 1,793 Program Director Survey respondents.30 Many program directors reported using a “target” Step l score in considering which applicants to interview,30 a practice shown to disproportionately negatively impact black applicants.32 Step l and Step 2 CK scores have also been shown to independently predict match success; higher-scoring U.S. medical students were more successful than their lower-scoring peers in gaining entry into their preferred residency training positions.33 Thus, our Step l and Step 2 CK findings may reflect, in part, differences in residency program characteristics that may be associated with K award receipt (e.g., availability and quality of research opportunities and mentoring for residents interested in research). Importantly, we note that since we had award receipt but not application data, our observations regarding Step scores might reflect not only differences in application success (if lower-scoring applicants were less likely to receive mentored K awards) but also the applicant pool (if lower-scoring graduates opted not to enter the mentored K applicant pool or were discouraged from doing so). Further research is warranted to determine whether a medical school graduate’s academic record serves as a barrier to entering the K award applicant pool.
We observed a single-mediator effect for specialty. Graduates who chose internal medicine were overrepresented among K award recipients. This finding aligns with other reports that many mentored K award applicants5 and about half of all physician recipients of K08 and K23 awards were affiliated with departments of medicine and related specialties.34,35 URM graduates were overrepresented in family medicine and obstetrics–gynecology, and both specialties had very low proportions of K award recipients. These findings extend the evidence for why there may be low levels of engagement of family medicine specialists in the NIH research enterprise.36 Specialty-specific interventions to promote research opportunities in family medicine and obstetrics–gynecology might serve to attract and/or retain interested URM graduates as funded researchers in the federally funded biomedical research workforce.37
The single-mediator effect observed for degree program at graduation provides evidence for the benefits of participating in MD–PhD programs1,3,38 as a route for training a more diverse physician–scientist workforce. Finally, we also observed a single-mediator effect for medical school research intensity. We speculate that greater availability of and access to highly accomplished research mentors and resources at research-intensive medical schools may help explain the effect of this institutional factor on the racial/ethnic disparity in K award receipt.
Our study has several strengths. We built a large, national cohort database using data from the AAMC, the National Board of Medical Examiners, and NIH IMPAC II. We included numerous variables not previously examined in association with racial/ethnic disparities in physicians’ federal research award receipt (e.g., total debt at graduation, specialty choice at graduation, medical school academic performance).
Our study also has limitations. Although the observed mediators of the association between race/ethnicity and K award receipt suggest potential areas for intervention that might serve to increase the diversity of the physician–scientist workforce, we cannot infer causation from these associations. Also, we relied on self-reported GQ data for several variables; self-reported data are prone to social desirability bias and reflect respondents’ interpretation of items. National GME Census data pertaining to GME year(s) of research were based on program-reported survey data; therefore, the total number of graduates in our study who had completed at least one year of research during GME may be underreported. Furthermore, because the academic and professional development continuum for physician–scientists is remarkably lengthy, K award receipt must be considered a long-term outcome. K01 applicants are typically three to five years past their terminal degree, while K08 and K23 applicants are typically seven to nine years beyond their terminal degree.5 At least some of the potential mediators we examined may have changed over time. In particular, reports from the National Resident Matching Program (NRMP) indicate that USMLE Step l and Step 2 CK scores, both overall and on a specialty-specific basis, have steadily increased in recent years among U.S. LCME-accredited medical student participants in the NRMP.39,40
Our findings are not generalizable to graduates of non-LCME-accredited medical schools (e.g., osteopathic or international schools). Also, our findings pertained only to those U.S. LCME-accredited medical school graduates who indicated on the GQ that they planned research-related careers; our results may not generalize to GQ nonrespondents or to GQ respondents who indicated other career plans at graduation.
For this study, we also were limited to analysis of award data that are publicly available under the Freedom of Information Act; we did not receive applicant data that were not in the public domain.41 Thus, our findings may reflect differences by race/ethnicity in application rates among graduates and/or funding rates among applicants. Finally, there may be other, unmeasured factors that mediate the association between race/ethnicity and K award receipt (e.g., other academic performance measures and the quality of specific research experiences during medical school and GME).
In summary, we identified multiple variables that explained the racial/ethnic disparity observed in K award receipt. Our findings suggest several research-related strategies that medical schools and GME programs might use to increase the diversity of the physician–scientist biomedical research workforce, including:
- Increasing opportunities for interested URM students to participate in productive research experiences during and after medical school,
- Recruiting greater numbers of interested URM students to joint MD–PhD programs,
- Increasing GME research opportunities for interested URM trainees in family medicine and obstetrics–gynecology that sustain their research-related career intentions.
Additionally, our findings may be of interest to the federal agencies and other institutions that support efforts to recruit and educate a diverse physician–scientist workforce.
The authors thank Paul Jolly, PhD, retired from the Association of the American Medical Colleges (AAMC), and Emory Morrison, PhD, formerly of the AAMC, for provision of the data and assistance with coding; the National Board of Medical Examiners for permission to use deidentified United States Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge scores; James Struthers and Maria Pérez, MA, in the Division of General Medical Sciences at Washington University School of Medicine, for assistance with data management and administrative support; and Andrea Myles, at a.m. graphics, llc, for graphic design services.
2. Garrison HH, Deschamps AM. NIH research funding and early career physician scientists: Continuing challenges in the 21st century. FASEB J. 2014;28:1049–1058.
3. Milewicz DM, Lorenz RG, Dermody TS, Brass LF; National Association of MD–PhD Programs Executive Committee. Rescuing the physician–scientist workforce: The time for action is now. J Clin Invest. 2015;125:3742–3747.
4. Ginther DK, Haak LL, Schaffer WT, Kington R. Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Acad Med. 2012;87:1516–1524.
10. Jeffe DB, Andriole DA. A national cohort study of MD–PhD graduates of medical schools with and without funding from the National Institute of General Medical Sciences’ Medical Scientist Training Program. Acad Med. 2011;86:953–961.
11. Ginther DK, Schaffer WT, Schnell J, et al. Race, ethnicity, and NIH research awards. Science. 2011;333:1015–1019.
16. Association of American Medical Colleges. Graduation Questionnaire (GQ). https://www.aamc.org/data/gq/
. Published 2017. Accessed June 15, 2017.
17. Federation of State Medical Boards of the United States Inc.; National Board of Medical Examiners (NBME). United States Medical Licensing Examination: Bulletin of information. http://www.usmle.org/pdfs/bulletin/2016bulletin.pdf
. Published 2016. Accessed June 8, 2017.
18. Moy E, Griner PF, Challoner DR, Perry DR. Distribution of research awards from the National Institutes of Health among medical schools. N Engl J Med. 2000;342:250–255.
20. Brotherton SE, Etzel SI. Graduate medical education, 2009–2010. JAMA. 2010;304:1255–1270.
21. Baron RM, Kenny DA. The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51:1173–1182.
22. Judd CM, Kenny DA. Process analysis: Estimating mediation in treatment evaluations. Eval Rev. 1981;5:602–619.
23. Freedman LS, Graubard BI, Schatzkin A. Statistical validation of intermediate endpoints for chronic diseases. Stat Med. 1992;11:167–178.
25. Case SM, Swanson DB, Ripkey DR, Bowles LT, Melnick DE. Performance of the class of 1994 in the new era of USMLE. Acad Med. 1996;71–S93:S91–S93.
26. Andriole DA, Jeffe DB, Hageman HL, Whelan AJ. What predicts USMLE Step 3 performance? Acad Med. 2005;80(10 suppl):S21–S24.
27. Zahn CM, Saguil A, Artino AR Jr, et al. Correlation of National Board of Medical Examiners scores with United States Medical Licensing Examination Step 1 and Step 2 scores. Acad Med. 2012;87:1348–1354.
28. Casey PM, Palmer BA, Thompson GB, et al. Predictors of medical school clerkship performance: A multispecialty longitudinal analysis of standardized examination scores and clinical assessments. BMC Med Educ. 2016;16:128.
29. Gliatto P, Leitman IM, Muller D. Scylla and Charybdis: The MCAT, USMLE, and degrees of freedom in undergraduate medical education. Acad Med. 2016;91:1498–1500.
31. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med. 2016;91:12–15.
32. Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76:1253–1256.
34. Jagsi R, DeCastro R, Griffith KA, et al. Similarities and differences in the career trajectories of male and female career development award recipients. Acad Med. 2011;86:1415–1421.
35. Jagsi R, Griffith KA, Stewart A, Sambuco D, DeCastro R, Ubel PA. Gender differences in salary in a recent cohort of early-career physician–researchers. Acad Med. 2013;88:1689–1699.
36. Lucan SC, Phillips RL Jr, Bazemore AW. Off the roadmap? Family medicine’s grant funding and committee representation at NIH. Ann Fam Med. 2008;6:534–542.
37. Fabris F, Rice TK, Jeffe DB, Czajkowski SM, Boyington J, Boutjdir M. Junior faculty career development through an NHLBI program to increase diversity in cardiovascular health-related research. J Am Coll Cardiol. 2016;67:2312–2313.
38. Jeffe DB, Andriole DA, Wathington HD, Tai RH. The emerging physician–scientist workforce: Demographic, experiential, and attitudinal predictors of MD–PhD program enrollment. Acad Med. 2014;89:1398–1407.
Reference cited in Table 4 only
42. Kaufman JS, MacLehose RF, Kaufman S. A further critique of the analytic strategy of adjusting for covariates to identify biologic mediation. Epidemiol Perspect Innov. 2004;1:1.