Ginther, Donna K. PhD; Haak, Laurel L. PhD; Schaffer, Walter T. PhD; Kington, Raynard MD, PhD
The National Institutes of Health (NIH) has sponsored many programs designed to increase the diversity of the biomedical research workforce in the United States; these include Minority Access to Research Careers, Minority Biomedical Research Support, Research Centers at Minority Institutions, and Diversity Supplements. However, it is not clear to what extent these efforts are translating into more diverse faculty at medical schools and other research-oriented institutions. The Association of American Medical Colleges (AAMC) recently reported that although there has been an increase in faculty diversity since the mid-1960s, there is greater diversity today among medical students than among medical school faculty.1
The tenure system at medical schools may also affect faculty diversity. According to a recent AAMC report, tenure may not be accompanied by financial support: Among medical schools offering tenure to faculty, 41% did not provide tenure-related financial guarantees to clinical faculty, and 38% did not provide such guarantees to basic science faculty.2 Thus, across demographic groups, medical school faculty may increasingly rely on external funding for salary support. Racial/ethnic differences in the probability of receiving external funding, such as NIH R01 grants, could therefore affect faculty diversity and, in turn, limit potential gains from the efforts the NIH and other organizations are making to train a diverse cadre of clinician researchers.
A recent National Academies report emphasized the need to increase the participation of underrepresented minorities in science and engineering,3 but there have been few studies of the mechanisms underlying underrepresentation.4 Most studies of biomedical researchers and academic faculty have focused on differences in representation by race and ethnicity.5,6 When we7 examined proportional representation by race and ethnicity at educational and career milestones in biomedicine, we found that underrepresented groups had lower transition rates from high school to college completion in the biological sciences and from college completion to graduate school degree completion. In other words, underrepresentation of minorities occurred prior to the point at which they would embark on academic careers. Interestingly, however, we found that the probability of transition from college to medical school was higher for blacks and for Hispanics than for whites. In addition, among faculty with PhDs in biomedicine, blacks were significantly more likely than whites to obtain tenure-track jobs and receive tenure.
Given the centrality of workforce diversity to the NIH objective of improving the nation’s health through translational biomedical research,8 it is important to examine the racial and ethnic composition of the investigators who make up the NIH research grant applicant pool. In a recent study, we found that NIH R01 Type 1 applications from black and Asian investigators with PhDs were significantly less likely than those from white PhDs to be awarded funding after controlling for many observable characteristics.9 However, we did not consider whether the same relationships held for applications from MD applicants. In this study, we explore the relationship between the race/ethnicity of MD and MD/PhD applicants and applications’ NIH R01 award probability. We then identify additional characteristics that may be associated with application award probability, such as applicants’ employment at a medical school and research productivity.
NIH peer-review process
NIH funding is awarded through a peer-review process in which an NIH review committee—a panel of subject matter experts—considers the grant application’s significance, innovation, and approach as well as its investigator(s) and research environment. About half of all applications are deemed competitive; these are discussed by the panel, and each is given a priority score. Funding is allocated to scored proposals according to priority score rank and considerations related to NIH institute priorities and budgets, which vary by year and by institute. Initially unsuccessful applications may achieve funding success after being revised and resubmitted one or more times.
We used information from several databases to perform this research. Administrative data from the NIH IMPAC II (Information for Management, Planning, Analysis, and Coordination) grants management system were made available under contract to Discovery Logic, with specific and stringent security regulations. (NIH also provides some information from this database publicly through the NIH RePORTER system.)10 Department of Education Integrated Postsecondary Education Data System (IPEDS) data were obtained from the National Center for Education Statistics Web site.11 Data from the AAMC Faculty Roster12 were linked to records in the NIH grant database, as were data from the National Science Foundation (NSF) Survey of Earned Doctorates (SED).13 These data were made available under license to the NIH and, after request and additional security review, to Discovery Logic under its contract with the NIH. Publication information was obtained from Thomson Reuters, of which Discovery Logic is a subsidiary.14 Matching of NIH R01 grant records to external data sets was performed in a secure environment by authorized staff. All statistical analysis was performed on a deidentified dataset provided to the authors by Discovery Logic.
The core data for this study were application and applicant data for R01 Type 1 Research Project Grants submitted to the NIH between fiscal year (FY) 2000 and FY 2006; these data came from the NIH IMPAC II grants management system.15 We focused on the R01 mechanism because it is the “gold standard” for investigator-initiated research funding. Type 1 denotes new proposals. The initial sample dataset was stored in a Microsoft SQL Server 2005 database.
Applicants could and did submit more than one R01 application during the study time frame. We collapsed any revised or related submissions that were received within two years of the original submission into one application for the purposes of determining the award probability of the application. Information about an application and its review were derived from the last funded or unfunded application submitted. We did not include new proposals submitted in 2007 and 2008 because we could not observe them for the additional two-year period without introducing changes that resulted from the new NIH scoring system implemented in 2009 and the dramatic increase in funding from the American Recovery and Reinvestment Act of 2009.
We limited the sample to R01 Type1 grant applications from PhD, MD, or MD/PhD investigators at U.S. institutions. We omitted applications from outside the United States, those that were missing information on both investigator race and age, and those without a PhD or MD investigator.
The analysis sample from IMPAC II was supplemented by institutional information from IPEDS. Race/ethnicity, sex, and age were derived from self-reports in IMPAC II as well as from information in the AAMC Faculty Roster and NSF SED. Information on PhD degrees for PhD and MD/PhD investigators was also derived from the NSF SED. We matched publication and citation information from Thomson Reuters Web of Science and Journal Citation Reports to individual applicants; 84% of grant applications were matched to publications with greater than 90% confidence. Details and limitations of the publications-matching process are available in the supplemental online material for our previous study of PhDs.16
As noted above, we determined applicants’ race and ethnicity using self-reported data from IMPAC II, the AAMC Faculty Roster, and the NSF SED. Individuals can change their gender, race/ethnicity, and birth date in IMPAC II, so when there were multiple observations in these fields, we used the one that was most frequently reported. There were small numbers of applications from Native Americans, Alaska Natives, Hawaiian and other Pacific Islanders, and individuals in the “other” race category. To protect confidentiality, we combined these data in reporting our descriptive statistics. Our regression models included separate controls for Native Americans, Alaska Natives, Hawaiian and other Pacific Islanders, other race, and race unknown, but we do not report results for these categories in this article. Reviewers do not directly observe race/ethnicity on NIH grant applications. However, it is possible that reviewers may use names, affiliations, and application subject matter to infer applicants’ race/ethnicity.
We evaluated racial/ethnic differences in R01 award probability by controlling for as many characteristics as possible, including those present on the application’s biographical sketch. As in our previous study,9 we included variables for
* demographic characteristics: race/ethnicity, gender, age, age-squared;
* education and training: degree type, NIH pre-/postdoctoral fellowships (F), pre-/postdoctoral traineeships (T), or career development awards (K);
* employer characteristics: NIH funding rank, institution type, geographic region;
* experience with the NIH funding process: indicators for NIH institute, prior NIH grants, NIH review committee member, fiscal year of application, inclusion of human subjects; and
* research productivity: publication quartiles, citation quartiles, maximum journal impact factor, median journal impact factor, ratio of first-authored to total publications, ratio of last-authored to total publications, ratio of single-authored to total publications.
We included these variables because of their likely association with R01 award probability. We expected that NIH F, T, or K training would provide good preparation for a research career and would be positively associated with funding. We included employer characteristics such as funding rank to control for employers’ research activities and resources. Additionally, we split the sample by two employer types—medical schools and other, non-medical-school institutions (referred to as “other institutions” in this study)—to explore whether affiliation affects funding probability. Our models controlled for differences in funding rates across NIH institutes, annual differences in success rates,17 and differences in applicants’ experience with the NIH funding process. We also included controls for research productivity at the time of application, which is a factor in the peer-review panel’s assessment of applicant qualifications. We were not able to control for applicants’ education characteristics and citizenship as we did in our previous study of PhDs9 because that information was derived from the NSF Doctorate Records File and is only available for PhDs educated in the United States.
The majority of this study’s variables were dummy variables with the exception of age, journal impact factor, and the ratio of first-, last-, and single-authored publications to total publications. The continuous variables of publications, citations, and employer NIH funding rank were divided into categories to improve the goodness of fit of the models. To determine an organization’s NIH funding rank, we averaged its annual NIH grant support for FY 2000 through FY 2006, using data from IMPAC II. We then ranked employers in descending order of the total grant dollars received. We divided citation counts and publication counts into quartiles at the 25th, 50th, and 75th percentiles. We included dummy variables that equaled 1 in the specification to control for missing information for variables. Our models included dummy variables to account for missing information in the MD sample on race (16% missing), inclusion of human subjects (two applications missing information), and publications (12% missing).
The analysis began with several one- and two-sample tests of means and proportions between medical schools and other institutions, and between MDs and PhDs. Throughout the analysis, we grouped MD/PhDs with MDs because there were no significant differences in outcomes between these two groups. Although most MDs in our sample worked at medical schools, a substantial number worked at other institutions (half at hospitals and the remainder at research institutions, higher education institutions, and other organizations).
Next, we used probit models to estimate the probability of an MD investigator’s application receiving an R01 award conditional on the applicant’s race/ethnicity and the explanatory variables. In place of reporting probit coefficients, we report the marginal effect of the variable on the award probability, which is the change in the award probability due to each predictor separately when other variables are evaluated at their mean values. In the case of applicant race/ethnicity, the marginal effects can be interpreted as the percentage point difference in R01 award probability between applications from white investigators (the omitted reference category in the regression) and applications from investigators of a given race/ethnicity. We used heteroskedasticity-robust standard errors that were clustered on the individual applicant to adjust for the fact that applicants could (and did) submit more than one R01 application in the study sample. The resulting regression estimates are correlations between the covariate and the probability that an application will receive an R01 award, and should not be interpreted as having a causal impact. The data were analyzed using Stata 12 software (StataCorp LP, College Station, Texas).
During FY 2000–2006, investigators submitted 106,368 R01 Type 1 grant applications. After we dropped applications from outside the United States, those missing both race and age data, and those without a PhD or MD investigator, our sample consisted of 97,877 applications from 47,424 applicants with an MD or PhD at U.S. institutions that had data available for most of the explanatory variables. Each applicant in this sample submitted an average of 2.06 applications.
Racial and ethnic distribution
In our sample, the racial and ethnic distribution of the R01 MD applicants at medical schools was not distinguishable from that of MD faculty in the AAMC Faculty Roster (AAMC MD faculty) for the period FY 2000–2006. Whites constituted 69% (109,372/157,679) of the AAMC MD faculty and 64% (6,402/9,993) of the R01 MD applicants, whereas blacks constituted 4% (5,574/157,679) of the AAMC MD faculty and 2% (159/9,993) of the R01 MD applicants. Asians represented the same proportion of both samples (12%; 19,046/157,679 AAMC MD faculty and 1,188/9,993 R01 MD applicants). Hispanics made up 5% (7,395/157,679) of AAMC MD faculty and 4% (400/9,993) of the R01 MD applicants. Native American, Alaska Natives, Hawaiian and other Pacific Islanders, other race, and unknown race groups together represented 10% (16,292/157,679) of the AAMC MD faculty and 18% (1,844/9,993) of the R01 MD applicants.
Applications and R01 Awards by Degree, Institution Type, and Race/Ethnicity
We first examined the relationships among investigator degree, institution type, and application R01 award probability. Table 1 shows the distribution of R01 Type 1 applications by investigator degree and institution type for the full sample. The majority of applications were from PhD applicants (69%), but their award probability (27.6%) was significantly lower (P < .001) than that of the full sample (28.2%). Institution type had a significant impact on award probability: Two-thirds (66.8%) of applications were received from investigators at medical schools, and their award probability (29.1%) was significantly higher (P < .001) than that of applications from the full sample (28.2%) and from investigators at other institutions (26.4%). Funding probability for applications from PhDs was also correlated with institution type, with higher probability for applications from medical schools than from other institutions (28.9% versus 25.2%; P < .001). Regardless of institution type, applications from MDs were more likely than those from PhDs to be awarded funding.
As in our previous study,9 award probabilities differed by applicant race/ethnicity and were affected by applicant degree and institution type. Figure 1 compares R01 award probability by race/ethnicity at medical schools and other institutions when all degree types (PhDs, MDs, and MD/PhDs) are combined. Overall, applications from blacks and Asians were less likely than those from whites to be awarded R01 funding (P < .001). At other institutions, applications from blacks were less than half as likely as those from whites to receive R01 funding (12.1% [52/429] versus 27.8% [6,255/22,483], P < .001). The difference narrows considerably for applications from medical schools but is still significant for blacks compared with whites (18.7% [182/975] versus 30.1% [13,638/45,245], P < .001).
Figure 2 shows racial/ethnic differences in R01 award probabilities by institution type for applications from MD investigators. Applicant degree appears to interact with institution type and to modify funding probability. There was no significant difference in R01 funding probability between applications from Asian MDs and white MDs at medical schools, but black MDs at medical schools were 7.8% less likely than whites to be awarded funding. (This difference, although significant, is about half the 13% difference we observed between white and black PhDs’ applications in our previous study.)9 In contrast, applications from black MDs at other institutions were less than half as likely as those from whites to be awarded funding (13.2% [17/129] versus 32.2% [1,747/5,418], P < .001).
The Association Between Covariates and Award Probability
In the probit analysis, we estimated the relationship between race/ethnicity and R01 award probability after controlling for demographic characteristics, education and training, employer characteristics, experience with NIH funding, and research productivity for all MDs and subsamples of that group. Table 2 shows the marginal effects from probit models for all MDs, MDs at medical schools, MDs at other institutions, MDs at medical schools whose applications included human subjects research, and MDs at medical schools whose proposals did not include human subjects research. Because the model includes 62 control variables, we report only selected coefficients that had a large and significant association with R01 award probability. The marginal effects can be interpreted as the percentage-point difference in receiving funding compared with the omitted category (for race, this is whites). Gender is not included in the table because it had no significant effect on R01 award probability. Applications from Hispanic MDs did not have a significantly different R01 award probability compared with those from whites, regardless of institution type. (This was similar to our finding about Hispanic PhDs in our previous study.9) Across all institutions, applications from Asian MDs were 2.6% less likely than those from whites to be awarded funding (P < .01), but there was no difference in funding probability for applications submitted by white and Asian MDs employed at medical schools.
As in our previous study, applications from blacks were significantly less likely than those from whites to be awarded R01 funding. However, the size of the difference in this study depended on the sample examined (Figure 3). When we examined all applications, we found that blacks’ applications were 9.3% less likely than whites’ to receive an R01 award (P < .001). However, this difference was significantly reduced when we limited the sample to applications from MDs at medical schools, where more than two-thirds of all MD applicants were employed.
The estimated effects of employer NIH funding rank, prior NIH grants, and serving on an NIH review committee significantly increased the probability of receiving R01 funding for applications from MDs. These estimated effects were quite similar to what we observed in our previous analysis of PhDs.9 However, the estimated correlations of citations, last-authored publications, and single-authored publications for MDs were consistently twice the size of what we observed for PhDs in the previous study. This suggests that research productivity has a larger impact on the likelihood of receiving R01 funding for applications from MDs compared with PhDs.
We conducted the same robustness tests we conducted for the PhD study9 on this study’s MD sample. First, we examined the relationships among race, priority score, and R01 award probability and found, as we did before for PhDs, that awards of R01 funding were largely determined by applications’ priority scores. Second, we examined whether black applicants were less likely than whites to resubmit unfunded proposals. In contrast to the PhD applicants in our previous study, we found that black MD applicants were more likely than white MD applicants to resubmit an unfunded proposal at least once. However, black MD applicants, like black PhDs in our earlier study, had to submit an application more times to receive funding (1.93 times for blacks versus 1.74 times for whites, P < .001). The difference was greater for MD applicants at other institutions (3.86 times for blacks versus 1.58 times for whites, P < .001). As Table 3 shows, almost half of the proposals from black MDs did not receive a priority score compared with approximately one-third of white MDs’ proposals (P < .001). Unscored applications explained only 1% of the award probability difference for black MDs, however.
Human Subjects Research
As Table 2 shows, in contrast to NIH-related and productivity characteristics, the inclusion of human subjects (a proxy for more clinically relevant research) significantly decreased the likelihood of receiving funding by 4.1% in the full sample of MDs (P < .001) and by 4.5% for MDs at medical schools (P < .001). Approximately half the applications from MDs at medical schools (49.6%; 10,876/21,945) and from MDs at other institutions (52.7%; 4,646/8,816) included human subjects. It should be noted that the vast majority of MD applicants at other institutions (97.9%; 3,907/3,989) worked in research hospitals where clinical research is more prevalent. Two sets of results related to the inclusion of human subjects stand out. First, applications from black MDs proposing human subjects research were 6.9% less likely to be awarded funding (P < .05) than those from white MDs. Second, among applications from MD investigators at medical schools, those from blacks that did not use human subjects were 5.4% less likely than those from whites to receive funding, though this result was not statistically significant (P < .17).
We investigated whether these findings were the result of the low number of applications from black MDs at medical schools with human subjects research (n = 200) and without human subjects research (n = 145). We estimated a parsimonious model that controlled only for gender, race/ethnicity, and age for these two samples. Among these black MDs, the estimated marginal effects for applications that did and did not include human subjects were practically identical: −8.3% with and −8.2% without human subjects, respectively (both P < .05). When we included all covariates for the full model, the marginal effect for applications from black MDs using human subjects remained statistically significant but decreased in magnitude, whereas the coefficient for proposals from black MDs not using human subjects decreased more and was no longer statistically significant. Taken together, these results suggest that part of the R01 award probability differential for MDs at medical schools can be explained by research proposals that include human subjects.
Accounting for the Medical School Advantage
Our results showed that applications from MDs at medical schools, regardless of the investigator’s race/ethnicity, were more likely to be awarded R01 funding than those from PhD applicants at medical schools and other institutions. Tobetter understand which characteristics may explain this, we examined racial/ethnic differences in key observable characteristics by institution type. Table 3 shows the distribution of observable characteristics that had a significant impact on R01 award probability for applications from MDs at medical schools and MDs at other institutions. Compared with applications from white MDs, applications from black MDs were more likely to be unscored (P < .001), had fewer citations (P < .001), and were less likely to be from investigators at the top 30 NIH-funded medical schools (P < .001). There were no significant differences between applications from black and white MDs in terms of prior grants, NIH review committee experience, inclusion of human subjects, median cites, maximum impact factor, and last-authored publications.
The results in Tables 2 and Table 3 provide some evidence of positive selection of applications from medical schools. Over half of medical school applications from investigators with MDs (56.8%; 12,465/21,945) came from the top 30 NIH-funded programs, and 93.0% (20,409/21,945) came from the top 100 NIH-funded institutions. Applications from top 30 institutions were 9.7% more likely and applications from institutions ranked 31–100 were 7.4% more likely to be awarded funding than applications from institutions ranked 200 and above in NIH funding. Applications from black MDs at medical schools were significantly less likely than those from white MDs to be associated with the top 30 NIH-funded institutions (P < .001); compared with applications from black MDs at other institutions, however, they were more likely to be in the top tier of employer funding (NIH funding rank 1–30 and 31–100, P < .001), were less likely to include human subjects research (P < .01), and had more published papers (P < .05). These same general relationships held for applications from white MDs at medical schools compared with those from white MDs at other institutions (P< .001), except that applications from other institutions had more published papers and more citations than those from medical schools (P < .001).
We began this study expecting that the racial and ethnic patterns of NIH R01 Type 1 award probabilities for MDs would be very similar to the ones we previously found for PhDs in our earlier research,9 but several new patterns emerged. In this study, applications from MDs had a higher probability of receiving R01 funding than did those from PhDs. In addition, applications from PhD investigators at medical schools were more likely than those from PhD applicants at other institutions to be awarded R01 funding. The differences in award probability by race/ethnicity were also influenced by institution type: Although applications from black and Asian MD and PhD investigators at medical schools and other institutions were less likely than those from white researchers to be awarded R01 funding, the differential was much narrower for applications from investigators at medical schools. When the sample was limited to applications from MDs, the differential for Asian applicants disappeared. The differential for applications from black MDs covaried with institution type, with those from applicants at medical schools having the smallest (but still significant) funding differential. We found evidence that some of the funding differential for applications from black MDs at medical schools could be explained by proposals that included human subjects research. Clinical research (in this study, defined as a grant application including human subjects) was less likely to be funded.
What accounts for the funding advantage of MDs at medical schools? Three factors may play a role. First, MDs at medical schools may be positively selected. A recent study showed that MD and MD/PhD faculty earn higher salaries than their PhD colleagues.18 Economists assume that higher salaries are associated with higher productivity. These higher salaries may also indicate that market salaries for MDs are greater than those for PhDs. Our review of key observable characteristics indicated that black MDs at medical schools were more productive, in terms of publication count, than were black MDs at other institutions, which is evidence for the positive selection of black MDs at medical schools. Second, the majority (93.0%) of applications from MDs at medical schools were from investigators at the top 100 NIH-funded institutions. This may improve funding outcomes to the extent that the employer’s NIH funding creates a positive environment for researchers and that the institutional environment is a component of the NIH proposal review criteria. Third, the majority of MDs at other institutions were employed by hospitals. The demands of clinical work and the length of clinical studies may reduce the number of publications that these researchers produce. Most likely, a combination of these explanations contributes to the funding advantage of MDs at medical schools.
Bias in the application review process may also play a role, as it did in our previous study on PhDs.9 Merton19 described the “halo effect” through which individuals from prestigious institutions are given disproportional credit for this affiliation. As noted above, the overwhelming majority of MD applicants at medical schools in this study were at top-funded institutions, and these affiliations may help to narrow the funding gap for black researchers. The NSF recently anonymized a subset of research proposals and found that these proposals were evaluated very differently from those that retained the investigators’ names and affiliations.20 Some researchers have suggested that the anonymized proposals allowed reviewers to focus more on the research question and the quality of the science than on the track record of the researcher. In addition, implicit bias and racial stereotypes may affect review outcomes.21,22 However, NIH reviewers do not directly observe the race of funding applicants, and direct evidence of implicit bias in peer review has not been documented.
In sum, we found there to be a funding gap between black and white investigators with regard to R01 Type 1 applications, but the gap was significantly reduced for applications from MD investigators at medical schools. The apparent mitigation of our earlier results9 raises questions about the contribution of selection and environmental factors within medical schools. Exploring the causal factors that account for the smaller difference for black MDs at medical schools may provide insights into the development of interventions to reduce the overall funding differential for blacks. We believe this study’s findings can inform future research being framed to improve the diversity of the biomedical workforce.8
Acknowledgments: The authors thank Emily Kennedy for editorial assistance and Leo DiJoseph for analysis assistance.
Funding/Support: Dr. Ginther acknowledges financial support from NIH grant 1R01AG36820-01. Dr. Schaffer is an employee of the NIH, and, as a result, the NIH supported this work. Dr. Haak of Discovery Logic, a Thomson Reuters company, acknowledges support from NIH contracts HHSN276200800458U and HHSN276200900100U, made possible through anNIH Evaluation Set-Aside Award to W.T.S. (07-6008 OD OER).
Other disclosures: Data access: The analytical files used in the studies described here contain personal information from individuals who have submitted applications and in some cases have received awards from the NIH. Many of these application records have been matched to records included in the Survey of Earned Doctorates as maintained by the National Science Foundation and to records included in the Faculty Roster maintained by the Association of American Medical Colleges. The information is therefore protected by the Privacy Act of 1974 as amended (5U.S.C. 552a) and the National Science Foundation Act of 1950 as amended (42U.S.C. 1873(i)). More complete information can be found in the NSF/SRS Restricted-Use Data Procedures Guide available at www.nsf.gov/statistics/license/forms/pdf/srs_license_guide_august_2008.pdf and the NSF Data and Tools Web site at www.nsf.gov/statistics/database.cfm. The NIH data access policy is described at http://report.nih.gov/pdf/DataAccessPolicy.pdf. A deidentified version of the data files is available upon request from Dr. Schaffer.
Ethical approval: Not applicable. This study was performed under contract with the NIH by a security-authorized contractor. No institutional review board review was required. All primary data were derived from the NIH IMPAC II grants system. Matching to external data sets was performed in a secure environment by authorized staff. All data were deidentified prior to statistical analysis.