In a series of studies, we have examined the effects of race/ethnicity on the scientific workforce in general1 and on the biomedical workforce in particular,2 looking separately at PhDs and MDs who apply for R01 research awards from the National Institutes of Health (NIH).3,4 In this study, we examined the effect of gender and gender combined with race/ethnicity on the probability of receiving an R01 Type 1 research award. Specifically, we examined whether women were less likely to receive an R01 award and whether women of color faced an added barrier, a “double bind,” where gender and race/ethnicity together further decreased the likelihood of receiving NIH funding. Given the importance of diversity to the NIH’s objective of improving the nation’s health through both basic and translational biomedical research,5 a complete understanding of the factors related to receiving NIH funding is essential.
The justification for increased gender and race/ethnicity diversity in academic medicine has been well supported in the literature. Female students perform better when they are taught by female teachers in the male-dominated science disciplines, and students of color perform better when they are taught by instructors of their same race.6–8 This research suggests that having women and particularly women of color among faculty ranks in academic medicine may increase the number of female physicians of color. However, women are less likely to be faculty in academic medicine. According to data from the Association of American Medical Colleges (AAMC), only 38.1% (55,352/145,172) of medical school faculty of any rank are women. A mere 2.6% (3,824/145,172) of faculty are black or African American, split almost equally among women (2,002) and men (1,822). Hispanics make up 3.7% of medical school faculty (5,413/145,172); however, Hispanic men (3,337) outnumber women (2,076).9 In departments where NIH funding is required for retention and promotion, significant gender and race/ethnicity differences in NIH funding might lead to less diversity among faculty. Thus, research to understand how gender and race/ethnicity contribute to differences in the probability of receiving NIH funding is needed.
Findings on the effects of gender on NIH funding are mixed. An early study found that women received significantly fewer grant dollars from NIH than men; however, this research did not account for the number of applications submitted or the amount requested.10 In another study, researchers examined gender differences in many NIH research grant types (loan repayment awards, K awards, and R mechanisms) using data from 2003 to 2007.11 They found that women more frequently dropped out of the NIH applicant pool between the early-career K mechanism and the independent researcher R01 mechanism. They also found that experienced women investigators had statistically significant and slightly lower success rates with the R01 mechanism than men. An NIH-commissioned study found that women had success rates similar to men across a number of NIH grant mechanisms.12 However, the study also found that women were less likely to have their R01 Type 2 applications funded. Overall, men submitted more applications and received more grants than women. Finally, a recent study examined whether the linguistic content of NIH proposal reviews differed by the gender of the investigator.13 The authors reported that proposals from women received more words of praise than those from men, although men received awards at the same rate despite their proposals receiving more negative words.
In previous work, we examined race/ethnicity differences in NIH funding and found that black and Asian PhD applicants were significantly less likely to receive R01 funding, after we controlled for many observable characteristics.3 In a second study, we examined race/ethnicity differences in NIH funding for MDs working at medical schools. We found that the black–white funding gap for MDs was narrower than that for PhDs and that this gap could be partially explained by the inclusion of human subjects in the proposal.4 In this study, we examined the situation for women and women of color. We hypothesized that, if all NIH applicants were equally qualified, neither race/ethnicity nor gender would have a significant impact on the probability of receiving NIH funding.
To receive NIH funding, principal investigators submit grant applications that are assigned to a study section consisting of subject matter experts who consider the merits of the proposal. Approximately half of proposals are discussed and assigned a priority score. NIH funding is awarded to the most meritorious proposals according to a ranking of the scores, the availability of funding, and the proposal’s alignment with identified research priorities, which vary from year to year and across NIH institutes. Researchers often have to revise and resubmit their proposals to receive funding. In this study, we focused on traditional research grant applications (R01 Type 1) and examined the effect of gender and gender combined with race/ethnicity on differences in R01 award probabilities.
As with our previous studies, we used administrative data from the NIH Information for Management, Planning, Analysis, and Coordination (IMPAC II) grants management database in combination with several other sources made available to Discovery Logic/Thomson Reuters under contract with specific and stringent security regulations. (Aggregate information from the IMPAC II database is publicly available through the NIH Research Portfolio Online Reporting Tools at https://projectreporter.nih.gov/reporter.cfm.) We combined IMPAC II data with (1) information on employer institutions from the Department of Education’s Integrated Postsecondary Education Data System (available at http://nces.ed.gov/ipeds/), (2) additional measures of race/ethnicity for MDs from the AAMC’s Faculty Roster (available at https://www.aamc.org/data/facultyroster/), and (3) information on the educational background and demographic characteristics of PhDs from the National Science Foundation’s Survey of Earned Doctorates (available at http://www.nsf.gov/statistics/srvydoctorates/). In addition, we matched publication and citation information from Thomson Reuters Web of Science and Journal Citation Reports to individual applicants. Through this process, we were able to match 84% of grant applications to publications with greater than 90% confidence. Additional information on this matching process is available in the supporting online material for Ginther et al.3 Data were stored in a Microsoft SQL Server 2005 database at Discovery Logic/Thomson Reuters; we performed all statistical analysis on a deidentified dataset made available by the company.
From the IMPAC II database, we collected application- and applicant-level data for R01 research project proposals submitted between fiscal year (FY) 2000 and FY2006. During the study time frame, applicants could submit more than one revision of an R01 application. We collapsed revised submissions received within two years of the original submission into a single application for the purposes of determining the award probability of the application. For our analysis, we used information from the last funded or unfunded application submitted. Of the applications we analyzed, 36.9% (36,157/97,877) were resubmissions.
In 2009, the NIH introduced a revised proposal scoring system, so we did not include proposals submitted after 2006 in our analysis because we could not observe them for a two-year period under the same system. Our sample was limited to R01 Type 1 grant applications from investigators with a PhD, MD, or MD/PhD at U.S. institutions (N = 106,368). Applications from outside the United States, with missing information on both race/ethnicity and age, or without a principal investigator with a PhD or MD, were dropped from the sample. We were left with 97,877 applications from 47,424 applicants with data available for most of the explanatory variables. We also calculated the number of total R01 applications and awards from 1980 to 2006, which we used when the applicant was the unit of analysis.
Race/ethnicity was based on self-reported information from IMPAC II, the Survey of Earned Doctorates, and the AAMC Faculty Roster. We used the most frequently reported race/ethnicity, gender, and birth date when values changed in IMPAC II. To protect confidentiality and to identify race/ethnicity outcomes, our regression models included separate controls for the Native American, other race, and race unknown (categories with small numbers) categories, but we do not report those results here. We instead focus on race/ethnicity outcomes for Asians, blacks, Hispanics, and whites.
In our application success analysis, we used the same modeling approach as our previous studies3,4—we examined race/ethnicity differences in R01 awards by controlling for all characteristics in the NIH biographical sketch that may be associated with the probability of receiving an NIH award. We started with demographic characteristics (race/ethnicity, gender, age, age-squared) in Model 1; added education and training variables (degree type and type of NIH training grants [F, T, or K]) in Model 2; included employer characteristics (employer NIH funding rank, employer is medical school, region of employer) in Model 3; incorporated institutional and individual NIH experience (NIH institute codes, prior NIH grants, NIH review committee member, application uses human subjects) in Model 4; and finally, controlled for research productivity (publication quartiles, citation quartiles, maximum journal impact factor, median journal impact factor, ratio of first-author/total publications, ratio of last-author/total publications, ratio of single-author/total publications) in Model 5. Because the overall NIH budget and individual institute budgets varied during our study period, we included dummy variables for NIH institute and year.14 All of the included variables have been associated with the probability of receiving an NIH award, so we compared the results for women with our previous results that focused exclusively on race/ethnicity.
We determined the NIH funding rank of the applicant’s employer by averaging the annual grant support received by the organization from FY2000 to FY2006 and ranking employers by total grant dollars in descending order. We then divided this list into categories: 1–30, 31–100, 101–200, 200+, and unranked. We divided the continuous variables (number of publications and number of citations) into quartiles and then assigned dummy variables to the quartiles to improve the fit of the models. We also included a dummy variable equaling 1 when information for a specific variable (e.g., number of publications) was missing. See Supplemental Digital Appendix 1 (available at https://links.lww.com/ACADMED/A371) for additional details on all variables.
On the basis of our previous study that found that MDs have better funding outcomes than PhDs,4 we divided the sample into MDs (with and without PhDs) and PhDs only. Supplemental Digital Appendix 2 (available at https://links.lww.com/ACADMED/A371) reveals that small degree-specific populations of color and small female populations in some race groups exist, so to preserve the confidentiality in our estimation, we did not separate MDs from MD/PhDs or separate men from women for each race/ethnicity. Supplemental Digital Appendix 3 (available at https://links.lww.com/ACADMED/A371) includes counts of the categorical variables and means and standard deviations of the continuous variables by degree and award status.
First, we conducted graphical comparisons and tests of gender differences in the mean probabilities of an application receiving an NIH award (2000–2006), within race/ethnicity and degree categories. Using the same tests, we then determined the probability that an applicant (2000–2006) had received at least one NIH award between 1980 and 2006.
Next, we used probit models to estimate the probability of receiving an R01 award for a given application, conditional on race/ethnicity and gender, using the models described above. We report the marginal effects in place of probit coefficients because they are easier to interpret; the marginal effect is the change in the award probability due to each predictor separately, with other variables evaluated at their mean values. In the case of gender, the marginal effect can be interpreted as the percentage point (ppt) difference in the probability of a woman receiving a grant compared with a man (the omitted category in the regression). The marginal effect of the race/ethnicity variables can be interpreted as the ppt difference between that race/ethnicity receiving a grant compared with whites (the omitted category). We used heteroscedasticity-robust standard errors that were clustered on the individual applicant to adjust for the fact that applicants could submit more than one proposal in the study sample. These estimates are associations between the covariate and the R01 award outcome and should not be interpreted as having a causal impact. We used STATA 14 software (StataCorp LP, College Station, Texas) to conduct all analyses.
This study was performed under contract with NIH by a security-authorized contractor, and we received an institutional review board waiver from the University of Kansas indicating that the analysis did not involve human subjects.
Applications and awards by gender and race/ethnicity
Table 1 shows the distribution of R01 Type 1 applications (2000–2006) by race/ethnicity, gender, and degree (n = 87,146; four races). The majority of applications came from PhDs, men, and whites, with only 5.1% (4,417/87,146) coming from women of color.
Table 1 and Figure 1 Panel A show the gender and race/ethnicity differences in the probability of an application receiving an R01 award. Applications from Asian women PhDs were significantly more likely to be funded than those from Asian men. For all other degree and race/ethnicity categories, we found no significant gender differences. In terms of race, applications from Asians and blacks were significantly less likely than those from whites to receive NIH funding, regardless of degree or gender.
Table 1 and Figure 1 Panel B show the probabilities of an applicant (2000–2006) receiving at least one R01 award between 1980 and 2006. As before, Asians and blacks were significantly less likely than whites to have received funding regardless of degree. Hispanics also were significantly less likely than whites to have received an award. In addition, some gender differences emerged within race/degree groups. Hispanic women PhDs were significantly less likely than Hispanic men PhDs to be funded; white women PhDs and MDs were significantly less likely than white men PhDs and MDs, respectively, to be funded. The small gender differences for Asians and blacks were insignificant.
Next, we compared applications from new investigators—those who had not previously received R01-equivalent funding—and those from experienced investigators. Within degree categories, women were more likely than men, and blacks, Asians, and Hispanics were more likely than whites, to be new investigators. The percentages of new investigators among Asians and Hispanics fell between those of whites and blacks. PhDs were significantly more likely than MDs to be new investigators.
Table 1 and Figure 2 show that within all degree/gender/race/ethnicity categories, applications from new investigators were somewhat less likely to be funded than those from experienced investigators.
Regardless of degree, applications from white women and white men new and experienced investigators were equally likely to be funded. Applications from Asian women PhDs were more likely to be funded than those from Asian men PhDs for new investigators. Regardless of degree and gender, award probabilities for blacks were lower than those for whites in both experience groups; however, with the exception of female experienced MDs, the difference was much greater among new investigators.
Probit estimates of award probability for applications
We used probit analysis to estimate how gender and race/ethnicity related to an application’s award probability, controlling for demographic characteristics, year, and NIH institute (Model 1), and adding education and training (Model 2), employer characteristics (Model 3), NIH experience (Model 4), and research productivity (Model 5) controls. Table 2 shows the marginal effects of gender and race/ethnicity from all five probit models for four samples: all PhDs, women PhDs, all MDs, and women MDs. The marginal effects (multiplied by 100) for women can be interpreted as the ppt difference in the probability of women receiving funding compared with men; for each race/ethnicity variable, the data presented are the ppt differences compared with whites.
The PhD full sample results show the same racial patterns as our previous studies.3,4 Concentrating on Model 5, which controlled for all covariates including NIH experience and research productivity, applications from blacks and to a lesser extent Asians were significantly less likely to be funded than applications from whites of the same sex. Hispanics were significantly less likely to receive funding in Models 1–3, but after adding the NIH experience and research productivity controls (Models 4 and 5), this was not true. The second section of Table 2 includes estimates for women PhDs only and shows racial differences similar to those in the full sample. The third and fourth sections of Table 2 show these analyses repeated for MDs. These findings were similar to those for PhDs, although the magnitudes of the racial differences were somewhat smaller in the full MD sample and larger in the women MD sample, most notably among blacks—applications from black women MDs were 12.9 ppt less likely to receive NIH funding than applications from white women MDs. Finally, overall racial patterns in applications by men PhDs and MDs were similar to those for the full sample of PhDs and MDs, respectively (see Supplemental Digital Appendix 4 at https://links.lww.com/ACADMED/A371).
These significant racial differences were in stark contrast to the gender differences. Models 1–3 of the full PhD sample (top section of Table 2) resulted in no significantly different award probabilities for women and men. When controls were added for NIH experience and research productivity (Models 4 and 5), women PhDs were significantly more likely than men PhDs to be funded (1.3 ppt, P < .01). For MDs, in Models 2 and 3, women MDs were significantly less likely to be funded than men MDs; however, adding controls for NIH experience and research productivity eliminated this difference.
Probit estimates by investigator experience, resubmissions, and applicant
Using Model 5, we estimated gender and race/ethnicity differences in R01 awards by experience group (new and experienced investigators), holding covariates constant (see Table 3). Applications from new-investigator women PhDs were 1.9 ppt more likely than those from comparable men to receive NIH awards, controlling for race/ethnicity and other covariates. Likewise, applications from new-investigator women MDs were 2.0 ppt more likely to receive NIH awards. We found no gender differences among experienced-investigator PhDs or MDs.
We did find racial differences for new and experienced-investigator PhDs. Whereas applications from Asian and black PhDs were significantly less likely than those from white PhDs in both experience groups to receive funding, the racial differences for Asians were more pronounced in the experienced-investigator group. In contrast, among Asian MDs, the new investigator funding gap was 3.6 ppt, but the experienced-investigator gap was not statistically significant. The funding gap for black PhDs was more than 10 ppt for both new and experienced investigators; for black MDs, the gap was somewhat smaller but roughly equivalent across experience groups.
In previous studies, we found that blacks were significantly less likely to resubmit unfunded applications.3,4 In this study, we found that women PhDs were significantly more likely than men PhDs to resubmit an unfunded application (see Table 3 Column 3). We also found that black PhDs and to a lesser extent Asian PhDs were significantly less likely to resubmit unfunded applications, as were black and Hispanic MDs. In the new-investigator group (see Table 3 Column 4), women PhDs were significantly more likely than men PhDs to resubmit unfunded applications (P < .10); we found no significant gender difference for MDs. However, we did find that new-investigator black PhDs and MDs were significantly less likely than their white counterparts to resubmit unfunded applications.
In Table 3 Column 5, we show whether investigators who applied for R01 funding between 2000 and 2006 had ever received an R01 award (1980–2006). We conducted these analyses to assess whether an investigator who did not receive an R01 award for an application submitted between 2000 and 2006 had received an award at another time in her or his career. We did not address whether gender or race/ethnicity differences in R01 awards existed during the entire 1980–2006 period, because we did not identify those investigators who applied only during the earlier decades (1980–1999).
Women PhDs were 1.9 ppt less likely than men PhDs to have received at least one R01 award. Asian and Hispanic PhDs were 4 to 5 ppt less likely than white PhDs to have received at least one R01 award. As before, the difference for black PhDs was much greater—black PhDs were 21.7 ppt less likely than white PhDs to have received at least one R01 award. Among MDs, women and men were equally likely to have received at least one award. However, racial gaps were larger for Asian (−8.4 ppt) and Hispanic (−6.8 ppt) MDs than for comparable PhDs, whereas the gap for black MDs was smaller than for comparable PhDs (−14.9 ppt). In Table 3 Column 6, we show our results controlling for the total number of applications submitted. The race/ethnicity coefficients for PhDs and MDs did not change appreciably. However, the gender difference for PhDs was no longer statistically significant, indicating that women were less likely than men to have received at least one R01 award because they applied for fewer awards.
We directly modeled the total number of R01 applications per investigator from 2000 to 2006 to determine whether gender differences in application rates could be explained by observable characteristics (see Table 3 Column 7). Controlling for all factors, women PhDs and MDs submitted 0.14 and 0.09 fewer applications, respectively, than comparable men, whereas Asian PhDs and MDs submitted 0.26 and 0.22 more applications, respectively, than comparable whites, relative to the overall average of 2.1 applications per investigator.
We also examined gender and race/ethnicity differences in the probability that an investigator under the age of 50 (a “young” new investigator) had submitted her or his first R01 application between 2000 and 2006, had not received funding for that application, and had neither resubmitted it nor submitted additional R01 proposals. Although investigators could have submitted an R01 application after our study period, we used this analysis as the closest available proxy for attrition from the NIH applicant pool.
In Table 1 and Supplemental Digital Appendix 5 (available at https://links.lww.com/ACADMED/A371), we compare the gender and race/ethnicity probabilities of investigators under the age of 50 submitting only a single, unfunded proposal. Within races and degrees, young women were more likely to have submitted a single, unfunded proposal than comparable men, often significantly so. Controlling for other factors, Table 3 Column 8 shows the marginal effects of a probit model that regresses the probability of young, new-investigator single submissions on gender and race/ethnicity. Young women PhDs were 2.7 ppt more likely than young men PhDs to submit only a single, unfunded proposal. This gender difference paled in contrast to that for black PhDs, who were 16.7 ppt more likely than white PhDs to submit only a single, unfunded proposal. We found no significant comparable differences between women MDs and black MDs; however, we did find that young Hispanic MDs were 4.9 ppt more likely than young white MDs to have submitted only a single, unfunded proposal.
Finally, within degrees, we found no significant gender differences in the probability of a single, unfunded proposal receiving a priority score or in the magnitude of that score. Applications by new-investigator black PhDs were significantly less likely than those by comparable whites to receive a priority score (83% [97/117] vs. 73% [1,855/2,538], P < .05), while scores for applications by black PhDs and MDs were significantly lower than those for comparable whites.
Given our previous findings about the disadvantages that blacks and Asians faced in securing R01 awards, in this study, we considered whether women of color faced additional barriers, termed a “double bind,” compared with men of color. Within degree and race/ethnicity categories, we found that women were not at a significant disadvantage in securing an R01 award (both on average and when controlling for covariates). However, we found differences by race/ethnicity that were large, significant, and comparable to estimates we have documented previously.3,4 Taken together, these results show that differences in funding by race/ethnicity were more prominent than those by gender.
We also examined whether significant gender and race/ethnicity differences in NIH funding existed between investigator experience groups (new vs. experienced). Within the degree category, more women and blacks who applied for funding were new investigators compared with men and whites. New-investigator women were 2 ppt more likely to be funded than comparable men. For experienced investigators, we found no significant gender differences. The level of funding among new-investigator black PhDs was comparable to that of experienced-investigator black PhDs.
Next, we evaluated gender and race/ethnicity differences in the probability of an investigator who applied for funding between 2000 and 2006 having received at least one R01 award between 1980 and 2006. Overall, race/ethnicity differences were similar to those for individual applications submitted between 2000 and 2006. However, significant gender differences emerged (see Table 1 and Table 3 Column 5). Controlling for covariates, women PhDs were significantly less likely than men PhDs to have received at least one R01 award. However, women submitted fewer R01 applications than men (see Table 3 Column 7). Adding a control for the number of R01 applications (see Table 3 Column 6) explained this gender gap. In other words, women were less likely to receive at least one R01 award over the decades studied because they applied for fewer grants.
Finally, we examined gender and race/ethnicity differences in the probability that a young (under the age of 50) new investigator would have submitted only a single unfunded application. This measure is our proxy for determining whether investigators left the NIH applicant pool. Here, we found evidence of a double bind for women of color (see Table 3 Column 8 and Supplemental Digital Appendix 5 available at https://links.lww.com/ACADMED/A371). Young women PhD (but not MD) new investigators were more likely to submit a single, unfunded proposal than comparable males (as were black PhDs and Hispanic MDs). This finding could be due to a lower likelihood of having the first application funded, a lower likelihood of resubmission, or a lower likelihood of submitting additional R01 proposals. Our findings suggest that applications by women new investigators were not less likely to be funded or to be resubmitted; therefore, women must have been substantially less likely to submit additional R01 proposals (see Table 3 Columns 1 and 4). These results are consistent with those of previous studies that found that women were more likely to drop out of the R01 applicant pool.11,12
Our finding that women were not at a significant disadvantage in securing R01 funding differed from the findings of previous studies.11 This difference could be because we limited our analysis to the Type 1 mechanism. In fact, we found that women PhD and MD new investigators were significantly more likely to be funded than men after controlling for covariates including NIH experience and research productivity. What explains this advantage? It could be that women write better proposals. Kaatz et al13 found that, at the University of Wisconsin–Madison, women’s NIH proposals received more words of praise than men’s. This advantage also could reflect a quantity/quality trade-off. Women submitted fewer NIH proposals, so they could devote more time and effort than men to refining each one. Finally, women applicants may be more positively selected than men—women are significantly more likely to leave the NIH applicant pool after one unsuccessful proposal; thus, the women who remain in the applicant pool may be of higher average quality than the men.
Our finding that women submitted fewer proposals is consistent with research showing that women avoid competition.15 Because men of color were also more likely to submit a single, unfunded proposal, stereotype threat may be an issue too.16,17 As we have hypothesized in previous studies, bias and racial stereotypes may affect application review outcomes for investigators of color.3,4 However, reviewers cannot see applicants’ race, and direct evidence of implicit bias in peer review has not been documented. Still, we were pleased to find only limited evidence of a double bind for women of color. That said, policies to encourage women to submit revised and additional grant proposals would likely increase the number of women of color receiving NIH funding. In addition, future research should closely examine the gender differences in R01 Type 2 awards that have been uncovered in previous studies.11,12