Secondary Logo

Journal Logo

Research Reports

Measuring Diversity of the National Institutes of Health–Funded Workforce

Heggeness, Misty L. PhD, MPP, MSW; Evans, Lisa JD; Pohlhaus, Jennifer Reineke PhD; Mills, Sherry L. MD, MPH

Author Information
doi: 10.1097/ACM.0000000000001209

Abstract

A diverse National Institutes of Health (NIH)-supported scientific workforce fosters scientific innovation, enhances global competitiveness, contributes to robust learning environments, improves researcher quality, enhances public trust, and increases the likelihood that underserved or health-disparity populations will participate in, and benefit from, health research.1–7 We highlight the need to make accurate comparisons when analyzing workforce diversity and recommend calculating representation ratios from a relevant labor market perspective. Representation ratios tell us whether a subpopulation is over- or underrepresented compared with its representation in a larger group or at a previous stage along the career path.8,9 In this article, we clarify the relevant labor market definition; calculate representation ratios along the career pathway, thus improving our understanding of focal points within the NIH biomedical research career path; and suggest areas for potential intervention.

Introduction

Any lack of diversity among biomedical research professionals is not merely an issue of demographic equity; it undermines the realization of our national research goals.10,11 Studies have reported that diversity leads to greater stability in the workforce,1 enhances intellectual engagement and motivation,2 improves decision making in groups,2–4 and is correlated with increased earnings and productivity of native-born individuals. Given the importance of diversity within the workforce, it is essential to accurately measure representation and clearly understand the most relevant goal to strive for, which includes focusing on areas where the organization has the most potential to influence representation.

Prior studies have examined differences in the biomedical workforce. Pohlhaus et al12 and Ginther et al13,14 found disparities in the NIH research award funding system by sex, race, and ethnicity, and an earlier study by Ginther et al15 also found that underrepresented racial and ethnic groups fall out of the biomedical educational pathway well before they reach the eligible NIH principal investigator pool. Myers and Husbands-Fealing9 examined representation of subgroups within biomedicine and found that not all groups benefit from broad diversity policies in the same way. These studies provide a larger context for understanding diversity within the biomedical workforce.

Our research emphasizes the importance of using the most accurate comparison group to measure diversity. By correctly comparing NIH-funded awardees with the relevant population of individuals who are eligible for such an award, we improve our ability to precisely measure diverse representation and better identify areas where NIH contributions can help move the dial on accelerating diversity within the workforce.

Relevant labor market

Recent discussions about diversity16–20 assume that the proportion of individuals engaged in biomedical research or receiving NIH funding should be equal to the same proportions of women, racial and ethnic minorities, or persons with disabilities in the total U.S. population. This assumption does not account for other determinants influencing changes in the distribution of the U.S. population. For example, approximately 24% of the U.S. population is under age 1821 and falls out of the scope for understanding diversity in the biomedical workforce. However, even limiting the relevant reference population to a relevant working age is not enough. In 2010, only 3.1% of the U.S. population aged 25 and older would be competitive for NIH funding based on holding an advanced degree, such as a PhD, MD, or equivalent.

The term “relevant labor market” is defined by the U.S. Department of Labor Office of Federal Contract Compliance Programs22 which requires federal contractors to identify potential barriers to equal employment opportunity by conducting utilization and availability analyses. Utilization analyses describe the percentages of a protected group (e.g., race, ethnicity, or sex) by occupation within the current employer’s workforce. Availability analyses describe a similar percentage breakdown by protected class, but for applicants and potential employees who reside within either the geographic area or the employer’s relevant labor market. A discrepancy between these percentages in which the employer’s workforce percentage of protected class members is less than the percentage observed for the relevant labor market is referred to as “underutilization.”

Courts have used the relevant labor market analysis to assess an employer’s workforce for potential discrimination for almost three decades. Such analysis assumes that the proportion of women, racial and ethnic minorities, or persons with disabilities in an employer’s workforce is equal to that of qualified individuals in a well-defined geographic area. In our analysis, we focus on the availability analysis to determine the individuals who would and could potentially seek NIH support at various funding stages.

The NIH-funded workforce

The NIH is a major funder of biomedical and behavioral research in the United States. The NIH announces the availability of funds and specifies any requirements for researchers and institutions in funding opportunity announcements. Fellowship programs provide direct support to applicants who are still in training. Training programs are administered by established senior investigators and allow trainees to receive research and career support. The K program is designed for early-career investigators who transition to independent positions, typically as assistant professors. The R01 program, which is the original and oldest funding mechanism used by the NIH, is part of a group of programs collectively known as research project grants (RPGs), which includes other research funding mechanisms. Several funding mechanisms are considered equivalent to R01 awards; collectively; they are referred to as R01-equivalent awards.

Although most funding opportunities do not require grantees to hold advanced degrees, more than 98% of the NIH-funded workforce during the time period we studied had attained a PhD, MD, or doctoral equivalent. This distribution is similar to the faculty at U.S. medical schools, where 96% have attained a terminal degree of PhD or MD or equivalent (based on the authors’ calculations from the Association of American Medical Colleges Faculty Roster data).23 All NIH funding opportunities encourage individuals from underrepresented racial and ethnic groups, as well as individuals with disabilities, to apply for NIH support, in alignment with the NIH’s Diversity Policy Statement.24

Method

Data

For our study, we used data from two data sets—the NIH Information for Management, Planning, and Coordination II database for fiscal years 2008–2012, and the Integrated Public Use Microdata Series American Community Survey (IPUMS-ACS) 2008–2012 five-year file25—to generate nationally representative data by educational attainment and occupation in the United States.

Methodology

Our primary method for analyzing a relevant labor market for biomedical researchers was to compare researchers who received NIH funding versus individuals with advanced degrees working as biological or medical scientists in the United States. Our methods stemmed from the economics of diversity and have a strong history within the literature.26–29 By using the IPUMS-ACS cross-sectional data, we avoid problems of selection bias that exist with panel data.29

Although our focus is on assessing those individuals engaged with NIH funding, we also analyzed the full spectrum of the educational pipeline from high school diploma to advanced degree (see Supplemental Appendix 1 at http://links.lww.com/ACADMED/A360). In our study, all Hispanic or Latino individuals were considered Hispanic, regardless of race. White non-Hispanics were classified as white, black non-Hispanics were classified as black, and Asian non-Hispanics were classified as Asian.

Representation ratios

We calculated representation ratios according to the method described by Myers and Husbands-Fealing.9 When a subgroup’s representation ratio is greater than 1, they exhibit greater representation in that educational or career stage compared with the previous stage or relevant labor market in terms of NIH-funded paths. When the ratio is less than 1, the subgroup is underrepresented; when the ratio is equal to 1, the representation of the subgroup among recipients of the next stage is equal to the subgroup’s representation in the relevant labor market.

We then calculated the standard error for each representation ratio30 and tested for the significance of the difference between ratios of interest by calculating the z score of the difference between the natural log of the ratios (data not shown).31 Comparisons discussed in the text are statistically different at the level of at least α = 0.05.

We calculate representation ratios at each stage of the educational and workforce pipeline including high school, bachelors, advanced degree, postdoc, K awardees, RPG awardees, and R01-equivalent awardees. For each of these stages, we calculate the ratios by demographic subgroups including sex (male/female), citizenship (citizen/noncitizen), and race/ethnicity (white, black, Asian, American Indian, and Hispanic). Representation of various subgroups in workforce diversity is complex. Given this, we calculate representation ratios for women and men by race/ethnicity and by race/ethnicity and citizenship. Additionally, we calculate ratios for citizen and noncitizen by race/ethnicity. By exploring the representation ratios of various combinations of diverse subgroups, we are able to more accurately identify trends in over- and underrepresentation throughout the pipeline while acknowledging that diversity is complex and involves identities that combine race with sex with citizenship.

Defining the relevant labor market or population

The relevant labor market is the “risk population” from which the person earning the degree or applying for the job in question comes (Table 1). For example, bachelor’s degree recipients are the relevant pool for potential advanced-degree earners (e.g., PhD or MD) because advanced-degree programs require bachelor’s degrees for entry. Thus, the number of individuals in a demographic subgroup with an advanced degree is divided by the number of individuals within said subgroup with a bachelor’s degree; this ratio is divided by the ratio of the total group of advanced-degree holders over the total group of bachelor’s degree holders to determine whether the subgroup earns advanced degrees at the same rate as others in the total group of people who have a bachelor’s degree.

Table 1
Table 1:
Definition of Educational and Career Stages and Relevant Labor Market or Population

This study was exempt from IRB approval. We used already-existing public use data and NIH administrative data to examine representation within the workforce as a means to improve our understanding of areas where NIH policies could more directly influence or diversify representation. Stata 13 statistical software (StataCorp LP, College Station, Texas) was used to conduct this analysis.

Results

Representation ratios for men and women by race and ethnicity

In general, within race and ethnicity men have higher representation than women along the pipeline through to advanced degrees. However, within the NIH-funded groups, this pattern is generally reversed in training programs (Table 2, Figure 1). We found that white, black, and Hispanic women were overrepresented in postdoctoral positions (white: 1.44; black: 1.61; Hispanic: 1.24) and in mentored K awards (white: 1.14; black: 1.43; Hispanic: 1.54). Asian women, while underrepresented in the postdoc and mentored-K-award pools, actually had higher representation ratios for NIH-funded postdoc positions than their male counterparts (Figure 1).

Table 2
Table 2:
Representation Ratios by Sex, Citizenship, and Race/Ethnicity for Specified Education and Career Levels of the U.S. Population and NIH-Funded Workforce, 2008–2012a
Figure 1
Figure 1:
Representation ratios of NIH-funded workforce versus the relevant labor market, by race and ethnicity, and by sex, 2008–2012. The representation ratios for males and females within four racial and ethnic groups are shown. The dashed line at 1.0 indicates the transition point between under- and overrepresentation. Source: Authors’ calculations using Integrated Public Use Microdata Series American Community Survey (www.ipums.org) and NIH Information for Management, Planning, and Coordination II data. Abbreviations: NIH indicates National Institutes of Health.aIncludes all NIH trainee and fellowship award mechanisms.bIncludes the following NIH mentored award mechanisms: K01, K07, K08, K22, K23, K25, K99, R00.cIncludes the following NIH independent research award mechanisms: R01, R23, R29, R37, DP2, R03, R15, R21, R22, R33, R34, R35, R36, R55, R56, RC1, P01, P42, PN1, U01, U19, UC1.dIncludes the following NIH independent research award mechanisms: R01, R23, R29, R37.

Across the various race and ethnic groups of RPG and R01-equivalent awardees, men had larger representation ratios than women. Particularly, white males had higher representation ratios than males of any other race and compared with all women. All groups except white males were underrepresented in RPG and R01-equivalent awardee pools compared with the relevant population (Figure 1).

Citizen and noncitizen representation ratios by race and ethnicity

Noncitizens had higher representation ratios by race and ethnicity than their citizen counterparts through to advanced degrees (Table 2). All noncitizen racial and ethnic groups were underrepresented in the NIH-funded pool (Table 2). For postdocs, this finding is consistent with the policies governing many NIH-funded training positions that stipulate a requirement for U.S. citizenship, such as the Ruth L. Kirschstein National Research Service Award T32 programs.

Representation ratios for men and women by citizenship, and race and ethnicity

We also calculated representation ratios by combining sex, race and ethnicity, and citizenship status (Table 2, Figure 2). While white male citizens were underrepresented in advanced degrees (0.71), they were overrepresented in all NIH-funded awards examined (postdoc: 1.18; RPG awardee: 1.47; R01-equivalent: 1.55), with the exception of mentored K awards (0.99). The reverse was true for white male noncitizens, which shows an overrepresentation in advanced degrees (6.09) and an underrepresentation for NIH-funded positions (postdoc: 0.26; K awardee: 0.66; RPG awardee: 0.76; R01-equivalent: 0.78). White female citizens were overrepresented in postdoctoral awards (1.63) and mentored K awards (1.20).

Figure 2
Figure 2:
Representation ratios of the NIH-funded workforce versus the relevant labor market, by race and ethnicity, sex, and citizenship, 2008–2012. The representation ratios for male and female citizens and noncitizens within four racial and ethnic groups are shown. The dashed line at 1.0 indicates the transition point between under- and overrepresentation. The ^ indicates data not included because of insufficient sample size. Source: Authors’ calculations using Integrated Public Use Microdata Series American Community Survey (www.ipums.org) and NIH Information for Management, Planning, and Coordination II data. Abbreviations: NIH indicates National Institutes of Health.aIncludes all NIH trainee and fellowship award mechanisms.bIncludes the following NIH mentored award mechanisms: K01, K07, K08, K22, K23, K25, K99, R00.cIncludes the following NIH independent research award mechanisms: R01, R23, R29, R37, DP2, R03, R15, R21, R22, R33, R34, R35, R36, R55, R56, RC1, P01, P42, PN1, U01, U19, UC1.dIncludes the following NIH independent research award mechanisms: R01, R23, R29, R37.

Other groups had mixed results throughout the pipeline. Black citizens, both male and female, were overrepresented in postdoctoral awards (black males: 1.24; black females: 1.65) and mentored K awards (black males: 1.37; black females: 1.49) but were underrepresented in the NIH RPG (black males: 0.54; black females: 0.49) and R01-equivalent pools (black males: 0.34; black females: 0.29). Hispanic male citizens were overrepresented in all NIH funding mechanisms (postdoc: 1.33; K awardee: 1.37; RPG awardee: 1.15; R01-equivalent: 1.18). Hispanic female citizens were overrepresented in postdoctoral positions (1.83) and mentored K awards (2.16) but underrepresented in RPGs (0.81) and R01-equivalent funding (0.70). Male Asian citizens were overrepresented in mentored K awards (1.04), RPGs (1.10), and R01-equivalent grants (1.16). Female Asian citizens were overrepresented in NIH-funded postdoctoral awards but were underrepresented in RPGs (0.54) and R01-equivalent grants (0.54) (Figure 2). Asian citizens had higher representation ratios than their noncitizen counterparts. Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A360 provides the standard error and N value for each representation ratio shown in Table 2.

Representation ratios using total population instead of relevant labor market

Does it matter whether we use a relevant labor market or total population comparison? To examine this, we compared the resulting representation using these two different reference populations. We found a robust difference in representation ratios for almost all of our demographic groups (Figure 3 and data not shown) across the educational and career levels studied. For example, in the postdoc group, Asians had a representation ratio of 0.49 using the relevant labor market, but a representation ratio of 3.35 using the total population instead. In this case, using the total population as the reference comparison group would lead to the conclusion that Asians are dramatically overrepresented when in fact they are underrepresented compared with the appropriate relevant labor market (Figure 3). The choice of the appropriate comparison is crucial to the accuracy of the representation ratios and can strongly alter the conclusions of a study.

Figure 3
Figure 3:
Representation ratios by race and ethnicity of NIH-funded workforce versus the relevant labor market, 2008–2012: comparing the relevant labor market vs. total population (age 25+). The representation ratios for NIH-funded postdoctoral positions (trainees and fellows) within four racial and ethnic groups as calculated against the relevant labor population or the total population over 25 years are shown. The dashed line at 1.0 indicates the transition point between under- and overrepresentation. Source: Authors’ calculations using Integrated Public Use Microdata Series American Community Survey (www.ipums.org) and NIH Information for Management, Planning, and Coordination II data. Abbreviations: NIH indicates National Institutes of Health.

Discussion

Implications

In general, we found the RPG and R01-equivalent awardee pools more likely to be white and male, and the trainee/fellow postdoc and K-awardee pools more likely to be a combination of minority races and ethnicities, and female. When we investigated this trend further by comparing sex differences within racial and ethnic groups, we found that females are more likely than counterpart males to be in the trainee/fellow postdoc and K-awardee pools and were less likely to be in the RPG and R01-equivalent awardee groups, without exception. Interestingly, between the RPG and R01-equivalent awardee pools, women of all racial and ethnic groups were less likely to be in the R01-equivalent pool than the RPG awardee pool, and men of all racial and ethnic groups except black were more likely to be in the R01-equivalent pool than the RPG awardee pool. The RPG awardee pool was more diverse than the R01-equivalent awardee pool.

Looking at sex only, race only, or even the combination of sex and race does not tell the entire story. Differences in citizenship may indicate differences in educational intensity, rigor, and experiences from the home country. Students who identify as underrepresented racial and ethnic minorities in the United States but who are from other countries may bring different educational experiences than similarly situated racial and ethnic minorities who received their educational training in the United States. African students may identify themselves as black or African American; however, their educational experience may differ from African American students who received their K–12 education solely through the U.S. system.

When we analyzed the data by also examining citizenship status, we found that the overrepresentation of white males in NIH-supported populations is only true for born and naturalized citizens and not for their noncitizen counterparts. Similarly, Asian and Hispanic noncitizens were consistently less likely to be in the NIH-supported populations than their citizen counterparts. We were unable to compare black citizens with black noncitizens because the noncitizen sample sizes were too small in the NIH population.

Though the trends for the trainee/fellow postdoc and K-awardee pool discussed above appear to be largely identical even after considering citizenship status, one exception deserves discussion. Asian female citizens were close to parity in the trainee/fellow postdoc and K-awardee pools (1.03 and 0.97, respectively), though when considering all Asian females regardless of citizenship status, they were underrepresented (0.38 and 0.65, respectively). This difference is likely due to the overrepresentation of Asian noncitizens in the advanced-degree population that do not appear in the NIH-supported population. The trend for Asian males, although similar, did not show as marked a difference as their female counterparts.

An interesting pattern emerges when comparing the citizens of each male racial and ethnic group with the combined citizens and noncitizens for those groups. Specifically, for RPG and R01-equivalent awards, while white males were similarly represented in the combined citizen and noncitizen groups, each of the other racial and ethnic groups showed a marked increase in representation for the citizen males compared with the combined citizen and noncitizen groups. For both Asian males and Hispanic males, the trend changed from underrepresentation to overrepresentation when excluding noncitizens from the NIH RPG and R01-equivalent awardee populations.

Limitations

We acknowledge that the relevant labor market concept may be a controversial means of evaluating the awarding of NIH grants because the grant-awarding process considers both the qualifications of the applicant and the merit of the proposed research, and also considers the research environment as evaluated during the peer review process. Furthermore, the nonanalogous positions to the labor markets noted above may result in an imperfect analysis of the NIH-funded scientific workforce because of the specialized backgrounds of its members as well as other factors. However, this may prove to be a minor concern, as courts have been able to approximate pools of qualified individuals.

Browne32 argues that over- and underrepresentation could result from any of several causes; for instance, people of different classes may not be equally interested in working for a given organization or be equally qualified. This is a valid critique, and the NIH should address this weakness by analyzing actual applicant or applicant flow data. Further studies are in progress for NIH-relevant populations.

Directions for future research

More exploration is needed to understand why some groups—for example, women—are not fully represented at the end stages of the biomedical career pathway in NIH-funded programs. Are we observing a demographic transition in the sex and racial composition of biomedical scientists? RPG and R01-equivalent awardee pools are generally composed of older individuals than are trainee/fellow postdoc and K-awardee pools. Will the diversity we see now in training and early career development programs eventually transition into the established investigator pool as more diverse cohorts age through the pipeline? Or, are some groups falling off between the postdoc and K-awardee pools and the RPG and R01-equivalent awardee pools, and if so, why?

Our approach reveals a more accurate picture of the representation of women and minorities among NIH awardees, which can lead to better-designed studies and interventions. For example, are there institutional barriers or biases that disproportionately impact women or persons from traditional “minority” backgrounds? Are these individuals self-selecting out of the pathway because of the lack of flexibility commonly found in academic cultures? What are the “push” and “pull” factors that influence persistence in scientific careers? If people are self-selecting out of the pathway, whether because of an increased value on work–life balance or for other reasons, then the implementation of additional diversity programs at the NIH would likely have little or no effect.

We have not yet examined the age distribution among racial and ethnic categories in the NIH-supported population for differences within award programs. The R01-equivalent awardees have a higher average age than the RPG awardees (a group primarily composed of R21, R03, and R01 awardees). Further studies may help determine whether the greater diversity within the trainee/fellow postdoc and K-awardee pool, and to an extent even the RPG awardee pool, would theoretically diminish over time as more diverse younger cohorts gradually age and move into the RPG and R01-equivalent awardee pools.

Conclusions

The diversity of an NIH-funded workforce needs to be understood within the context of the relevant labor market, consistent with the long-standing principles of employment law and economic theory. For the purposes of our study, we identified the relevant labor market as those persons with advanced degrees who work in biological or biomedical research. When we compare the pool of NIH-funded individuals versus the relevant labor market, we found that women and minority groups are generally overrepresented in training and early-career mentored programs and underrepresented in independent research awards. These findings are not surprising, given the NIH’s efforts to provide pathways for groups that have historically been underrepresented in NIH-funded research.6

The full story, however, is more complex. When we examined the data by citizenship status, we found that citizens who are male, white, Asian, or Hispanic are overrepresented in the NIH independent researcher pool, whereas white women are almost equally represented in the NIH independent researcher pool. Black citizens (men and women), Hispanic women, and Asian women are underrepresented. All noncitizens are underrepresented in all aspects of NIH funding mechanisms.

The NIH continues building its capacity to answer important questions related to diversity. It is essential for the United States to maintain a viable and innovative biomedical research workforce at all levels to ensure the most productive biomedical research endeavors and the most effective use of taxpayer dollars. This study moves one step closer to providing baseline information that supports the efforts of the NIH to develop and advance comprehensive long-term strategies to address all components of the biomedical research enterprise—including trainees, biomedical researchers in academia and industry, and scientists in research-related occupations.

While our analysis describes the current state of affairs regarding diversity within the NIH-funded workforce and its relative labor market, further research is needed. Follow-up studies should focus on identifying whether younger cohorts will become more diverse older cohorts in future years. They should also assess the representation ratios of racial and ethnic subgroups within major race and ethnic groups; the results reported in this study for Hispanics, for example, might not be consistent when comparing Hispanic individuals with backgrounds from different countries. Our study did not attempt to address current policy or program-related issues, but it does provide an essential lens for understanding NIH workforce diversity and sets the stage for continued analysis.

Acknowledgments: The authors thank Katrina Pearson, Deepshikha Roychowdhury, PhD, and the National Institutes of Health (NIH) Division of Statistical Analysis and Reporting for the data retrieval and for providing advice and suggestions regarding the data retrieved. They thank Samuel L. Myers Jr, PhD (University of Minnesota) for feedback and guidance, Cindy Clark (NIH Library Editing Service) for reviewing the manuscript, as well as Ying Zeng (PhD student, University of Maryland) and Julia Rollison, PhD, Paul Lagasse, MA, MLS, and Elyse Sullivan, PhD (Ripple Effect Communications, Inc.), for critically reading the manuscript and providing thoughtful feedback and comments.

References

1. Leonard JS, Levine DI. The effect of diversity on turnover: A large case study. Ind Labor Relat Rev. 2006;59:547572.
2. Gurin P. The compelling need for diversity in education. Expert report prepared for the lawsuits Gratz and Hamacher v Bollinger, Duderstadt, the University of Michigan, and the University of Michigan College of LS&A, Civil Action No. 97-75231 (U.S. District Court, Eastern District of Michigan) and Grutter v Bollinger, Lehman, Shields, the University of Michigan and the University of Michigan Law School, Civil Action No. 9775928. http://www.umich.edu/~urel/admissions/legal/expert/gurintoc.html. Published 1999. Accessed February 25, 2016.
3. Hong LP, Page SE. Problem solving by heterogeneous agents. J Econ Theory. 2001;97:123163.
4. Hong L, Page SE. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc Natl Acad Sci USA. 2004;101:1638516389.
5. Ottaviano GIP, Peri G. The economic value of cultural diversity: Evidence from U.S. cities. J Econ Geogr. 2006;6:944.
6. National Institutes of Health. Office of Extramural Research. Notice of NIH’s interest in diversity. Notice number: NOT-OD-15-053. http://grants.nih.gov/grants/guide/notice-files/NOT-OD-15–053.html. Released January 12, 2015. Accessed February 25, 2016.
7. National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. 2011.Washington, DC: National Academies Press.
8. Myers SL, Turner CS. The effects of Ph.D. supply on minority faculty representation. Am Econ Rev. 2004;94:296301.
9. Myers SJ, Husbands-Fealing K. Changes in the representation of women and minorities in biomedical careers. Acad Med. 2012;87:15251529.
10. Denson NC, Chang MJ. Racial diversity matters: The impact of diversity-related student engagement and institutional context. Am Educ Res J. 2009;46:322353.
11. National Academy of Sciences (U.S.), National Academy of Engineering (U.S.), and Institute of Medicine (U.S.) Committee on Maximizing the Potential of Women in Academic Science and Engineering. Beyond Bias and Barriers: Fulfilling the Potential of Women in Academic Sciences and Engineering. 2007. Washington, DC: National Academies Press; http://www.nap.edu/openbook.php?record_id=11741. Accessed February 25, 2016.
12. Pohlhaus JR, Jiang H, Wagner RM, Schaffer WT, Pinn VW. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med. 2011;86:759767.
13. Ginther DK, Haak LL, Schaffer WT, Kington R. Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Acad Med. 2012;87:15161524.
14. Ginther DK, Schaffer WT, Schnell J, et al. Race, ethnicity, and NIH research awards. Science. 2011;333:10151019.
15. Ginther DK, Schaffer WT, Schnell J, et al. Diversity in academic biomedicine: An evaluation of education and career outcomes with implications for policy. http://ssrn.com/abstract=1677993. Published September 22, 2009. Accessed February 25, 2016.
16. National Institutes of Health. Advisory Committee to the Director. Working Group on Diversity in the Biomedical Research Workforce. http://acd.od.nih.gov/dbr.htm. Last reviewed February 12, 2011. Accessed February 25, 2016.
17. National Institutes of Health. Advisory Committee to the Director. Biomedical Workforce Task Force. http://acd.od.nih.gov/bwf.htm. Last reviewed February 12, 2011. Accessed February 25, 2016.
18. Building Engineering & Science Talent. The talent imperative: Meeting America’s challenge in science and engineering, ASAP. http://www.bestworkforce.org/PDFdocs/BESTTalentImperativeFINAL.pdf. Published 2004. Accessed February 25, 2016.
19. National Science and Technology Council. Ensuring a Strong U.S. Scientific, Technical, and Engineering Workforce in the 21st Century. 2000.Washington, DC: Office of Science and Technology.
20. National Institute of General Medical Sciences. Investing in the future: National Institute of General Medical Sciences strategic plan for biomedical and behavioral research training 2011. http://publications.nigms.nih.gov/trainingstrategicplan/Strategic_Training_Plan.pdf. Accessed February 25, 2016.
21. United States Census Bureau. American FactFinder. http://factfinder.census.gov. Accessed February 25, 2016.
22. U.S. Department of Labor Office of Federal Contract Compliance Programs. Federal contractors’ obligation to maintain and analyze race and ethnicity data of applicants and employees in AAPs prepared in accordance with executive order 11246, as amended (referenced as transmittal No. 283 and ADM notice/other), OFCCP directive 2008-02. http://www.dol.gov/ofccp/regs/compliance/directives/dir283.htm. Accessed March 5, 2016.
23. Association of American Medical Colleges. Faculty roster. https://www.aamc.org/data/facultyroster/. Accessed May 11, 2016.
24. Collins FS. EEO and Diversity and Inclusion Policy Statement. December 19, 2014. Bethesda, Md: National Institutes of Health, U.S. Department of Health and Human Services; http://edi.nih.gov/sites/default/files/public/EDI_Public_files/policy/eeodm-06.pdf. Accessed February 25, 2016.
25. Ruggles S, Alexander JT, Genadek K, Goeken R. Integrated Public Use Microdata Series 2010.Version 5.0. Minneapolis, Minn.
26. Smith JP, Welch FR. Closing the Gap: Forty Years of Economic Progress for Blacks. 1986.Santa Monica, Calif: Rand Corporation.
27. Myers SL Jr. Chen S. The economics of diversity: The efficiency vs. equity trade-off. In: Diversity Management: Theoretical Perspectives and Practical Approaches. 2011.Hauppauge, NY: Nova Science Publishers.
28. Myers SL Jr. Johnson NJ, Svara JH. The economics of diversity. In: Justice for All: Promoting Social Equity in Public Administration. 2011.Armonk, NY: M.E. Sharpe.
29. Darity WA Jr, Myers SL Jr. Black Economic Progress: A Case Against the Dramatic Improvement Hypothesis. 1980. Madison, Wisc: University of Wisconsin–Madison Institute for Research on Poverty.
30. U.S. Census Bureau. American Community Survey: Multiyear accuracy of the data (3-year 2010–2012 and 5-year 2008–2012). http://www2.census.gov/programs-surveys/acs/tech_docs/accuracy/MultiyearACSAccuracyofData2012.pdf. Published October 28, 2013. Accessed February 25, 2016.
31. Altman DG, Bland JM. Interaction revisited: The difference between two estimates. BMJ. 2003;326:219.
32. Browne KR. Statistical proof of discrimination: Beyond “damned lies.” Wash Law Rev. 1993;68:477484503.

Supplemental Digital Content

Copyright © 2016 by the Association of American Medical Colleges