Concerns abound regarding the physician–scientist pipeline in the United States,1 particularly given the increasingly competitive funding environment.2,3 Prior research has demonstrated that fewer than half of the recipients of prestigious K series career development awards from the National Institutes of Health (NIH) from 1997 to 2003 went on to achieve independent R01 awards, and women fared worse than men.4 A prior survey study from our group demonstrated that gender differences in outcomes persisted in that older cohort even when considering a broader definition of success that included other grant funding, publication track record, or administrative leadership, but the cross-sectional nature of that evaluation precluded ascertainment of what other factors influenced success and whether they might have mediated the relationship between gender and success.5
Research and commentary have identified a number of factors that may influence the success of clinicians who pursue research careers in academic medicine and that might mediate gender differences in outcomes that cannot be explained by differences in basic job characteristics. These include mentorship,6–14 access to resources,15,16 time allocation, and other aspects of the work environment.17–20 They also include competing demands in the domestic sphere,21–26 along with the priorities of the individuals themselves.27
To investigate the outcomes of a recent cohort of clinician–researchers whose careers have evolved after the economic downturn and the restriction of funding for biomedical research, we surveyed a cohort of clinician–researchers who received new K08 and K23 career development awards in 2006 through 2009. We employed a longitudinal design to allow for the evaluation of the influence of various factors on success, including demographics, basic job characteristics, the work environment, priorities, and domestic responsibilities. We have previously reported findings regarding the baseline characteristics of this sample6,15,25; this report constitutes the analysis of the primary objective of the longitudinal study, which was funded by the NIH through an R01 research grant.
Survey sampling and administration
Using the NIH RePORTER database, we identified 1,719 clinician–researchers who received new K08 and K23 awards in 2006–2009. After gaining approval from the University of Michigan institutional review board, we conducted Internet searches and made telephone calls to obtain background information and current U.S. mailing addresses for these recipients.
In 2010–2011 (hereinafter called T1), we conducted a baseline survey of 1,708 clinician–researchers who received NIH K08 and K23 awards in 2006–2009, and for whom we were able to obtain valid U.S. mailing addresses. We mailed a paper survey along with a $50 incentive. We used a modified Dillman approach28 to remind nonrespondents and maximize response rates. We received 1,275 responses (75% response rate).
Approximately four years later (in 2014, hereinafter called T2), we conducted a follow-up survey, administered exclusively to the 1,275 respondents to the initial questionnaire, such that all individuals with T2 data also had T1 data available for analysis. Prior to distributing this follow-up survey, we once again conducted Internet searches and made telephone calls to obtain current contact information. We were able to verify current U.S. mailing addresses for 1,258 out of the 1,275 who responded to our initial survey mailing. We mailed the paper follow-up survey questionnaire along with a $50 incentive and received 1,066 responses (85% response rate).
Study data were managed using REDCap electronic data capture tools hosted at the University of Michigan. REDCap (Research Electronic Data Capture)29 is a secure, Web-based application designed to support data capture for research studies, providing an intuitive interface for validated data entry; audit trails for tracking data manipulation and export procedures; automated export procedures for seamless data downloads to common statistical packages; and procedures for importing data from external sources.
Survey design and measures
We designed the two survey questionnaires after consideration of the published literature and previous instruments used to determine the characteristics and outcomes of academic careers. After developing a conceptual model that identified hypothesized relationships between key constructs, we conducted additional literature review to identify measures of those constructs and followed standard techniques of survey design and validation where existing measures were insufficient. This included intensive cognitive pretesting of the entire survey instruments with individuals similar to the intended target population, using verbal probing and think-aloud reasoning. Both questionnaires were 12-page booklets (Supplemental Digital Appendices 1 and 2, https://links.lww.com/ACADMED/A450).
Our primary dependent variable of interest was a composite measure of success measured at T2. This was defined as in our previous work as having accomplished any one or more of the following: attainment of > $1 million in grant funding as principal investigator (PI), publication of 35 or more peer-reviewed articles, or administrative leadership (service as dean, department chair, or division chief).
We also evaluated the individual components of the composite success measure separately (grant funding, publications, and leadership), along with several other measures of outcome at T2: academic rank, promotion (defined as increase in academic rank from T1 to T2), laboratory space, research time, and the individual’s own perceptions of his or her success. We assessed whether respondents left or considered leaving the institutions at which they received their K awards, along with the reasons for this. We also evaluated whether respondents remained at T2 in the same jobs they had at T1 or whether they had changed institutions or careers.
We defined five categories of independent variables measured in the survey at T1: demographics, basic job characteristics, aspects of the work environment, priorities, and domestic responsibilities.
Demographic characteristics included gender, age, and race. Basic job characteristics included K award type (K08 vs. K23 to distinguish those pursuing basic science research from those pursuing patient-oriented research), year of K award (2006–2009), NIH funding institute tier (broken into three tiers as defined as in our previous work based on the total amount of R01 awards granted by the institute that funded the individual’s K award), institution tier (broken into four tiers as defined in our previous work based on the NIH funding ranking of the individual’s academic institution), degree (whether the individual held a PhD in addition to a clinical doctorate), and specialty (grouped by nature as in our prior work).16
Aspects of the work environment included measures of time allocation at work (overall work hours, hours devoted to clinical care, and hours devoted to research). We also included several measures of the mentoring received by the individual: frequency of weekly contact with the primary designated K award mentor, the number of monthly hours of one-on-one contact with the primary mentor, a scaled measure of the primary mentor’s behavior (defined in our previous work),6 a scaled measure of the prestige of the primary mentor, the collegiality of the relationship with the primary mentor, and satisfaction with mentoring from all sources. We included measures of access to resources, including perceived adequacy of research space, research equipment, secretarial support, access to grant administrators, and access to statistical support. Finally, we evaluated climate with items evaluating perceptions of being treated unfairly in one’s job and perceptions of the supportiveness of the individual’s department/division.
Measures of individual priorities included items evaluating the extent to which the individual indicated feeling a responsibility to contribute to the teaching mission of his or her department/division, conduct his or her own research, support the research of his or her colleagues, contribute to the clinical care provided by his or her department/division, and play a role in department/division administrative issues. We also included items evaluating the importance to the individual of leadership, high-quality research, publishing prolifically, and balancing work and other activities. Finally, we included measures of preferences for time spent on patient care and time spent on research.
Domestic responsibilities were evaluated through measures of marital status, parental status, spousal employment, weekly hours devoted to parenting and domestic tasks, elder care responsibilities, and personal/family dependence on respondent’s income.
We first compared the characteristics of individuals in our analytic sample of respondents (all of whom had responded to both surveys) versus those in the initial target population who failed to respond, for those characteristics which were known in the entire target population (determined based on Internet searching). We then described the demographics, basic job characteristics, aspects of the work environment, priorities, and domestic responsibilities reported by the analytic sample at T1 by gender.
We next evaluated whether those in our analytic sample remained at T2 in the same jobs they had at T1 or whether they had changed institutions or careers, as well as whether they had at least considered doing so. We then compared, by gender (after adjustment for basic job characteristics), the reasons for leaving or considering leaving. We described career outcomes by gender, reporting P values comparing these outcomes by gender after adjustment for basic job characteristics in multivariable regression models.
Next, we evaluated the extent to which the measured factors mediated the relationship between gender and the composite measure of success. To conduct this mediation analysis, we first created a logistic regression model to evaluate the unadjusted association between gender and success, and then evaluated how the coefficient for gender changed in a model that included gender and all potential mediators (those variables found to be significantly associated on univariate analysis with the outcome variable—success—as well as significantly associated on univariate analysis with the principal predictor of interest—gender).
Finally, we constructed a multiple-variable logistic regression model of the composite success outcome, after considering all of the T1 characteristics listed in the measures section and Table 1 as candidate independent variables. Interactions between gender and the demographic, basic job characteristics, and domestic responsibilities in Table 1 were screened for during the model-building process. A backward elimination process was used to select important covariates based on consideration of the Akaike information criterion,30,31 to reduce the number of theoretically important variables to those which are most important without sacrificing explanatory power. The model started with all covariates, including significant interactions. Variables were removed iteratively and the model recalculated until only significant covariates remained. For the final model, we report the area under the receiver operator curve (AUC), a measure of the predictive ability of the model, with 1 being perfect prediction and 0.5 being expected from random chance. Finally, to explore whether different factors explain success in men versus women, we also constructed separate models of success for men and for women, again using backward elimination, as described earlier.
Analyses were conducted using SAS statistical software, version 9.4 (SAS Institute Inc., Cary, North Carolina); P values < .05 were considered significant throughout.
Although response rate did not differ by gender, several other differences were observed upon comparison of the 1,066 individuals who responded to both our baseline and follow-up surveys versus the 653 who did not respond but who also received new K08 or K23 awards in 2006–2009. Specifically, response rate varied significantly by degree (69.5% [189/272] among non-MDs, 61.5% [630/1,024] among MDs, and 58.8% [247/420] among MD/PhDs, P = .02); K award type (58.6% [520/888] among K08s vs. 65.7% [546/831] among K23s, P = .002); year of K award (57.7% [235/407] for 2006, 62.0% [253/408] for 2007, 59.8% [265/443] for 2008, and 67.9% [313/461] for 2009, P = .01); specialty (65.8% [221/336] for women/children/families, 56.2% [123/219] for hospital-based, 61.2% [60/98] for surgical, 60.0% [443/738] for medical, 55.6% [30/54] for basic science, and 69.5% [189/272] for other, P = .02); K award funding institute tier (59.3% [264/445] for first, 58.8% [419/712] for second, and 68.1% [383/562] for third, P = .001); and K award institution tier (58.2% [198/340] for first, 59.7% [290/486] for second, 60.5% [287/474] for third, and 69.6% [279/401] for fourth, P = .004).
Table 1 describes the demographics, basic job characteristics, aspects of the work environment, priorities, and domestic responsibilities reported by the respondents at T1 by gender. Women and men did not differ by age or race. Several basic job characteristics did differ by gender, with 65% (322) of women holding K23s and 61% (349) of men holding K08s, suggesting that women were more likely to be among the patient-oriented researchers and men among the laboratory researchers. Women were also less likely to hold awards from the NIH institutes in the highest tier for funding (17%  vs. 31% ). Women were less likely to be MD/PhDs (15%  vs. 31% ) and more likely to be non-MDs (26%  vs. 11% ). Women were more likely to have a specialty focused on women/children/families (32%  vs. 21% ) and less likely to have a surgical specialty (3%  vs. 10% ).
Other features of the work environment also differed by gender, with men reporting longer mean weekly total work hours (58 vs. 53), hours in clinical care (9.7 vs. 7.1), and hours in research (38.5 vs. 34.8). Most mentoring measures did not vary significantly by gender, with the exception of longer mean monthly one-on-one meeting time for men (5.1 vs. 4.6 hours) and greater proportions of men perceiving collegiality in the mentoring relationship (69%  vs. 61% ). Although women did not report less adequate space or equipment, they did report less adequate access to human resources, including secretaries (56%  vs. 49% ), grants administrators (36%  vs. 27% ), and statisticians (48%  vs. 41% ). Women were more likely to have perceived being treated unfairly in their jobs (38%  vs. 28% ).
Few differences were observed in the priorities of the men and women among these respondents, but men were more likely to report a strong sense of responsibility to contribute to clinical care in their departments (64%  vs. 55% ), and women were more likely to report strong importance of work–life balance (96%  vs. 91% ). Domestic responsibilities differed substantially between men and women, with women less likely to be married or partnered (87%  vs. 93% ), less likely to have children (76%  vs. 83% ), and more likely to have a full-time employed partner if married or partnered (85%  vs. 47% ). Women spent more time on parenting or domestic tasks (mean weekly hours 37.8 vs. 30.9) and reported less dependence on their own income for family support (23%  vs. 10%  reporting being somewhat or not at all dependent).
Figure 1 shows respondents’ career paths. Overall, 765 (71.8%; 362 [73.4% of females] and 403 [70.3% of males]) were still at the institution at which they received their initial K awards. Of those who had left their original K award institution, about three-quarters (76.4%) had gone to work for another academic institution, with 12.6% leaving for a community-based or private medical practice, and 7.3% having gone to work for industry. Overall, 315 (63.9%) women and 382 (66.2%) men had at least considered leaving their institution. Although few gender differences were observed in these outcomes, significant gender differences did emerge when those who left or considered leaving were asked about their reasons. Men were more likely to have left or considered leaving for a better opportunity elsewhere (60.8% [129/212] of men vs. 34.2% [63/184] of women, P = .05), and women were more likely to have left or considered leaving because of job opportunities for a spouse/partner/significant other (13.6% [25/184] of women vs. 8.0% [17/212] of men, P = .04). Similar proportions of men and women reported that they left or considered leaving because of dissatisfaction with mentoring (29.9% [55/184] of women and 28.8% [61/212] of men, P = .42), inadequate protected research time (31.0% [57/184] of women and 22.2% [47/212] of men, P = .55), or dissatisfaction with work–life balance (48.9% [90/184] of women and 29.2% [62/212] of men, P = .10).
Table 2 summarizes the career outcomes of our respondents by gender, with comparisons by gender adjusted for basic job characteristics in multivariable regression models. Women were more likely than men to be at the rank of assistant professor (51% of women  vs. 42% of men ) at the time of the follow-up survey. Women were less likely to hold institutional (6% of women  vs. 11% of men ) or national leadership positions (41% of women  vs. 49% of men ). Women reported less grant funding as PI (mean $780,000 vs. $1,120,000, P = .002), with 28% of women (131) and 38% of men (201) (P = .006) reporting that they had attained > $1 million in funding as PI or co-PI. Women published fewer papers overall (mean 33 vs. 45) and as first author (13 vs. 15) or senior author (6 vs. 10), along with significant differences in other types of publications, presentations, and perceptions of success and career progress.
With a composite measure of success, defined as funding success (R01 attainment at T2 or >$1M in other PI grant funding), publication success (35 or more peer-reviewed publications), or institutional leadership (becoming a dean, department chair, or division chief), 648 respondents (60.8%) reported success: 53.5% (264/493) of women and 67.0% (384/573) of men (Figure 2). In a logistic regression model of this composite success outcomes measure explained by gender alone, the odds ratio (OR) for females versus males was 0.56 (CI: 0.44–0.72) and was highly significant (P < .001). After including all variables from Table 1 that were potential mediators, the OR increased only slightly to 0.62 (CI: 0.46–0.84) and was still highly significant (P = .002).
Table 3 presents a best parsimonious multiple-variable logistic regression model of the composite measure of success, with an AUC of 0.76. The one significant interaction observed, between gender and K award grant type, was retained in this multiple-variable model. The direction of this interaction was such that the gender association with success appeared confined primarily to those with K23 grants, the grant type awarded to 65.3% (322) of female and 39.1% (224) of male respondents. Female K23 recipients were much less likely to be successful than male K23 recipients (OR = 0.32). The direction of this interaction effect is apparent in the observed rates of success in the four groups of gender by K award type without covariate adjustment: 59.0% (206/349) among male K08 awardees, 79.5% (178/224) among male K23 awardees, 51.5% (88/171) among female K08 awardees, and 54.7% (176/322) among female K23 awardees.
In the multiple-variable model, in addition to gender and K award type, variables significantly associated with success, listed in order of importance after standardizing each coefficient estimate by a function of its variance, were specialty, K award year, work hours, funding institute tier, higher feeling of responsibility to play a role in department/division administration, high importance of publishing prolifically, lower feeling of responsibility to contribute to clinical care, higher importance of publishing high-quality research, greater collegiality of the mentoring relationship, adequacy of research equipment, and supportive climate of the primary department or division.
In a separate model considering women alone, variables significantly associated with success in a best parsimonious model, again listed in order of importance (full model details provided in Supplemental Digital Appendix 3 at https://links.lww.com/ACADMED/A450), were K award year, funding institute tier, higher feeling of responsibility to play a role in department/division administration, lack of desire to increase or decrease time spent on research, hours spent on research, and importance of publishing prolifically. For men alone, the variables in the best parsimonious model of success were K award grant type, K award year, work hours, specialty, supportive climate of the primary department or division, higher importance of publishing high-quality research, greater collegiality of the mentoring relationship, race, and lower feeling of responsibility to contribute to clinical care.
Our work sought to understand barriers facing junior physician faculty researchers, particularly women, to inform efforts to improve gender equity in academic medicine. We focused on an extremely selective cohort of research-oriented junior faculty members—namely, recipients of K08 and K23 mentored career development awards from the NIH. In so doing, our study design minimized the impact of potential gender differences in the desire to pursue research and in access to monetary support, allowing for the impact of other challenges and barriers to be isolated. We included a large number of factors that have previously been hypothesized to affect success and to mediate gender differences in outcomes.
Our findings suggest that men and women continue to have different experiences both at work and at home, as well as different career outcomes and academic success as traditionally defined. We evaluated whether the substantial differences we have observed in factors such as domestic responsibilities, work hours, and adequacy of human resources could explain gender differences in outcomes in this cohort, finding a strong association between gender and success even after accounting for these factors. We found that beyond basic job characteristics (including K award type and year, specialty, and funding institute tier), certain factors are associated with success. These include aspects of the work environment (including work hours, a collegial relationship with one’s mentor, adequacy of research equipment, and a supportive institutional climate) and certain personal priorities. Yet even after including these mechanistic factors, gender differences in outcomes persisted, primarily among those clinically oriented researchers holding K23 awards. Other research has focused on the particular challenges of obtaining funding for clinical research,32–34 which was the focus of the majority of the female K awardees in our study population; why male K awardees pursuing clinical research (who constituted only a minority of the men) were more successful requires further investigation. Given that prior work has identified the submission of competing continuation applications to be a potentially important explanation for differences in funding success between clinical and nonclinical applications, it would be particularly interesting to examine whether gender differences in this regard might contribute. Our findings suggest that much remains to be done to understand the causes of gender differences in career outcomes and to ensure gender equity and success of the promising pipeline of individuals pursuing careers in academic medicine.
Like any survey study, our findings are vulnerable to bias. Although our response rates were high, our respondents were dissimilar from nonrespondents in certain meaningful ways, although not by gender. Survey responses are further vulnerable to measurement error, despite our best efforts to use valid measures of the key constructs we considered. In particular, if men are more prone to inflating their responses in comparison with women when reporting variables such as hours worked or achievements, this could explain some of the gender difference observed; still, given the number of outcomes evaluated and consistent findings even with respect to outcomes such as academic rank, which should not be vulnerable to misestimation, we do not believe this is the sole explanation for our observations. Although our study included many factors, our model had only moderate predictive ability, suggesting that unmeasured factors exist that are important in explaining success. Finally, the experiences of this elite group of career development award recipients may differ from those of less able or motivated junior faculty without such awards.
Nevertheless, these findings from a large national sample of clinician–researchers are timely and relevant for policy makers seeking evidence to inform their efforts to improve gender equity and the health of the physician–scientist pipeline. In particular, given that time for research appears important for women’s success, institutions must learn from one another to adopt best practices that may increase women’s access to time for research. Creative approaches, ranging from unique bridge funding programs35 that provide support for “extra hands” to allow women to make best use of their time to interventions that call attention to implicit gender biases36 that may affect women’s ability to secure protected time for research, have been shown to have meaningful impact. Our current study underscores the need for more widespread adoption of such approaches, as well as the need for further research in this area, as the goals of gender equity and more widespread success in academic medicine remain unfulfilled, even amongst the best and brightest who have recently entered our field.
The authors wish to thank the K award recipients who took the time to participate in this study.
1. Schafer A. The Vanishing Physician–Scientist? 2009.Ithaca, NY: Cornell University Press.
2. Chakma J, Sun GH, Steinberg JD, Sammut SM, Jagsi R. Asia’s ascent—Global trends in biomedical R&D expenditures. N Engl J Med. 2014;370:36.
3. Moses H 3rd, Dorsey ER. Biomedical research in an age of austerity. JAMA. 2012;308:23412342.
4. Jagsi R, Motomura AR, Griffith KA, Rangarajan S, Ubel PA. Sex differences in attainment of independent funding by career development awardees. Ann Intern Med. 2009;151:804811.
5. Jagsi R, DeCastro R, Griffith KA, et al. Similarities and differences in the career trajectories of male and female career development award recipients. Acad Med. 2011;86:14151421.
6. DeCastro R, Griffith KA, Ubel PA, Stewart A, Jagsi R. Mentoring and the career satisfaction of male and female academic medical faculty. Acad Med. 2014;89:301311.
7. DeCastro R, Sambuco D, Ubel PA, Stewart A, Jagsi R. Mentor networks in academic medicine: Moving beyond a dyadic conception of mentoring for junior faculty researchers. Acad Med. 2013;88:488496.
8. Pololi L, Knight S. Mentoring faculty in academic medicine. A new paradigm? J Gen Intern Med. 2005;20:866870.
9. Bussey-Jones J, Bernstein L, Higgins S, et al. Repaving the road to academic success: The IMeRGE approach to peer mentoring. Acad Med. 2006;81:674679.
10. Mayer AP, Files JA, Ko MG, Blair JE. The academic quilting bee. J Gen Intern Med. 2009;24:427429.
11. Moss J, Teshima J, Leszcz M. Peer group mentoring of junior faculty. Acad Psychiatry. 2008;32:230235.
12. Lewellen-Williams C, Johnson VA, Deloney LA, Thomas BR, Goyol A, Henry-Tillman R. The POD: A new model for mentoring underrepresented minority faculty. Acad Med. 2006;81:275279.
13. Files JA, Blair JE, Mayer AP, Ko MG. Facilitated peer mentorship: A pilot program for academic advancement of female medical faculty. J Womens Health (Larchmt). 2008;17:10091015.
14. Pololi LH, Knight SM, Dennis K, Frankel RM. Helping medical school faculty realize their dreams: An innovative, collaborative mentoring program. Acad Med. 2002;77:377384.
15. Holliday E, Griffith KA, De Castro R, Stewart A, Ubel P, Jagsi R. Gender differences in resources and negotiation among highly motivated physician–scientists. J Gen Intern Med. 2015;30:401407.
16. Jagsi R, Griffith KA, Stewart A, Sambuco D, DeCastro R, Ubel PA. Gender differences in salary in a recent cohort of early-career physician–researchers. Acad Med. 2013;88:16891699.
17. Price EG, Gozu A, Kern DE, et al. The role of cultural diversity climate in recruitment, promotion, and retention of faculty in academic medicine. J Gen Intern Med. 2005;20:565571.
18. Shollen SL, Bland CJ, Finstad DA, Taylor AL. Organizational climate and family life: How these factors affect the status of women faculty at one medical school. Acad Med. 2009;84:8794.
19. Foster SW, McMurray JE, Linzer M, Leavitt JW, Rosenberg M, Carnes M. Results of a gender-climate and work-environment survey at a midwestern academic health center. Acad Med. 2000;75:653660.
20. Dannels S, McLaughlin J, Gleason KA, McDade SA, Richman R, Morahan PS. Medical school deans’ perceptions of organizational climate: Useful indicators for advancement of women faculty and evaluation of a leadership program’s impact. Acad Med. 2009;84:6779.
21. Carr PL, Ash AS, Friedman RH, et al. Relation of family responsibilities and gender to the productivity and career satisfaction of medical faculty. Ann Intern Med. 1998;129:532538.
22. Warde CM, Moonesinghe K, Allen W, Gelberg L. Marital and parental satisfaction of married physicians with children. J Gen Intern Med. 1999;14:157165.
23. Sobecks NW, Justice AC, Hinze S, et al. When doctors marry doctors: A survey exploring the professional and family lives of young physicians. Ann Intern Med. 1999;130(4 pt 1):312319.
24. Uhlenberg P, Cooney TM. Male and female physicians: Family and career comparisons. Soc Sci Med. 1990;30:373378.
25. Jolly S, Griffith KA, DeCastro R, Stewart A, Ubel P, Jagsi R. Gender differences in time spent on parenting and domestic responsibilities by high-achieving young physician–researchers. Ann Intern Med. 2014;160:344353.
26. Dyrbye LN, Shanafelt TD, Balch CM, Satele D, Sloan J, Freischlag J. Relationship between work–home conflicts and burnout among American surgeons: A comparison by sex. Arch Surg. 2011;146:211217.
27. Jones RD, Griffith KA, Ubel PA, Stewart A, Jagsi R. A mixed-methods investigation of the motivations, goals, and aspirations of male and female academic medical faculty. Acad Med. 2016;91:10891097.
28. Dillman D, Smyth J, Christian L. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 2009.3rd ed. Hoboken, NJ: Wiley & Sons.
29. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
30. Akaike H. Petrov BN, Csáki F. Information theory and an extension of the maximum likelihood principle. In: 2nd International Symposium on Information Theory, Tsahkadsor, Armenia, USSR, September 2–8, 1971. 1973:Budapest, Hungary: Akadémiai Kiadó; 267281.
31. Burnham KP, Anderson DR. Multimodel inference: Understanding AIC and BIC in model selection. Sociol Methods Res. 2004;33:261304.
32. Martin MR, Lindquist T, Kotchen TA. Why are peer review outcomes less favorable for clinical science than for basic science grant applications? Am J Med. 2008;121:637641.
33. Kotchen TA, Lindquist T, Miller Sostek A, Hoffmann R, Malik K, Stanfield B. Outcomes of National Institutes of Health peer review of clinical grant applications. J Investig Med. 2006;54:1319.
34. Kotchen TA, Lindquist T, Malik K, Ehrenfeld E. NIH peer review of grant applications for clinical research. JAMA. 2004;291:836843.
35. Jagsi R, Butterton JR, Starr R, Tarbell NJ. A targeted intervention for the career development of women in academic medicine. Arch Intern Med. 2007;167:343345.
36. Carnes M, Devine PG, Baier Manwell L, et al. The effect of an intervention to break the gender bias habit for faculty at one institution: A cluster randomized, controlled trial. Acad Med. 2015;90:221230.