Secondary Logo

Journal Logo

Innovation Reports

Multidisciplinary Mentoring Programs to Enhance Junior Faculty Research Grant Success

Freel, Stephanie A. PhD, PMP; Smith, Paige C. MEd; Burns, Ebony N. MHA; Downer, Joanna B. PhD; Brown, Ann J. MD, MHS; Dewhirst, Mark W. DVM, PhD

Author Information
doi: 10.1097/ACM.0000000000001620

Abstract

Problem

Junior faculty are beginning their careers in research environments significantly different from those of their mentors’ early careers. Over the past 30 years, there have been major enterprise-level changes, including the declining availability of research funding and an enhanced focus on multidisciplinary science.1,2 Yet despite these changes, the parameters for success—independent research funding and publication—have largely remained the same. This dichotomy imposes new challenges on traditional dyadic mentoring structures.3 To improve junior faculty success, institutions must address these enterprise-level changes by incorporating mentoring strategies that actively encourage multidisciplinary collaborations and provide guidance on obtaining research funding in that context.

As outlined by Alberts et al1 in 2014, research funding challenges have a significant negative impact on junior investigators, who are 4 to 6 years older when they receive their first independent National Institutes of Health (NIH) grant (R series) compared with investigators 30 years ago. In that same time span, the success rate for R01-equivalent grants has dropped nearly 15 percentage points to a 17% nadir in 2013 (13% for new investigators).4 The transition to independence therefore remains a challenge, especially for those in the clinical and translational sciences.5

Coinciding with this funding decline is a fundamental shift in how research is conducted, requiring increasingly larger teams of scientists.2 These challenges are intertwined, as research funds often are earmarked for collaborative science initiatives: As of 2010, nearly half of NIH and National Science Foundation funding opportunities called for collaborative approaches.6 Academic institutions are recognizing that retention of junior investigators in this new environment requires multidisciplinary mentorship.3,5,7

At Duke University School of Medicine, to address low NIH grant success rates in 2010 for junior faculty—11% (21 awards/184 applications) overall and 9% (10/107) in clinical departments—the Office for Faculty Mentoring (OFM) launched two grant-writing programs: the Path to Independence Program (PtIP) for NIH R applicants in 2011 and the K Club for NIH K (career development) applicants in 2012. As part of the Clinical and Translational Science Award (CTSA) consortium, we engaged with a number of similarly minded institutions, including the University of Utah, which recently described a successful two-year mentorship model8 based on many of the elements that we also felt were essential for our programs. These elements include real-time grant development support, multidisciplinary mentorship, and coordinated access to institutional support offices. Our innovative programs go beyond this model by supporting a wide array of investigators with multimodal learning experiences in an efficient four-month program. In this report, we describe our programs and preliminary outcomes.

Approach

Program design

The PtIP and K Club were designed to achieve three goals: (1) increase grant success rates of junior faculty, (2) establish a sustainable multidisciplinary mentorship model, and (3) improve feelings of satisfaction and support. Program development involved a process of iterative refinement, starting with acknowledging the need for idea conceptualization (i.e., NIH grant application’s Specific Aims page) and adding/revising elements based on participant and mentor feedback and recorded observations.

Funded by the Duke University School of Medicine and CTSA, the PtIP and K Club are offered at no cost to participants or their departments. The four-month programs are offered three times per year, coinciding with NIH submission cycles. (For the annual program cycle, see Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/A432.)

Participants develop full grant applications during each four-month program. Each application is reviewed by at least seven senior faculty members and two professional grant writers. Projects reviewed reflect the full spectrum of research typically funded by the NIH.

Program structure

The Duke OFM developed a conjoined four-month, 20-hour curriculum for the two programs that incorporates didactic lectures, mentored workshops, consultations, and structured review (Figure 1). This curriculum allows participants to operate within three learning domains (cognitive, affective, and behavioral), maximizing the likelihood that effective strategies will be recognized, heard, and employed. New cohorts of the PtIP and K Club move together through the one curriculum, with an additional career development session for K Club participants. Previous K Club participants are permitted to reenroll as PtIP participants in later cycles, which allows PtIP participants who have held K awards to serve as peer mentors.

F1
Figure 1:
Structure of the conjoined four-month grant-writing programs for junior faculty at Duke University. The Path to Independence Program (PtIP) and K Club run concurrently three times per year. Program-staff-coordinated events occur between the orientation session and the internal review of NIH grant applications prior to submission. For each program component, the number of participants per session and the number of sessions or small groups held in each four-month cycle are shown. All program components are offered for both PtIP and K Club participants, except the individual career development sessions (K Club only). Office for Faculty Mentoring (OFM)-led sessions are indicated as circles; OFM-coordinated sessions led by partnering offices are indicated by hexagons. Each participant is expected to attend one workshop of each type listed. Throughout the program, participants are expected to work closely with their personal research mentors.

Enrollment and orientation.

We limit combined enrollment to 24 participants per cycle. To ensure that each is prepared to participate effectively, we require applicants to submit an enrollment packet with a project abstract, specific aims, and a memorandum of understanding signed by themselves, their department chair or division chief, and, for K Club applicants, their primary research mentor. Program staff select participants based on time of submission and completeness of enrollment materials. Scientific validity is not considered during selection because it is a focus of program activities. The few applicants who are not accepted are deferred to a later cycle.

Each cycle begins with a two-hour orientation lecture on grant-writing strategies (M.W.D), common pitfalls, and key resources including Regulatory Affairs and the Office for Clinical Research.

Specific Aims and Significance and Innovation workshops.

Recognizing that the scientific reviewers of NIH grant applications place great importance on three critical areas—Specific Aims, Significance, and Innovation—we designed two-hour workshops during which participants refine these sections of their applications. The first workshop, offered during month one, focuses on the Specific Aims page; the second occurs during month two and covers both the Significance and Innovation sections. Each small-group workshop (two to three participants) is mentored by three to four senior faculty mentors. To ensure fresh perspectives, these workshop mentors are from a wide array of research disciplines and are independent of any participant’s specific research mentor.

Each workshop begins with a brief introductory lecture, given by a professional grant writer (S.A.F.) discussing the NIH criteria on which the grant section is evaluated and specific strategies for addressing these criteria. Afterward, participants’ Specific Aims pages, or their Significance and Innovation sections, are projected, read aloud, and critiqued as a group (30 minutes each). These workshop reviews focus on the quality of the scientific proposal as well as effective delivery of the content.

Effective communication workshop.

This workshop, given at the end of month one and beginning of month two, is designed and led by a professional grant writer who serves as the associate dean for research development (J.B.D.) and emphasizes meeting readers’ expectations. Prior to the workshop, participants review a series of five video modules that convey strategies for constructing compelling text. During the hands-on small-group workshop, each of the approximately five participants is guided in applying these writing and decision-making strategies to their own Specific Aims page and is provided tools to effectively address the feedback they receive from program mentors as well as their primary research mentor.

Peer groups.

In 2013, we added a peer-mentoring component to promote cooperative development of the grant application elements not covered during workshops. Peer groups of five to eight participants meet for an hour seven or eight times during their program cycle. During the initial meeting, each peer group develops a mission statement and agenda through program-staff-facilitated brainstorming and nominal group techniques. Example grant documents are provided as requested. To enhance accountability, each group divides five roles among its members: facilitator, minute taker, timekeeper, group organizer, and liaison to program staff.

Career development sessions.

Each K Club participant meets individually with a senior faculty mentor or professional grant writer to discuss professional development aspects of their grant application. Participants select from a pool of three to four mentors who have experience in K grant applications and have volunteered to serve in this role. Participants are also counseled on the development and coordination of their mentorship team. After the initial one-hour consultation, follow-up advice may be provided via e-mail or a subsequent meeting if necessary.

Regulatory and operational review.

Participants conducting clinical research may have their grant applications reviewed by experts from partnering offices, including Regulatory Affairs, Biostatistics and Bioinformatics, and the Office for Clinical Research. These individual sessions provide feedback to the participant on operational aspects of the grant, including licensing, protection of human subjects, data safety and monitoring plans, clinical study design, and data analysis.

Internal review.

Participants are invited to submit drafts of their completed grant application six to eight weeks prior to the NIH deadline. Applications are reviewed by two to three senior faculty whose expertise matches at least one critical scientific area. These reviewers provide feedback both in an NIH review template and in person. In 2014, we established a standing study section of senior faculty reviewers. During a four-hour study section simulation, participants view discussions of their applications via live video streaming. Afterward, each participant meets with their primary reviewers for one hour to discuss recommendations and next steps.

Program evaluation

Under a Duke University Health System institutional review board exemption, evaluation data were collected in March 2015. All prior participants (n = 197 participants, October 2011 to March 2015) were invited to participate in a postprogram survey. Data were collected using REDCap (Research Electronic Data Capture) tools hosted at Duke University.9 This postprogram survey addressed feelings of support, satisfaction, and perceived changes in skill level. We analyzed changes in retro preprogram measures (i.e., participants were asked to reflect back to before they began the program to rate their satisfaction/competency at that time) and postprogram measures using the Mann–Whitney U test in R v.3.1.3: A language and environment for statistical computing (R Core Team, R Foundation for Statistical Computing, Vienna, Austria; 2015.)

Demographic data for program participants, as stored in the Duke faculty employee database, were provided to the authors by the Duke Office for Faculty in March 2016.

We evaluated grant success by mining the Duke University Sponsored Projects Systems database in March 2016 to determine the cumulative number and types of grant applications submitted by each program cohort since that cohort enrolled in the program (October 2011 [fiscal year 2012] through March 2016 [fiscal year 2016]). Submissions were stratified by program (PtIP, K Club), sponsor type (NIH, non-NIH government, commercial, foundation/nonprofit), and award status (awarded, unfunded, pending). The success rate for each category was determined as:

Outcomes

Mentors and participants

From October 2011 to March 2016, we recruited 265 senior faculty mentors. Over the same period, the K Club and PtIP served 138 and 145 participants, respectively, including both MD and PhD scientists. These participants included 268 unique individuals; 15 individuals participated in both programs. Both mentors and program participants came from all Duke University School of Medicine departments as well as the Duke University School of Nursing, Pratt School of Engineering, Trinity College of Arts & Sciences, and Nicolas School of the Environment.

The majority of the participants were assistant professors or medical instructors (73%; 196/268), followed by postdocs and fellows (19%; 51/268), and associate professors (7%; 20/268). Available demographics demonstrated that 53% (122/231) of the participants were male, 68% (157/231) were Caucasian, and 11% (26/231) were from underrepresented minority groups. (Grant outcomes by participant demographics are shown in Supplemental Digital Appendix 2 at https://links.lww.com/ACADMED/A432.)

Satisfaction, support, and competency

While the 2015 postprogram survey response rate was low (36%; n = 70/197; 35 PtIP, 35 K Club), respondents and nonrespondents were similar in baseline characteristics (see Supplemental Digital Appendix 3 at https://links.lww.com/ACADMED/A432). Respondents from each program found all program and related components (workshops; peer groups; mentors; written review; study section feedback; career development sessions; and OFM office hours) to be effective, with median scores of 4 (effective) or 5 (very effective) on a five-point scale. They also felt significantly more supported following participation. On a scale of 1 (not at all supported) to 10 (extremely supported), median postprogram ratings of feelings of support ranged from 7 to 8, an increase of 1 or 2 points over retro preprogram ratings in each of the three areas measured (P < .001): obtaining research funding, establishing an independent research program, and professional development.

Respondents rated their competency in six skills critical for grant success (Figure 2). On a scale of 1 (not at all competent) to 10 (extremely competent), they expressed large, and statistically significant, increases (2–4 points) in competency in all areas following program participation. The largest increases were in communicating the significance of research plans and designing and communicating career development plans. When asked to indicate factors that contributed to their feelings of increased competency, 96% (67/70) attributed the change to the OFM grant-writing programs. (Additional contributing factors are shown in Supplemental Digital Appendix 4 at https://links.lww.com/ACADMED/A432.)

F2
Figure 2:
Enhanced grant-writing skills across six competencies, as reported by participants in the Path to Independence Program (PtIP) and K Club, Duke University’s conjoined four-month grant-writing programs for junior faculty. On a 2015 postprogram survey, PtIP and K Club participants were asked to rate their feelings of competency in six core areas following completion of the program (after) and to reflect back on their competency level before participation (before). Ratings used an ordinal scale from 1 (not at all skilled) to 10 (extremely skilled). The median of individual changes (pre-to-post) are indicated on each panel. P values were determined by Mann–Whitney U test. Change in individual scores ranged from 0 to +8 points, and no scores declined. All comparisons were significant, with P values < .001 as follows. Panel A: Ratings of research plan competencies, PtIP and K Club respondents (n = 70; 35 PtIP, 35 K Club): left, P = 7.23–11; middle, P = 4.98–13; right, P = 1.26–12. Panel B: Ratings of career development competencies, K Club respondents only (n = 35): left, P = 2.32–7; middle, P = 2.17–7; right, P = 5.43–6. The data for each respondent are shown as symbols in scatter plots. The box and whisker plot around the data depict the median/interquartile range; the error bars depict the 95% confidence limits of the data.

Grant success

Between the time of their program enrollment and March 2016, 94 PtIP participants submitted a total of 325 grant applications to the NIH with a success rate of 28% (61 awards/220 decisions; 105 pending; fellowships and subcontracts not included; Figure 3). These successful applications represented several research program grant (RPG) mechanisms, including 24 R01 awards. Similarly, 72 K Club participants submitted 93 grant applications with a success rate of 64% (38 awards/59 decisions; 34 pending). Additional details are shown in Supplemental Digital Appendix 2 at https://links.lww.com/ACADMED/A432.

F3
Figure 3:
Grant success rates for the Path to Independence Program (PtIP) and K Club, Duke University, October 2011 (FY 2012)–March 2016 (FY 2016). Numbers of awarded grants, unfunded grants, and applications still pending are shown for the Path to Independence Program participants (panel A) and the K Club participants (panel B). Listed above the bars for each year are program success rates for that fiscal year. Below that, are NIH R01-equivalent success rates for new investigators (panel A) or success rates for NIH K-equivalent mechanisms (panel B). (The NIH success rates were obtained from https://report.nih.gov/success_rates.) Abbreviation: FY indicates fiscal year. Duke FY = July 1–June 30; NIH FY = October 1–September 30.

Non-NIH external grants were awarded to 29 K Club and 42 PtIP participants. Overall return on investment is shown in Supplemental Digital Appendix 5 at https://links.lww.com/ACADMED/A432.

Next Steps

In fewer than five years, we have improved RPG success rates for program participants by nearly 17 percentage points over the 2010 baseline for junior faculty at Duke University School of Medicine. In addition, participants have reported feeling better supported in their research and their careers, as well as being more competent in critical grant-writing skills. These factors have been shown to be key to faculty job satisfaction and retention.10

We have identified three important next steps to improve the impact of our mentorship model: (1) add elements on research methodology, (2) develop infrastructure to enhance equity and inclusion, and (3) promote implementation of our model across academic research institutions.

A continuing concern is the lack of closely matched research expertise between program mentors and participants, specifically as it affects writing the Scientific Approach section. Additional programmatic elements to address this need—and their sustainability—are being considered.

As is common in biomedical research, representation of diverse racial, ethnic, and cultural groups needs improvement in our programs. In addition to continuing our efforts to recruit a diverse pool of program mentors, we have engaged with the Office of Equity and Inclusion to explore options for encouraging and supporting a diverse participant population. We have also developed a pipeline through which KL2 scholars can enter directly into these programs to promote the next phase of their careers.

Broad implementation of similar programs elsewhere could bolster success, satisfaction, and retention of junior faculty investigators. In 2015, we engaged with the CTSA of nearby University of North Carolina at Chapel Hill to develop a collaborative mentorship program for junior faculty investigators that could be implemented across CTSA hubs. Through this and continuing efforts with Duke’s research-intensive schools, we are making strides toward providing effective support for junior faculty investigators across a spectrum of research foci.

Acknowledgments:

The authors acknowledge Nancy Andrews, dean, Duke University School of Medicine, for her support; Dede Crosmer for providing demographic information; Ceci Chamorro for building REDCap registration and survey databases; Carolyn Eckhardt for designing a program-specific SPS ART query form; Rebecca Brouwer and Dr. Steven Grambow for developing individual operational counseling sessions; Dr. Erika Segear Johnson and Dr. Amanda Parrish for developing individual regulatory counseling sessions; and all the faculty mentors who contributed time to support their junior colleagues.

References

1. Alberts B, Kirschner MW, Tilghman S, Varmus H. Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci U S A. 2014;111:57735777.
2. Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316:10361039.
3. DeCastro R, Sambuco D, Ubel PA, Stewart A, Jagsi R. Mentor networks in academic medicine: Moving beyond a dyadic conception of mentoring for junior faculty researchers. Acad Med. 2013;88:488496.
4. National Institutes of Health. NIH data book. 2015. https://report.nih.gov/nihdatabook/index.aspx. Accessed August 27, 2016.
5. Yin HL, Gabrilove J, Jackson R, et al. Sustaining the clinical and translational research workforce: Training and empowering the next generation of investigators. Acad Med. 2015;90(7):861865.
6. Begg MD, Vaughan RD. Are biostatistics students prepared to succeed in the era of interdisciplinary science? (And how will we know?). Am Stat. 2011;65(2):7179.
7. Ameredes BT, Hellmich MR, Cestone CM, et al. The multidisciplinary translational team (MTT) model for training and development of translational research investigators. Clin Transl Sci. 2015;8:533541.
8. Byington CL, Keenan H, Phillips JD, et al. A matrix mentoring model that effectively supports clinical and translational scientists and increases inclusion in biomedical research: Lessons from the University of Utah. Acad Med. 2016;91:497502.
9. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
10. Pololi LH, Krupat E, Civian JT, Ash AS, Brennan RT. Why are a quarter of faculty considering leaving academic medicine? A study of their perceptions of institutional culture and intentions to leave at 26 representative U.S. medical schools. Acad Med. 2012;87:859869.

Supplemental Digital Content

Copyright © 2017 by the Association of American Medical Colleges