In its 2011 Strategic Plan, the National Institutes of Health (NIH) identified diversity as “an indispensable component of research training excellence” that “must be advanced across the entire research enterprise.”1 Yet despite significant investment of effort and funding by major academic medical centers, foundations, and federal agencies including the NIH, Department of Health and Human Services, National Science Foundation (NSF), National Aeronautics and Space Administration, and Agency for Healthcare Research and Quality, the number of underrepresented minority (URM) researchers in medicine remains alarmingly low.2,3 Although many URM students start college majoring or intending to major in a science field, they leave science and research in disproportionately high numbers,4 contributing to the oft-mentioned leaky pipeline.2–4 The lack of diversity is especially severe with respect to African Americans, Hispanics, and Native Americans. Despite together constituting over 30% of the U.S. population, African Americans, Hispanics, and Native Americans only accounted for 9% of the PhDs earned in science, technology, engineering, and mathematics fields.2 The picture gets even bleaker continuing down the pipeline. According to a 2017 report from the NSF, of faculty positions in scientific research in 2015, only 4% were held by African American researchers, 4% by Hispanic researchers, and 0.2% by Native American researchers.5
These disappointing numbers point to the importance of finding new and better ways not only to attract more people from URM groups to biomedical research but also to ensure that they flourish while in this field. Productive scientists are made, not born6; thus, as McGee et al2,4 suggest, the focus should be on understanding what is happening within the pipeline rather than simply measuring flow through the pipeline. Yin and colleagues7 have called for the development of new mechanisms to increase the success of women and members of URM groups to “ensure a diverse and vibrant [clinical and translational research] workforce.” This emphasis on talent development calls for new approaches to science training.7
There have been efforts at several institutions to ensure that URM researchers within the pipeline flourish, but they are far from common. A 2012 environmental scan of faculty diversity programs at U.S. medical schools found that less than one-third had programs targeting URM faculty—and only one of these was at a historically black medical school.8 Beech and colleagues’9 2013 systematic review identified mentoring programs for URM faculty at only 13 academic medical centers. Few programs cast a broad net along the length of the pipeline. One exception is the Research Education Program to Increase Diversity in Health Researchers at the College of Human Medicine at Michigan State University, whose trainees include undergraduate, graduate, and medical health professions students and lifelong learners from underrepresented, minority, and diverse backgrounds.10 These programs commonly consist of some combination of standard basics of research training; career development classes or workshops; and mentorship, networking, and/or peer support, and are offered by individual institutions to their own audiences. Although some institutions with clinical and translational science awards (CTSAs) (such as Vanderbilt University and University of California, Los Angeles) have partnered with minority-serving institutions (MSIs) before, most of those partnerships have involved the provision of clinical and translational research degree programs and did not address training needs outside the traditional curriculum.
Our own CTSA institution at the University of Pittsburgh established a partnership with five MSIs (Charles R. Drew University, Howard University, Morehouse University, University of Hawaii Manoa, and Universidad De Puerto Rico Medical Sciences Campus) to create an intensive, one-year online training program called Leading Emerging and Diverse Scientists to Success (LEADS) for junior faculty and postdoctoral trainees (junior investigators) at the partnering institutions. The goal of LEADS is to help early-career scientists at MSIs launch successful biomedical research careers, thus expanding research capacity at their home institutions. However, before designing the program, we wanted to make sure we fully understood the needs and wants of prospective trainees, and to develop our training in close collaboration with, and with ample input from, our MSI partners. To this end, we conducted a needs assessment with junior and senior researchers from each of the partnering MSIs. Our goal was to identify specific areas of training and support that would help junior investigators at MSIs develop and sustain research careers. We present the methodology and results of the needs assessment below.
We began the needs assessment process by forming a leadership team with a senior researcher from each of the participating MSIs. Before conducting the focus groups and interviews, we conducted a series of virtual meetings with the leadership team in which we discussed the skills and abilities junior investigators need to launch successful research careers and identified specific areas of training that were not taught at the participating institutions or that required further reinforcement. A list of competencies began to emerge from these meetings that was consistent with two other needs assessments conducted at MSIs.11,12
To determine whether the competencies we identified aligned with the perceived training needs, motivations, and constraints of potential trainees, we collected data from junior investigators via focus groups and interviews. Moreover, on the assumption that junior investigators “don’t always know what they don’t know,” we also consulted additional senior investigators from the partnering institutions via in-depth interviews.
We conducted focus groups and interviews with 23 junior investigators from four of the five participating MSIs (we were unable to coordinate times with the remaining institution) between March and August 2016. Our intention and preference would have been to conduct focus groups at each institution, guided by the thinking that the social interaction of a focus group would provide a good opportunity for junior scholars to compare and contrast their experiences. However, at one of the participating institutions, we opted for individual interviews instead of a focus group because participants’ schedules proved impossible to coordinate. Additionally, one focus group had too few participants to accurately be described as a focus group and, hence, was technically a group discussion. Although unintended at the outset, the use of multiple data collection methods allowed for us to compare what scholars shared across institutions but also what they said in larger groups, smaller groups, or when interviewed individually.13,14 As can be seen from the wide range of dates during which data were collected, recruitment was difficult, perhaps because of the sensitive nature of the topics to be discussed. Discussions were moderated and interviews conducted by experienced moderators from Qualitative, Evaluation, and Stakeholder Engagement Research Services (Qual EASE) at the University of Pittsburgh (M.H. and research assistants). Both focus groups and interviews were conducted remotely, via video conferencing for the focus groups so that participants could pick up on each other’s cues in the group discussion, and via telephone for the interviews. Participants were postdocs and early-career faculty at MSIs. Our goals were to identify training areas with the highest potential impact, to gauge the interest of potential participants in the program, and to identify a manageable workload. Participants were asked about the importance of research to them personally, their perception of the importance of research at their institution, institutional resources supporting their research, barriers to their launching or sustaining a research career, and their current training needs with respect to research. They were also asked about gaps in their training and how these gaps could be filled. No demographic data were collected, to ensure the anonymity of individual comments. They were then presented with a summary of what we were envisioning for the program, and asked what might encourage them or discourage them from participating in such a program, as well as any specific topics they would like to see covered in the program. For the focus group guide, see Supplemental Digital Appendix 1, available at http://links.lww.com/ACADMED/A644; for the scholars with whom we did interviews, the same guide was used. From these data sources, we collaboratively distilled a set of skill areas that informed the design of the curriculum.
To supplement data from junior investigators, we interviewed six senior investigators from four participating institutions, from March to August 2016, to gather their perspectives on the role of research in the careers of junior investigators at their institution, and to get their feedback on preliminary plans for the structure and content of the program. As with junior investigators, it was difficult to find senior investigators willing to be interviewed, perhaps because of the sensitive nature of the topic. To ensure preservation of anonymity, demographic information was not collected. Senior investigators were asked about the research environment at their institution, barriers to junior investigators launching and sustaining research careers, and the training needs of junior investigators at their institution. For the interview script used with senior investigators, see Supplemental Digital Appendix 2, available at http://links.lww.com/ACADMED/A644. Interviews were conducted telephonically by experienced moderators from Qual EASE.
Focus group discussions and in-depth interviews were transcribed verbatim. The “editing” organizing style was used to inductively develop a codebook representing the range of topics found in the transcripts.15 Two independent coders (M.H. and an experienced coder from Qual EASE) then applied the codebook to all transcripts using the qualitative analysis software ATLAS.ti (Scientific Software Development GmbH, Berlin, Germany), and all differences in coding were adjudicated by the coders to full agreement to ensure consistency in coding. The primary coder (M.H.) then developed themes and subthemes from the data, using a combination of thematic analysis16,17 and the constant comparative method.18 Analytic results were discussed with the broader study team, including the leadership team of senior scholars from partnering MSIs, as a form of analyst triangulation.19
The University of Pittsburgh’s institutional review board deemed that this was not research but, rather, information gathering for program development.
Results from virtual meetings, focus groups, and interviews are discussed below. Study subjects from each institution are summarized in Table 1, where institutions are identified by a letter only in order to preserve anonymity of the participants. When direct quotes are presented, the institution and scholar type that they came from are provided in parentheses. Although we made an effort to quote widely from the different institutions and scholars, some participants were more loquacious than others and, hence, are represented more frequently in direct quotations. Results applied broadly across the institutions.
The virtual meetings we conducted with institutional leaders at the MSIs uncovered a need to teach specific skills not typically taught in a master of science in clinical research or a master of public health degree program. These leaders expressed an interest in teaching the mentees skills that mentees frequently learn from their mentors, such as how to identify an area of research and formulate a hypothesis. They unanimously agreed that students did not need training in areas such as biostatistics that were well represented in the formal curricula; rather, they needed more systematic training in skills that are not formally taught but are essential for successful research careers (e.g., critical and creative thinking, problem formulation, collaboration, communication).
Junior investigators with whom we spoke were overwhelmingly interested in engaging in research. Both senior and junior investigators described widely varying degrees of support and access to resources for research at their institutions. However, regardless of the resources available, they identified a common set of barriers that educational programming could help them address, as well as a core set of features that they would like such programming to have. These barriers and programmatic features clustered around the following themes: informal curriculum, protected time, work product, and networking opportunities.
Both junior and senior investigators described a lack of systematic training in the “informal curriculum,” particularly in topics such as grantsmanship, the successful administration of funded grants, management of a laboratory or research team, time management, developing research questions, and writing papers. This lack of systematic training was linked to greater mentorship needs than the institutions could sometimes support. Junior investigators noted that these issues were frequently related, and expressed interest in a curriculum that might cover these topics holistically. As one junior investigator (institution B) described the link between publishing and successful grantsmanship, “I think the major reason [for us] not getting funded is that we’re not able to tell our story properly [and are] not getting published in good-impact journals that improve your grants.” Similarly, a junior investigator (institution C) highlighted the relationship between administering grants and launching a research career and expressed a desire to have more training in financial management of research.
Senior investigators additionally identified the need for training in what might be termed the “politics of science.” One senior investigator (institution A) described the conundrum in which “it’s becoming more and more difficult to secure NIH funding,” and yet “our students don’t have those opportunities, to develop those … networks and partnerships” that allow you to “[get] your name out there” and have a better chance at getting a foothold.
Additionally, senior investigators noted that mentoring within their institutions is a challenge, owing to the dearth of mentors who can teach young investigators the skills and knowledge that constitute the informal curriculum. Moreover, those mentors who are available, particularly those from underrepresented backgrounds, face a greater demand for their mentorship than they can meet. One such senior investigator (institution A) described how “it’s less than a handful of people that [are] carrying the load to really provide mentorship and training to a number of minority investigators interested in health disparities.” The dearth of mentors creates a heavy mentorship load for senior investigators while also depriving junior investigators of adequate mentorship.
Both junior and senior investigators described their institutions as supportive of research in spirit but noted that, for a variety of reasons, their institutions did not provide the protected time that junior investigators need to launch a research career. One junior investigator (institution A) described the difficulty of juggling so many competing demands—working with residents and medical students, as they are encouraged to do as a faculty member, while at the same time “seeing patients … and … trying to find the time to actually do research.” Other institutions were described as facing severe financial shortfalls and lacking resources to provide even minimal research support, as in the case of a junior faculty member who had to purchase her own computer to use in her research. Similarly, whereas some institutions provided programmatic support for grant writing through formal channels for proposal critique, others had no such supports in place. Without resources, devoted research time, and grant writing support, junior scholars felt that they were fighting an uphill battle to launch their research careers.
This combination of the lack of institutional support and the increasing challenge of securing research funding was described as a vicious cycle, in which failure to secure grants decreased the likelihood that the institution would invest the resources to ensure that faculty had time and administrative support to write a successful grant, which further decreased the likelihood of success. In addition to their concern about the lack of protected time to launch their research careers, junior investigators were uncertain whether they would be able to devote the time needed to participate in additional training—even just eight hours per week. Whereas some felt that they could accommodate such time demands, others thought that it would not be feasible without protected time from their institution. Additionally, they felt that the time demands would be more manageable if scheduled time could be limited and they could complete as much of the training on their own time as possible.
Some junior investigators voiced the opinion that the program would be of maximal impact if, as one individual (institution D) explained, there were “a clear outcome at the end … a finalized proposal that is ready, [and] has been revised by various individuals.” Junior investigators were not interested in a program that would focus solely on abstract principles; rather, they were only interested in investing their time if, upon completion of the program, they had generated a product that would advance their research.
Several of the senior investigators with whom we spoke felt that networking was key to launching a successful research career. One senior investigator (institution A) stated that “in science, you are as good as your network” and noted that creating opportunities for networking and collaboration was vital to improving their students’ and mentees’ chances of securing research funding. Another senior investigator noted that the better networked scholars are, the more likely other scholars are to be aware of and cite their work. In one individual’s view (institution A), the increased awareness of a scholar’s research that could come from networking might also lead to better chances of securing funding, in that the more they can “generalize and transfer what we’re learning here out to other communities across the U.S., the more favorable our applications will look to reviewers.” The potential for networking, particularly multi-institutional, was a facet of the program that the senior investigators thought held great potential. This would enable the scholars to tap into expertise that their home institution may not have.
The focus group and interview data we collected in the needs assessment were pivotal to informing key aspects of the LEADS program. We structured LEADS to include topics derived from the informal curriculum (see Table 2), which were offered online to make the training available to participating scholars in different geographical regions and time zones and to provide flexibility in where and when they could complete the work. We required protected time for participation and structured the work to focus on deliverables that would help build LEADS scholars’ research careers. We decided on a one-year program because, as our partners from the MSIs noted, it would be difficult for the junior investigators to get protected time to engage in the program beyond one year. However, to ensure success of the program, after completion LEADS scholars continue to complete an individualized development plan and an annual report noting their progress.
We addressed the four themes identified in the needs assessment in the following ways.
Modules that target key skills in the “informal” curriculum
Historically, in academic medicine, budding researchers are most often trained using a variant of an apprenticeship model—the famous “see one, do one, teach one.” Success when using this model is heavily dependent on the research experience, availability, and willingness of individual mentors, as well as research opportunities that may or may not be available at a given institution. When the expertise does not exist at a particular institution, this model fails the mentee. Also, this model does not ensure that the skills that fall within the informal curriculum are taught in a systematic, consistent, or efficient way—a deficit identified by the junior and senior investigators. Therefore, we decided to include modules (listed in Table 2) that would provide formal instruction in some of the elements of the informal curriculum. Examples include a module that guides scholars through thinking creatively and critically to help them approach their research in innovative ways, and another module in which scholars learn to identify a significant research problem and a theoretical and conceptual model. In the Asking the Right Question module, scholars take the problem they identified in earlier modules and formulate it into testable hypotheses. The Launching Your Research Career module focuses on personal and professional development by helping scholars develop a strategic plan for their careers, and covers topics such as mentoring, time management, and negotiation.
Departments that guarantee protected time
As one of our focus group participants said, “Research takes time … freedom to think, and peace to think.” To ensure that our LEADS scholars would have the necessary freedom and peace to think, we required LEADS applicants to include a letter of support from their department chair in their application packet. In this letter, the department chair committed to providing 20% protected time for the applicant during the program. This equates to approximately 8 to 10 hours per week—the time we anticipated was needed to fully participate in the program, complete the online modules, and generate meaningful products.
Module assignments that yield concrete work products
Junior investigators told us they wanted to complete the training with a practical work product, such as a draft of a grant or manuscript. To this end, we included Grant Writing and Medical Writing modules to reinforce these high-value skills through additional practice and feedback. Moreover, we designed assignments to focus on specific deliverables that built over a series of modules into a significant work product. LEADS scholars begin to identify and develop a significant research problem in the Identifying the Problem module, articulate a research question and hypothesis in the module Asking the Right Question, write specific aims and draft an innovation and approach section in the Grant Writing module, and craft abstracts and outline the sections of a manuscript in the module Medical Writing. Our goal was to ensure that the work that scholars did in their LEADS modules provided a structure and incentive to bring a grant proposal and manuscript closer to fruition.
Asynchronous and synchronous activities that encourage networking across institutions
The senior investigators we interviewed believed that opportunities to connect with a national network of scientists would increase the program’s impact and improve junior investigators’ funding prospects. We took this feedback to heart when designing LEADS and included opportunities for LEADS scholars to interact and network with other participating scholars from the partnering MSIs as well as with senior researchers from the University of Pittsburgh and other institutions. We did this in part by recruiting a diverse group of successful biomedical researchers to design and teach LEADS modules, which immediately expanded the network available to LEADS scholars and gave them the opportunity to hear strategies, advice, and stories from a range of experienced senior investigators. To give scholars the opportunity to interact with peers across institutions, we incorporated weekly, two-hour synchronous sessions and asynchronous discussions, where scholars could discuss problems, ask questions, share concerns, and develop strategies within the group. Our goal was to foster stronger professional networks by encouraging as much interaction as possible among researchers at the participating institutions.
The goal of LEADS is to provide junior investigators at MSIs with the resources and training they need to flourish in biomedical research. Following the suggestion of McGee and colleagues,2,4 we focused on what is happening within the pipeline rather than simply measuring flow through the pipeline. LEADS provides a new approach to developing the URM talent that already exists within the pipeline by teaching an “informal curriculum” identified in part through our needs assessment. In addition, our goal is that LEADS will help to redress the “leaky pipeline” problem by providing participants with systematic training, protected time, networking opportunities, and the creation of a concrete work product.
The MSI leadership’s conclusions dovetailed in large part with a needs assessment conducted by Estapé-Garrastazu and colleagues11 with faculty and students at the three minority medical and health science institutions constituting the Puerto Rico Clinical and Translational Research Consortium. This study identified a strong desire to develop better translational teamwork and communication skills among associate and assistant professors. A second, informal needs assessment conducted at Howard University’s Department of Medicine12 identified the need for training in areas such as research design, grant writing, research administration and leadership, data management and analysis, and disseminating research findings. These needs assessments, in combination with our interviews and focus groups, helped us craft an initial set of topic areas to address in our online training.
At the same time, it is important to note that LEADS does not address all the impediments to a successful research career that were revealed by the needs assessment; for example, junior investigators sometimes cited financial shortfalls and lack of institutional resources, which suggests the need for interventions that go beyond training. Moreover, both senior and junior investigators whom we interviewed indicated the need for more mentors and more formal recognition of mentors. We constrained the LEADS program to mentee development, leaving the task of mentor development to the National Research Mentoring Network, which is charged with developing and training mentors committed to mentoring people from diverse backgrounds. However, we recognize that more consideration is needed as to how to improve the mentorship for junior investigators at MSIs.
Our conclusions were drawn from conversations, interviews, and focus groups with stakeholders at five MSIs. Although we feel that these compose a representative sample, it is possible that leaders and senior and junior investigators from other MSIs would identify different concerns. Also, we only conducted a limited number of focus groups and interviews, so it is possible that the data are not sufficient to inform the development of the program. However, we did reach thematic saturation with the interviews and focus groups, which gives us confidence that we identified the most pressing areas where junior investigators need training.
Conventional training is not, by itself, sufficient to overcome the lack of diversity in the biomedical workforce, even though it is an important component. For a diversity-focused program to be effective, it must be grounded solidly in an understanding of the training deficits identified or experienced by junior investigators. The needs assessment described here generated insights that helped us design a program specifically targeted to overcome identified deficits, develop talented individuals already in the pipeline, retain them in biomedical research careers, and improve the diversity of the clinical and translational research workforce.
The authors acknowledge Ethan Lennox, MA, for his assistance in editing this manuscript.
1. National Institutes of Health. Investing in the future: National Institute of General Medical Sciences strategic plan for biomedical and behavioral research training 2011. https://publications.nigms.nih.gov/trainingstrategicplan/theme4.htm
. Published 2011. Accessed January 18, 2019.
2. McGee R Jr, Saran S, Krulwich TA. Diversity in the biomedical research workforce: Developing talent. Mt Sinai J Med. 2012;79:397–411.
3. Milewicz DM, Lorenz RG, Dermody TS, Brass LF; National Association of MD–PhD Programs Executive Committee. Rescuing the physician–scientist workforce: The time for action is now. J Clin Invest. 2015;125:3742–3747.
4. McGee R. Biomedical workforce diversity: The context for mentoring to develop talents and foster success within the “pipeline.” AIDS Behav. 2016;20(suppl 2):231–237.
5. National Science Foundation, National Center for Science and Engineering Statistics. Women, minorities, and persons with disabilities in science and engineering. Special report NSF 17-310. www.nsf.gov/statistics/wmpd
. Published January 2017. Accessed January 18, 2019.
6. Cox WM, Alm R. Scientists are made, not born. N Y Times. February 28, 2005:A25.
7. Yin HL, Gabrilove J, Jackson R, Sweeney C, Fair AM, Toto R; Clinical and Translational Science Award “Mentored to Independent Investigator” Working Group Committee. Sustaining the clinical and translational research workforce: Training and empowering the next generation of investigators. Acad Med. 2015;90:861–865.
8. Adanga E, Avakame E, Carthon MB, Guevara JP. An environmental scan of faculty diversity programs at U.S. medical schools. Acad Med. 2012;87:1540–1547.
9. Beech BM, Calles-Escandon J, Hairston KG, Langdon SE, Latham-Sadler BA, Bell RA. Mentoring programs for underrepresented minority faculty in academic medical centers: A systematic review of the literature. Acad Med. 2013;88:541–549.
10. Crockett ET. A research education program model to prepare a highly qualified workforce in biomedical and health-related research and increase diversity. BMC Med Educ. 2014;14:202.
11. Estapé-Garrastazu ES, Noboa-Ramos C, De Jesús-Ojeda L, De Pedro-Serbiá Z, Acosta-Pérez E, Camacho-Feliciano DM. Clinical and translational research capacity building needs in minority medical and health science Hispanic institutions. Clin Transl Sci. 2014;7:406–412.
12. Rubio DM. Personal e-mail communication with S.M. Nouraie, March 2015.Associate vice provost, University of Pittsburgh.
13. Carter N, Bryant-Lukosius D, DiCenso A, Blythe J, Neville AJ. The use of triangulation in qualitative research. Oncol Nurs Forum. 2014;41:545–547.
14. Morse JM. Mixing qualitative methods. Qual Health Res. 2009;19:1523–1524.
15. Crabtree BF, Miller WL. Doing Qualitative Research. 1999.2nd ed. Thousand Oaks, CA: Sage Publications.
16. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.
17. Guest G, MacQueen KM, Namey EE. Applied Thematic Analysis. 2012.Thousand Oaks, CA: Sage Publications.
18. Glaser BG. The constant comparative method of qualitative analysis. Soc Probl. 1965;12:436–445.
19. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999;34(5 pt 2):1189–1208.