Outcomes From the NIH Clinical Research Training Program: A Mentored Research Experience to Enhance Career Development of Clinician–Scientists

Abstract

Purpose 

Clinician–scientists are considered an endangered species for many reasons, including challenges with establishing and maintaining a career pipeline. Career outcomes from yearlong medical and dental students’ research enrichment programs have not been well determined. Therefore, the authors assessed career and research outcome data from a cohort of participants in the National Institutes of Health (NIH) Clinical Research Training Program (CRTP).

Method 

The CRTP provided a yearlong mentored clinical or translational research opportunity for 340 medical and dental students. Of these, 135 completed their training, including fellowships, from 1997 to January 2014. Data for 130 of 135 were analyzed: time conducting research, types of public funding (NIH grants), and publications from self-reported surveys that were verified via the NIH Research Portfolio Online Reporting Tools Web site and PubMed.

Results 

Nearly two-thirds (84 of 130) indicated that they were conducting research, and over half of the 84 (approximately one-third of the total cohort) spent more than 25% of time conducting research. Of those 84, over 25% received grant support from the NIH, and those further in their careers published more scholarly manuscripts.

Conclusions 

Data suggest that the CRTP helped foster the careers of research-oriented medical and dental students as measured by time conducting research, successful competition for federal funding, and the publication of their research. Longer follow-up is warranted to assess the impact of these mentored research experiences. Investments in mentored research programs for health professional students are invaluable to support the dwindling pipeline of biomedical researchers and clinician–scientists.

Physicians and dentists as clinician–scientists are critical to the vitality of the biomedical research enterprise in the United States and around the world. Since the 1970s, researchers, educators, and others have addressed the shrinking numbers of physicians engaged in clinical research careers.^1–4 Physicians participating in clinical research were part of the rapid growth in biomedical basic sciences in the last quarter of the 20th century. Physician–scientists have made critical contributions to basic studies that led to Nobel Prizes in physiology or medicine and chemistry. The concern has been raised that the decline in the number of physicians engaged in clinical research was possibly related to increased competition for resources, and data and perceptions suggested that academic medical centers and National Institutes of Health (NIH) study sections seemed to favor research conducted by laboratory-based scientists with PhDs.^2,5

In 1995, an NIH Director’s Panel on Clinical Research was convened by Harold Varmus, MD, with David G. Nathan, MD, as chair, to “make recommendations that might guide the NIH toward policy changes that could alleviate the distress in the clinical research community.”⁶ One panel outcome was to broaden the definition of clinical research to include patient-oriented research, epidemiologic and behavioral studies, outcomes research, and health services research. Recommendations were made related to federal grants funding for clinical research, including special emphasis study sections to review clinical research grant applications; didactic clinical research training grants for support of fellows and junior faculty in academic medical centers; grants for early career clinical investigators and their mentors; and enhancement of the budgets for General Clinical Research Centers.^5–7 Another recommendation stated that the NIH should initiate an intramural training program to enhance the attractiveness of careers in clinical research to medical students, analogous to the Howard Hughes Medical Institute (HHMI)-NIH Research Scholars Program.⁶ This led to the establishment of the NIH Clinical Research Training Program (CRTP) in 1997. The panel recommended that this program be conducted in the NIH Clinical Center as an intensive preceptorship that would include formal clinical research training. The CRTP flourished for 15 years as a public–private partnership with support from the NIH as well as grant support from a private funder (Pfizer Inc.) to the Foundation for the National Institutes of Health. Outcome data from the 15-year history of the NIH CRTP have not been reported. However, early data from the HHMI-NIH Research Scholars Program were reported in 2003⁸ and form a basis for some comparison. To our knowledge, those are the only comparable outcome data available from the funded programs that have encouraged medical and dental students to pursue biomedical research.

Presentation of these CRTP data is timely. Two yearlong research programs similar to the NIH CRTP have been discontinued: the HHMI-NIH Research Scholars Program and the Doris Duke Charitable Foundation Clinical Research Fellowship Program. When the HHMI-NIH Research Scholars Program was dissolved in 2012, the NIH leadership decided to transform the CRTP into a new entity for student trainees, the NIH Medical Research Scholars Program (MRSP). This amalgamated program combined features of the HHMI-NIH Research Scholars Program, initially targeted to laboratory research training, and the NIH CRTP. The MRSP provides yearlong mentored research experiences, ranging from basic sciences to translational and more clinically oriented research, for outstanding medical, dental, and veterinary students. The NIH MRSP had its first class of 45 students in 2012–2103.

Comprehensive outcome data for many of the current and former yearlong research programs for medical and dental students are not available. The aforementioned programs varied somewhat in research opportunities, experiences, and instructional methodologies. However, they all had the overarching goal of harnessing early interest in research and then sustaining it as clinician–scientists progress in their careers. Our study assesses career progression and level of research commitment based on self-reporting of professional time allocated to research, as well as research funding and productivity for individuals who participated in the early years of the NIH CRTP and have fully completed their formal training, including professional school, a primary residency, and fellowships.

Method

Background

When the CRTP was established in 1997 with NIH intramural program support, the class size was 9. In 1998, Pfizer Inc. became a private partner and supported the CRTP with contributions to the Foundation for the National Institutes of Health, allowing the annual class size to grow to 15 students. From 2004 until 2012, the CRTP received additional support from the NIH Roadmap (Common Fund), which enabled growth to 30 students. For all CRTP classes, applications were submitted electronically and included a personal statement, curriculum vita, three letters of recommendation, and university transcripts. A subset of applicants was interviewed, students were ranked, and slots filled. From its inception in 1997 until 2012, the CRTP received nearly 1,300 applications and enrolled 340 participants.

Data analysis

We collated data from the academic files of the 340 CRTP participants in 2014. These included baseline demographic characteristics (name, sex, undergraduate degree, academic year in CRTP, and professional school), which were entered into a comprehensive database. As a next step, an in-depth, institutionally approved survey was distributed to all alumni starting in 2012, and surveys have continued annually. The survey asked for residency type and institution; fellowship type and institution; whether an advanced degree was obtained; current faculty position if applicable; publications through December 31, 2013; research funding from public and private sources; and a self-report of participants’ professional time conducting research. Because data were desired to have a quantifiable metric for comparisons, queries about research time required one of four self-reported responses: none; less than 25%; 25%–50%; and over 50%. All participants identified their primary residency from the following categories: anesthesiology, dermatology, emergency medicine, internal medicine, internal medicine and pediatrics, neurology, obstetrics–gynecology, ophthalmology, pathology, pediatrics, physical medicine and rehabilitation, psychiatry, radiation oncology, radiology, and surgery (general, neurosurgical, oral and maxillofacial/dental, orthopedic, otolaryngology, plastic, and urology).

Data provided in the database as well as in the follow-up survey were collated and tallied. Undergraduate degrees were simply counted and collated on the basis of information in the demographic database obtained at time of participation in the program. From the list of postgraduate primary residencies provided in the follow-up surveys, responses were simply counted and quantified on the basis of the aforementioned categories. The survey asked for one of three self-reported professional designations of either faculty position (including clinical), clinical practice, or industry. Responses were tallied.

To supplement survey responses and obtain as much information as possible, we requested curricula vitae from all 340 program participants for corroborative information about career progression, grant support, and publications. Sources of both private and public research support as well as lists of publications were provided in curricula vitae. For this analysis, “publications” are defined as all peer-reviewed articles and invited chapters/reviews. We accessed the NIH Research Portfolio Online Reporting Tools (RePORT) Web site for verification and additional details about NIH grant support.⁹ Public (federal) research support was verified by cross-references to the RePORT Web site.⁹ PubMed¹⁰ was accessed to verify publication details of manuscripts conveyed in curricula vitae, which were then tallied. The combination of these methods provided a cross-reference for self-reported data and ensured, to the best of our abilities, data accuracy.

Because the majority of CRTP participants were still in residency and fellowship training at the time of data collection, we focused on career progression of CRTP participants who had fully completed residency and fellowship training by January 1, 2014, and who had made transitions to academic medical centers with junior faculty positions, clinical practice, or industry (pharmaceutical, biotechnology, or device companies).

The NIH office of human subjects research protections determined that the annual survey and the analysis of these research data were exempt from IRB review under 45 CFR 46.101(b)(2).

Results

The CRTP cohort of 340 consisted of 163 women (47.9%) and 177 men (52.1%). Participants were ethnically diverse and, by self-report, there were 55 (16.2%) underrepresented minorities (using established NIH definitions). For the outcomes and career paths analyses, we included only participants who had completed both primary residency and subspecialty training by January 2014. Therefore, data for 135 (39.7%) of the 340 CRTP participants were analyzed; 74 (54.8%) were males and 61 (45.2%) were females; 19 of 135 (14.1%) were underrepresented minorities. Of the 135, only 5 did not provide follow-up regarding their careers and research commitment—a 96.3% response rate.

We were able to determine the undergraduate degrees for 334 of the 340 participants. Table 1 reports the categories of undergraduate degrees in four numerically similar cohorts of class years, as well as their associated percentages. The overwhelming majority (225 students [67.4%]) of CRTP participants received an undergraduate degree in the life sciences (169 students) or chemistry/biochemistry (56 students).

Table 1: Undergraduate Degrees of Fellows Entering the National Institutes of Health Clinical Research Training Program, From a Study of Career and Research Outcomes of Program Participants, 1997–2012

Research commitment

CRTP participants who had completed all of their training were asked to indicate whether they were conducting research and to provide a self-reported allocation of time committed to research activities. Research time was sorted on the basis of their primary professional attribution to an academic institution, private practice, or industry. Of the 130 respondents, 84 (64.6%) indicated that they were conducting research; 46 (35.4%) were not. Table 2 demonstrates that of those 84, most (74 [88%]) were in faculty positions at academic medical centers, and 46 of those 84 (54.8%) reported spending more than 25% of their time conducting research. Within this cohort, there were no differences in self-reported research commitment between men and women. All three in industry reported spending over 50% of their time conducting research.

Table 2: Research Time Allotted and Professional Categories of National Institutes of Health Clinical Research Training Program Alumni Who Were Conducting Research as of 2014, From a Study of Career and Research Outcomes of Program Participants, 1997–2012

Residency training

We reviewed the primary residencies of those 130 CRTP alumni to determine whether the type of residency undertaken had any relationship to whether they were conducting research. Table 3 lists the distribution of residencies for the 130 and whether they were conducting research. For most residencies and subspecialties, the majority of CRTP graduates reported conducting research for 25% or more of their time. The proportion was most striking for individuals who had completed internal medicine and pediatrics residencies. However, less than 50% of CRTP alumni trained in dermatology and radiology reported conducting any research. Three individuals did not complete residencies: One practices law at a major pharmaceutical company, one became a writer, and one died.

Table 3: National Institutes of Health Clinical Research Training Program Alumni Research Based on Residency, From a Study of Career and Research Outcomes of Program Participants, 1997–2012

Research funding sources

The extent and type of NIH/public funding support (grants) received by the 84 alumni who completed training and are conducting research were analyzed. Many reported small grants-in-aid from either a professional society or private foundation. However, according to self-report, 23 of the 84 (27.4%) alumni who completed training and are conducting research have received NIH grants. Of those 23 alumni, 17 were principal investigators on 1 or more grants, and the remainder served as grant coinvestigators (Chart 1). Of the 17 principal investigators, 3 have received Research Project (R01) Grants; 5 received Mentored Patient-Oriented Research Career Development (K23) Awards; 5 received Mentored Clinical Scientist Development Program (K12) Awards from their institutions; 3 received Mentored Clinical Scientist Career Development (K08) Awards; and 1 received a Mentored Cancer Prevention, Control, Behavioral Sciences and Population Sciences Career Development Award (K-07). Other NIH grants included 3 Ruth L. Kirschstein National Research Service Awards for individual postdoctoral fellowships (F32) and 2 NIH Exploratory/Developmental Research Grant Awards (R21).

Chart 1: NIH Clinical Research Training Program Alumni and NIH Funding as a Principal Investigator as of 2014, From a Study of Career and Research Outcomes of Program Participants, 1997–2012

Publications

The 84 CRTP alumni conducting research have been productive on the basis of peer-reviewed and other scholarly publications. Using submitted curricula vitae and bibliographies, and confirmation by PubMed,¹⁰ works of scholarship published by December 31, 2013, were tallied. Figure 1 shows the average number of scholarly publications per individual in three CRTP-alumni cohorts, based on the year of entry into the CRTP—1997–1999; 2000–2002; and 2003–2007—that is, numerically similar groups. As seen in the bar graphs, all individuals published, on average, one to two manuscripts while working with their CRTP mentors. Consistent with productive research careers, the average number of publications by the alumni conducting research is highest (23.5) in the cohort representing the first three years of the program, followed by an average of 17.6 publications in the 2000–2002 group, with an average of 8.6 in the group representing classes from 2003 to 2007.

Figure 1: Average number of scholarly publications per individual (y-axis) by National Institutes of Health Clinical Research Training Program cohorts (x-axis) who completed all clinical training and reported conducting research as of 2014. From a study of career and research outcomes of program participants, 1997–2012. Abbreviation: CRTP indicates Clinical Research Training Program.

Additional degrees

Of the 84 alumni who have completed clinical training and are conducting research, 28 (33.3%) earned an advanced degree after completing their professional doctorate. Two received additional doctorates (one physician obtained a PhD, and one dentist obtained an MD degree). The remaining 26 individuals received master’s degrees, either in clinical research or in public health.

Discussion

Publicly and privately funded yearlong research opportunities for medical and dental students have existed for years. These programs include the Sarnoff Cardiovascular Research Foundation Research Fellowship, the HHMI Medical Fellows and the HHMI-NIH Research Scholars Programs, the Doris Duke Clinical Research Fellowship Program, the Centers for Disease Control and Prevention Experience Applied Epidemiology Fellowship Program, and the NIH CRTP. Although components of these programs varied, they all provided students with common elements. These included didactic (clinical research curriculum, journal clubs, lectures, etc.) and practical or “hands-on” mentored basic, translational, or clinical research experiences.

The NIH CRTP had as its primary focus a mentored clinical or translational research experience. Those students selected a mentor from the pool of NIH intramural program investigators in Bethesda, Maryland. At least 90% of participants’ time was devoted to research. Students additionally participated in a journal club which illustrated the best principles and practices of clinical and translational research. Also, they learned about clinical protocol implementation through lectures and case presentations by NIH intramural principal investigators. Furthermore, students learned about drug and device discovery and development as well as leadership skills, helpful in their future careers.

Outcome data on the research careers of participants from the above programs are quite limited.⁸ The HHMI-NIH Research Scholars and HHMI Medical Fellows Programs had been in existence since 1985 and 1989, respectively. Both provided mentored (primarily basic laboratory) research experiences. In 2003, Fang and Meyer⁸ analyzed early career outcomes in both programs and determined that participation in those HHMI programs increased the likelihood of alumni receiving NIH postdoctoral awards, with award rates slightly lower when compared with NIH-supported Medical Scientist Training Programs (MSTPs) but very comparable to awards for those in non-MSTP MD/PhD programs.

Outcome data have also been published for the Doris Duke Clinical Research Fellowship Program, based on follow-up after the first 3.5 years of that program.¹¹ However, those surveyed were still in residency training; hence, there were no data comparable to those of the HHMI data in terms of success rates for funded postdoctoral awards. The Doris Duke data focused on satisfaction with the clinical research training experiences, which was almost uniformly positive, and nearly half of those alumni surveyed indicated an increased commitment to a research career after their year of research.

Our findings about outcomes from the NIH CRTP provide career outcomes for a cohort of program graduates who have completed all of their training, post professional school. To our knowledge, there are no comprehensive data assessing the potential association between yearlong research programs and the ultimate research careers for medical and dental students. The sample size is relatively small, and data collection was based on alumni self-reporting coupled with data we collected from public sites.^9,10 However, the data indicate an association between research careers and research productivity based on participation in the CRTP.

Eighty-four of 130 (almost 65%) CRTP alumni, who have completed all clinical training, reported spending some professional time conducting research. Focusing on this subset, nearly 55% (46 of 84) reported at least 25% of their time conducting research, and 37% reported spending over 50% or more of their professional activities on research. Although determining comparable data is not easy, 2011 questionnaire data indicated that only 1.6% of the U.S. physician population reported research as their primary professional livelihood.¹² The same authors also pointed out that research aspirations for medical graduates (as recent as 2009) were 18.8%, pointing to a sharp drop-off in aspirations from the time of graduation to current professional activity. Compared with both, CRTP data, in an admittedly small cohort, indicate a higher percentage of research engagement, albeit in a skewed group with a declared interest in pursuing research. No contemporary data are available from other yearlong medical and dental student research programs, to our knowledge. However, a recent report of career outcomes of graduates from 24 MD–PhD programs indicated that nearly two-thirds of graduates spent at least 50% of their time in research activities, with 39% of MD–PhDs from those programs reporting at least 75% of their time conducting research.¹³

The NIH grant data for the 23 members of the cohort of CRTP alumni shown in Chart 1 indicate that these individuals are competitive in receiving K or R awards. Three individuals from among the earliest CRTP classes (1997, 1998, and 2001) received R01 awards; 2 of those 3 had received an early career development award prior to their R01. The numbers are small but imply that progression to independent support is a goal and an outcome for some CRTP alumni. Longer follow-up and tracking of the CRTP cohort are needed to determine ability to receive NIH funding as independent investigators. It is hoped that additional, successful public funding of the alumni with R01 grants will also be reported in other class cohorts as they progress in their career development and research post completion of training. With longer follow-up and larger samples, additional comparisons of the CRTP alumni with individuals with MD–PhD degrees should be possible. Of note, the percentage of CRTP alumni who have been awarded any postdoctoral NIH grants (27.4%) is roughly comparable to that seen with the HHMI-NIH Research Scholars Program (21%) and the HHMI Medical Fellows Program (24%).⁸ Furthermore, CRTP alumni research productivity, as measured by publications in the biomedical science literature, indicated that those most engaged in research and furthest along in their careers had the highest average number of publications.

It is worth noting the costs of the program as well, though not a formal component of this study. The average direct cost per year per student in the CRTP is not precisely known, because many research mentors also provided laboratory support. However, the annual cost per student per year provided with central support was approximately $45,000 when the program ended in 2012. The current (2015) direct cost per student per year in the NIH MRSP is approximately $51,000, which is comparable to the average cost of nearly $48,000 in fiscal year 2014 for MD–PhD students funded by the NIH through its Medical Scientist Training Program.¹⁴

Articles have focused on what is necessary to sustain the pipeline of clinical researchers. The concept of “early capture” has been used, and most agree that waiting until completion of professional school, residency, and a subspecialty fellowship to learn how to conduct clinical research may be too late to initiate and sustain interest in a biomedical sciences career. In June 2014, the NIH Physician–Scientist Workforce Working Group released a report that highlighted a number of distressing trends in the demographics of the physician–scientist workforce.¹⁵ A recent perspective from four educators who lead training programs for MD–PhDs has further highlighted the declining trends of physicians primarily focused on research careers.¹⁶ They have proposed some ideas to augment the physician–scientist workforce and have urged that the broad biomedical community including universities and academic medical centers, the NIH, and professional societies should all work proactively as partners to address the challenges in maintaining and hopefully enhancing the physician–scientist pipeline. Advocating for targeted medical school education to heighten the importance of clinical research as it relates to patient care has been raised for years.¹⁷ In 2006, the Association of American Medical Colleges advocated for the mandatory inclusion of education about clinical and translational research into the curriculum of medical schools.¹⁸ Others have reinforced this notion.^19,20 Progress is being made, based on modifications to curricula in medical schools.^21,22 The NIH commitment and its catalytic role in helping to develop and sustain the pipeline of clinician–scientists have been stated,²³ and its support of the MRSP articulated.²³

Although the CRTP cohort is small and the time of follow-up relatively short, these data provide initial and compelling evidence that there are outcome benefits, based on conducting research as part of one’s career, for medical and dental student graduates of yearlong research enrichment programs. Unfortunately, there are minimal comparable, benchmark outcome data, but the data herein demonstrate that the majority of CRTP alumni who have completed clinical training are conducting research and publishing manuscripts. Many are successful in competing for postdoctoral NIH grants. Long-term follow-up is needed to confirm the importance of early, mentored research experiences in the development of research careers for medical and dental students. Completing subspecialty training and establishing research careers takes years. The average age at which a physician–scientist receives a first R01 award has continued to rise and is now approximately 45.^15,24 Academic research environments are challenging based on this and other factors. Indeed, many young clinician–scientists are either deciding to not enter into biomedical research at all, or they are forced to abandon such careers if their research funding never materializes or is not sustained.

The clinical research pipeline is significantly threatened, and further dismantling of the limited existing mentored research opportunities as well as research funding challenges for individuals in their formative years could lead to the complete extinction of a generation of young investigators. This has been and remains a very real threat.^1,2,15,16,24 Therefore, support from public and private entities for mentored research programs, such as the former NIH CRTP, is critical to protect and sustain future generations of clinician–scientists.

Acknowledgments: The authors wish to thank the 130 Clinical Research Training Program alumni for participating in this long-term follow-up, and for providing their responses to annual career outcome surveys. The authors also thank Ms. Jennifer Simmons and Mr. Kenny J. Williams from the National Institutes of Health (NIH) Clinical Center’s Office of Clinical Research Training and Medical Education for their assistance in collecting and providing baseline analysis of the data and Dr. Elaine K. Gallin for her thoughtful review of this manuscript. The authors thank the NIH Common Fund and the staff of the NIH Office of Strategic Coordination, Division of Program Coordination, Planning, and Strategic Initiatives for their guidance and program support.