Secondary Logo

Journal Logo

The Effect of an Intense Mentoring Program on Junior Investigators’ Preparation for a Patient-Oriented Clinical Research Career

Burns, Linda J. MD; Clayton, Charles P.; George, James N. MD; Mitchell, Beverly S. MD; Gitlin, Scott D. MD

doi: 10.1097/ACM.0000000000000742
Innovation Reports
Free
SDC

Problem There is a recognized need to translate scientific discoveries to patient-oriented clinical research (POCR). Several obstacles interfere with the successful recruitment and retention of physicians for POCR careers.

Approach The American Society of Hematology developed a yearlong educational and mentoring experience, the Clinical Research Training Institute (CRTI), for early-career physician–scientists from multiple institutions throughout the United States and Canada pursuing POCR careers. Several academic outcome measures of the 140 participants in the first seven years (2003–2010) of CRTI were evaluated by reviewing former trainee participants’ curriculum vitae and survey responses.

Outcomes Ethnic, racial, and gender diversity of CRTI trainees was reflective of the proportions represented across U.S. hematology/oncology fellowship programs. Eighty-six percent (109/126) of trainees reported success establishing a POCR study; nearly half (62/126) had primarily research-focused jobs. Former CRTI trainees received at least 262 external grant awards and published 1,035 peer-reviewed manuscripts, 173 chapters, and 115 review articles.

Next Steps Because mentorship is key to developing a successful career, the CRTI program is being modified to enhance longitudinal mentorship by CRTI faculty mentors and mentors at trainees’ home institutions, as well as to encourage the establishment of collaborations and the potential for research project success. Efforts to make the CRTI experience available to more phy sicians, include more CRTI graduates as faculty, and increase participation by hematologists from backgrounds under represented in medicine are under way.

L.J. Burns is professor of medicine, Division of Hematology, Oncology and Transplantation, University of Minnesota, Minneapolis, Minnesota.

C.P. Clayton is senior director of education and training, American Society of Hematology, Washington, DC.

J.N. George is professor of medicine, Departments of Medicine and Biostatistics and Epidemiology, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma.

B.S. Mitchell is George E. Becker professor of medicine, Divisions of Hematology and Oncology, Stanford Cancer Institute, Stanford University, Stanford, California.

S.D. Gitlin is associate professor of internal medicine, Division of Hematology/Oncology, University of Michigan, and staff physician, Veterans Affairs Health System, Ann Arbor, Michigan.

Funding/Support: The ASH CRTI program has received funding via various sources and institutes at the National Institutes of Health (NIH; see below). In addition, support has been received from the Wallace Coulter Foundation in the early years of the program. None of these funds were used in the conduct of the study that is reported in this manuscript.

NIH funding (including years not involved in this manuscript’s data collection): 2006: NIH Research Grant #R13 HL84862 funded by the National Heart, Lung, and Blood Institute, the National Cancer Institute, and the National Institute on Aging. 2009: Award Number R13HL097393 from the National Heart, Lung, and Blood Institute, the National Cancer Institute, and the National Institute for Diabetes and Digestive and Kidney Diseases. 2010: NIH Research Grant 1R13HL104875-01 from the National Institutes of Health, the National Heart, Lung, and Blood Institute, the National Institute of Aging, the National Center for Research Resources, the National Cancer Institute, and the National Institute of Diabetes and Digestive and Kidney Diseases. 2011: NIH Research Grant R13HL110719-01 funded by the National Heart, Lung, and Blood Institute, the National Institute of Diabetes and Digestive and Kidney Diseases, and the National Center for Research Resources. 2012: NIH Research Grant R13HL115845-01 funded by the National Heart, Lung, and Blood Institute and the National Institute of Diabetes and Digestive and Kidney Diseases. 2013: NIH Research Grant 1R13HL120372-01 funded by the National Heart, Lung, and Blood Institute and the National Institute of Diabetes and Digestive and Kidney Diseases. 2013: NIH Grant R25CA168526 funded by the National Cancer Institute.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Previous presentations: Some of the content of this manuscript was presented as an abstract and poster at the Annual Meeting of the American Society of Hematology, December 2013, New Orleans, Louisiana.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A280.

Correspondence should be addressed to Scott D. Gitlin, University of Michigan Medical Center, C345 Med Inn Building, SPC 5848, 1500 E. Medical Center Dr., Ann Arbor, MI 48109-5848; telephone: (734) 936-5410; e-mail: sgitlin@umich.edu.

Back to Top | Article Outline

Problem

There is a great need to translate basic research discoveries into patient-oriented clinical research (POCR) to improve human health. Unfortunately, for over two decades there has been a continuing decline in the number of physicians who pursue research careers.1 Many obstacles interfere with the successful recruitment and retention of physicians into POCR careers, including lack of effective clinical-research-specific training programs and of skilled and committed mentorship.2 The mentor’s involvement in the development and implementation of a POCR project has been shown to be a significant factor in the career development of hematology/oncology trainees.3 Often, the availability, quantity, and quality of research mentorship is not adequate in all of the areas that a junior investigator needs to prepare for a POCR career.3 Furthermore, availability of mentoring and educational programs to optimize a physician’s preparation for a successful POCR career varies widely.3

Back to Top | Article Outline

Approach

In 2003, recognizing the potential career development benefit of creating a program focused on providing an enhanced and prolonged educational and mentoring experience for early-career physician–scientists pursuing POCR in hematology, the American Society of Hematology (ASH) implemented a novel Clinical Research Training Institute (CRTI). Here, we describe the structure of the ASH CRTI and the trainees’ career development outcomes for the program’s first seven years.

Back to Top | Article Outline

Trainees

Applicants must either be in a hematology-related fellowship training program or a junior faculty member no more than three years from completion of fellowship at any institution in the United States or Canada, and must hold a medical degree (MD, DO, or international equivalent). Applicants are selected once a year according to the following criteria: the perceived potential for the applicant to have a successful POCR career in hematology, the experience and training record of the applicant’s mentor at his or her home institution (“home mentor”), the home mentor’s plan for developing the applicant’s career, the applicant’s institutional environment, a concept paper for a clinical research protocol, and the feasibility of the proposed project. A study section of established clinical investigators, including CRTI faculty, assesses and ranks all applications using these selection criteria. The number of selected trainees is limited to 20 in order to maintain a 1:1 ratio of trainees to faculty—an important aspect of the program. To enhance geographic diversity among the trainees, no more than one trainee per academic program per institution per year can be selected.

Back to Top | Article Outline

Faculty

The CRTI faculty represent a broad range of hematology disciplines and include biostatisticians. Faculty are selected on the basis of their research or biostatistics skills and their demonstrated interest and ability to interact with trainees, serve as mentors during the yearlong program, and participate in the program’s various aspects. These faculty are suggested by current and former CRTI faculty, self-identified, or identified by members of a CRTI Oversight Subcommittee. Final selection of CRTI faculty is by the CRTI codirectors followed by approval by the CRTI Oversight Subcommittee and ASH’s officers. Diversity in representation of ethnic, racial, and gender groups, viewpoints, and experiences is a goal in faculty selection.

Faculty play two roles in the CRTI. Some faculty are paired with a trainee to become the trainee’s “CRTI mentor.” The CRTI mentor maintains a defined relationship with the trainee throughout the CRTI year (and often beyond) to facilitate the trainee’s project and career development. All faculty also participate in the small group for a defined number of trainees. During the summer workshop these faculty, in addition to the CRTI mentor, contribute to the development of their small group trainees’ projects and career planning. Often, these faculty remain available to help the trainee beyond the summer workshop.

Back to Top | Article Outline

Curriculum

The yearlong CRTI program (August 1–July 31) begins with a weeklong workshop that provides direct interaction between faculty mentors and trainees for 13 hours or more each day. The didactic content includes interactive sessions on how to develop and conduct POCR projects; funding, regulatory, and ethical issues; life–work balance skills; and others.

Each trainee develops an original research proposal with help from a research-theme-based small group that includes three to five trainees, a biostatistician, and three to four other faculty mentors. A biostatistician was included with each small group after it was recognized that the trainees needed intensive biostatistics input on their research projects. An important aspect of the small groups is trainees’ peer review of each other’s research proposals.

Faculty mentors lead discussions on career development and help the trainees develop individualized career development plans. CRTI faculty who are home mentors for CRTI trainees do not serve as their trainees’ CRTI mentors and do not participate in those trainees’ small groups. Whenever possible, CRTI faculty who are from the same institution as a trainee (but not the home mentor) are not assigned to the trainee’s small group or as the trainee’s CRTI mentor. A summary report of CRTI mentor–trainee interactions is provided to each home mentor at the conclusion of the summer workshop. Upon return to their home institutions, trainees are expected to further develop and implement their clinical trials and participate in additional educational and training experiences under the guidance of their home mentors.

The small groups reconvene in person twice during the year to review both research and career development progress. A gathering of trainees, CRTI mentors and faculty, and trainees’ home mentors is also held during ASH’s annual meeting to promote communication between CRTI mentors and home mentors for trainees’ long-term benefit.

Back to Top | Article Outline

Program evaluation

Trainees complete self-assessments of knowledge gained during the workshop and evaluate the effectiveness of the curriculum and the faculty. In 2010, a more thorough evaluation of the outcomes of the trainees’ career development from the initial seven CRTI cohorts was undertaken. Data, including current position, number of publications, and funding awards, were extracted from curricula vitae (CVs) solicited from trainees and from a SurveyMonkey (SurveyMonkey, Palo Alto, California) questionnaire (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A280) that was distributed electronically.

Back to Top | Article Outline

Outcomes

Trainee demographics

Demographic characteristics of the 140 trainees who composed the first seven CRTI cohorts are shown in Table 1. The mean number of applications received was 46 per year (range: 39–61) for the 20 available trainee positions. Table 1 shows participants’ diversity by gender, advanced degree, hematology specialty, and academic position. Selected applicants represented 52 different institutions; data from 140 applications by geographic region of the United States at the time of application included 43 East, 34 South, 35 Midwest, and 18 West; 10 were from Canada. From 2003 to 2010, trainees represented a variety of institutions. The range in number of trainees from individual successful institutions during this time period was from 1 to 10 with a median of 4. Although participation was originally limited to trainees at U.S. or Canadian institutions, 15 (10.7%) participants identified themselves as citizens and/or natives of other countries.

Table 1

Table 1

Beginning in 2006, CRTI applicants were asked to self-identify their gender, race/ethnicity, and disability status on their applications. Compiled information on applicants and accepted trainees from 2006 to 2010 is presented in Table 2.

Table 2

Table 2

Back to Top | Article Outline

Self-assessment

Pre- and post-summer-workshop questionnaires included a self-assessment of trainees’ competence and confidence in their ability to pursue POCR. In addition, between 2011 and 2012, we collected 126 responses from a questionnaire that asked former CRTI trainees to reflect on the impact of CRTI on their career. Not all questions were answered by all respondents. When asked whether the CRTI program provided them with an opportunity to develop their career as an independent researcher, 56.6% of respondents (69/122) strongly agreed that it did, 38.5% (47/122) agreed, and 4.9% (6/122) were neutral. Further, in reporting agreement with the statement that the CRTI experience was instrumental in retaining trainees in a hematology research career, 40.3% (50/124) strongly agreed, 40.3% (50/124) agreed, 13.7% (17/124) were neutral, 3.3% (4/124) disagreed, and 2.4% (3/124) strongly disagreed.

Back to Top | Article Outline

POCR project and career success

A wide range of nonmalignant and malignant hematology areas were represented in the research projects developed at the summer workshop. Trainees had a high degree of success establishing a POCR study (either their CRTI project or another study) at some time following the CRTI summer workshop. This included 42.1% (53/126) who described their success as “excellent,” 44.4% (56/126) who described it as “satisfactory,” 7.9% (10/126) who described it as “poor,” and 5.6% (7/126) who had not tried to establish a POCR study. At the time of assessment, 29.3% (37/126) of CRTI research projects were completed and 25.4% (32/126) were ongoing. Four percent (5/126) of projects were closed at some point after trainees initiated them, and 37.3% (47/126) of proposed projects were never opened. The primary reasons that projects never opened were the lack of availability of a drug from a pharmaceutical company, feasibility concerns that were identified by CRTI biostatisticians, and trainee relocation to another institution.

Interinstitutional collaborations were promoted both during and following the CRTI program. These efforts resulted in 14 trainees establishing collaborations with other trainees, 29 with faculty, and 22 with both trainees and faculty. These 65 collaborations are not likely to have occurred without planned opportunities to network and build relationships with the other CRTI trainees or faculty.

Ninety-four percent (119/126) of participants continued to participate in POCR activities, and 87% (109/126) considered themselves to be “clinical investigators” at the time of assessment. The majority (83%; 107/129) of trainees were in academic positions, whereas others were in government (1%; 1/129), industry (2%; 3/129), academic private practice (1%; 1/129), academic/industry (2%; 3/129), or private practice (5%; 7/129) environments. The vast majority (84%; 109/129) of former CRTI trainees continuing to conduct POCR were pursuing hematology-related projects.

Back to Top | Article Outline

Publications and grants

The 133 trainees for whom data have been collected have published 1,035 peer-reviewed articles (range: 0–94), 173 chapters (range: 0–16), 115 review articles (range: 0–9), and 69 other publications (e.g., editorials, letters to the editor; range: 0–21) since the calendar year in which the individual trainees attended the CRTI summer workshop (Table 1). Although individual trainees had accomplished a wide range of publications, the average publication productivity was 7.8 peer-reviewed articles, 1.3 chapters, and 0.9 review articles per trainee over the 1- to 7-year experience.

CRTI trainees received a total of at least 262 noncorporate, extramural financial awards (Table 1 and Figure 1); only 13% (17/126) of trainees reported receiving no research funding. This latter group included those working in non-research-related careers. Trainees also received many internal awards from their academic institutions (data not shown), and there were over 44 grants provided from corporate sources for the conduct of individual clinical trials.

Figure 1

Figure 1

Back to Top | Article Outline

Mentoring

Review of 14 ongoing mentor–mentee relationships revealed that CRTI trainees and their CRTI mentors jointly published an average of 13 papers and submitted a total of 4 grant applications. Not all data on mentoring were objective; subjective comments were also informative. One CRTI graduate recognized the impact that ongoing mentoring relationships had on his career, stating:

One of the most powerful benefits of CRTI is the doors that it opens. Many of the speaking, writing, and research opportunities that I have been afforded over the last four years can be traced directly back to my participation in this program.4

A second graduate reflected,

It has been gratifying, beyond my expectations, to see how powerfully catalytic the ASH CRTI has been in bestowing the skills, knowledge, and confidence to push something as small as an idea into an exciting, international, and important research project….5

Back to Top | Article Outline

Outcomes summary

The ASH CRTI mentoring and career development model has had an exceptionally positive impact on promoting the hematology POCR careers of participants by both objective and subjective measures. For objective criteria, we examined those that are generally used by academic promotions committees to assess academic achievement and success, including publications and the acquisition of funding to support the individual’s academic activities. Subjective criteria included trainee assessment of mentoring and networking experiences that provided them with increased knowledge, confidence, and sense of preparedness for pursuing a POCR career. As there was a high level of retention in hematology-based careers, this model may be extrapolated to other specialties in need of larger cadres of clinical investigators.

The lack of an adequate “control” group for comparison limits the ability to establish a causal effect of CRTI on the participants’ research successes. Although a small sample was assessed, another limitation of this report is the lack of verification that all publication and funding claims actually occurred as stated in CVs and survey responses. Self-assessment of an increase in knowledge is also a subjective measurement and may be inaccurate.

Back to Top | Article Outline

Next Steps

As a result of CRTI program evaluations, several changes have been implemented. An initial goal and one measure of program success was to have former CRTI trainees make up an increasing number of the faculty. For 2014–2015, nine former CRTI trainees will be faculty, and one is the chair of the CRTI Oversight Subcommittee. In recognition of the impact that more continuous mentoring had on trainees’ academic success, more formal trainee–CRTI mentor relationships with prescribed continued communications have been introduced into the program. To increase the likelihood of project implementation, drugs to be studied must be available at the time of application. Recognizing the value of global collaborations, CRTI now accepts qualified international trainees. Follow-up evaluation will reveal whether these interventions provide added benefit.

The CRTI program is resource-intensive in terms of trainee and faculty time and financial resources, but the return on these investments is high. Program costs are estimated at approximately $15,000 per trainee, and in the initial years all costs were borne by ASH with some supplemental funding from the Wallace Coulter Foundation and the National Institutes of Health. In 2013, ASH was awarded a training grant (T25E) from the National Cancer Institute that has offset a substantial portion of the cost. Challenges to sustain and improve the program include acquiring the necessary financial resources for the program and identifying mechanisms to make an equivalent type of experience available to greater numbers of trainees.

Another challenge is increasing the diversity of both the CRTI trainees and faculty. In 2004, demographic information of hematology/oncology fellows in the United States revealed that 41% were female, 3.4% were black, 59.6% were white, 4.0% were Hispanic, and 30.4% were Asian.6,7 Although the CRTI applicant and trainee pools have been roughly equivalent to the possible catchment pool (by chance), ASH has initiated new efforts to encourage applications from underrepresented groups.

The ASH CRTI model has been very successful at preparing early-career clinical investigators to be significant contributors to POCR. A focus on mentoring and training for these careers appears to be important in increasing the numbers of academically oriented investigators committed to finding, evaluating, and delivering effective therapies for human diseases. Although already very successful, ongoing evaluation of the program and its outcomes will identify areas for continued improvement.

Acknowledgments: The authors thank Karen Kayoumi, Joseph Basso, Kathleen Komarinski, and the American Society of Hematology (ASH) for their assistance and support in the creation and distribution of the SurveyMonkey instrument and the collection of data from the former Clinical Research Training Institute (CRTI) trainees. The authors also thank the former CRTI trainees for completing our survey and for providing their curricula vitae and other requested information.

Back to Top | Article Outline

References

1. Hahm J-o, Ommaya ACommittee on Opportunities to Address Clinical Research Workforce Diversity Needs for 2010, Committee on Women in Science and Engineering Policy and Global Affairs, Board on Health Sciences Policy-Institute of Medicine, and National Research Council and Institute of Medicine of the National Academies. Opportunities to Address Clinical Research Workforce Diversity Needs for 2010. 2006 Washington, DC National Academies Press
2. Varmus H. Statement Before Subcommittee on Public Health and Safety. October 9, 1997. http://www.cancer.gov/aboutnci/director/speeches/statement-before-phs-1997. Accessed March 20, 2015
3. Gitlin SD, Yuan Z, Little RJ, Todd III RF. Factors that influence successful training and faculty career development in hematology/oncology patient-oriented clinical research. J Cancer Educ. 2005;20:72–78
4. Cuker A. Assistant professor, University of Pennsylvania School of Medicine. Personal communication with S.D. Gitlin. April 9, 2013.
5. Cserti C. Assistant professor, transfusion medicine specialist, and consultant hematologist, University Health Network, University of Toronto, Toronto General Hospital. Personal communication with S.D. Gitlin. April 9, 2013.
6. American Medical Association. . Graduate medical education. JAMA. 2005;294:1129–1143
7. Center for Workforce Studies. Physician Specialty Data: A Chart Book. Washington, DC Association of American Medical Colleges August 2006.

Supplemental Digital Content

Back to Top | Article Outline
© 2015 by the Association of American Medical Colleges