There is a great need to translate basic research discoveries into patient-oriented clinical research (POCR) to improve human health. Unfortunately, for over two decades there has been a continuing decline in the number of physicians who pursue research careers.1 Many obstacles interfere with the successful recruitment and retention of physicians into POCR careers, including lack of effective clinical-research-specific training programs and of skilled and committed mentorship.2 The mentor’s involvement in the development and implementation of a POCR project has been shown to be a significant factor in the career development of hematology/oncology trainees.3 Often, the availability, quantity, and quality of research mentorship is not adequate in all of the areas that a junior investigator needs to prepare for a POCR career.3 Furthermore, availability of mentoring and educational programs to optimize a physician’s preparation for a successful POCR career varies widely.3
In 2003, recognizing the potential career development benefit of creating a program focused on providing an enhanced and prolonged educational and mentoring experience for early-career physician–scientists pursuing POCR in hematology, the American Society of Hematology (ASH) implemented a novel Clinical Research Training Institute (CRTI). Here, we describe the structure of the ASH CRTI and the trainees’ career development outcomes for the program’s first seven years.
Applicants must either be in a hematology-related fellowship training program or a junior faculty member no more than three years from completion of fellowship at any institution in the United States or Canada, and must hold a medical degree (MD, DO, or international equivalent). Applicants are selected once a year according to the following criteria: the perceived potential for the applicant to have a successful POCR career in hematology, the experience and training record of the applicant’s mentor at his or her home institution (“home mentor”), the home mentor’s plan for developing the applicant’s career, the applicant’s institutional environment, a concept paper for a clinical research protocol, and the feasibility of the proposed project. A study section of established clinical investigators, including CRTI faculty, assesses and ranks all applications using these selection criteria. The number of selected trainees is limited to 20 in order to maintain a 1:1 ratio of trainees to faculty—an important aspect of the program. To enhance geographic diversity among the trainees, no more than one trainee per academic program per institution per year can be selected.
The CRTI faculty represent a broad range of hematology disciplines and include biostatisticians. Faculty are selected on the basis of their research or biostatistics skills and their demonstrated interest and ability to interact with trainees, serve as mentors during the yearlong program, and participate in the program’s various aspects. These faculty are suggested by current and former CRTI faculty, self-identified, or identified by members of a CRTI Oversight Subcommittee. Final selection of CRTI faculty is by the CRTI codirectors followed by approval by the CRTI Oversight Subcommittee and ASH’s officers. Diversity in representation of ethnic, racial, and gender groups, viewpoints, and experiences is a goal in faculty selection.
Faculty play two roles in the CRTI. Some faculty are paired with a trainee to become the trainee’s “CRTI mentor.” The CRTI mentor maintains a defined relationship with the trainee throughout the CRTI year (and often beyond) to facilitate the trainee’s project and career development. All faculty also participate in the small group for a defined number of trainees. During the summer workshop these faculty, in addition to the CRTI mentor, contribute to the development of their small group trainees’ projects and career planning. Often, these faculty remain available to help the trainee beyond the summer workshop.
The yearlong CRTI program (August 1–July 31) begins with a weeklong workshop that provides direct interaction between faculty mentors and trainees for 13 hours or more each day. The didactic content includes interactive sessions on how to develop and conduct POCR projects; funding, regulatory, and ethical issues; life–work balance skills; and others.
Each trainee develops an original research proposal with help from a research-theme-based small group that includes three to five trainees, a biostatistician, and three to four other faculty mentors. A biostatistician was included with each small group after it was recognized that the trainees needed intensive biostatistics input on their research projects. An important aspect of the small groups is trainees’ peer review of each other’s research proposals.
Faculty mentors lead discussions on career development and help the trainees develop individualized career development plans. CRTI faculty who are home mentors for CRTI trainees do not serve as their trainees’ CRTI mentors and do not participate in those trainees’ small groups. Whenever possible, CRTI faculty who are from the same institution as a trainee (but not the home mentor) are not assigned to the trainee’s small group or as the trainee’s CRTI mentor. A summary report of CRTI mentor–trainee interactions is provided to each home mentor at the conclusion of the summer workshop. Upon return to their home institutions, trainees are expected to further develop and implement their clinical trials and participate in additional educational and training experiences under the guidance of their home mentors.
The small groups reconvene in person twice during the year to review both research and career development progress. A gathering of trainees, CRTI mentors and faculty, and trainees’ home mentors is also held during ASH’s annual meeting to promote communication between CRTI mentors and home mentors for trainees’ long-term benefit.
Trainees complete self-assessments of knowledge gained during the workshop and evaluate the effectiveness of the curriculum and the faculty. In 2010, a more thorough evaluation of the outcomes of the trainees’ career development from the initial seven CRTI cohorts was undertaken. Data, including current position, number of publications, and funding awards, were extracted from curricula vitae (CVs) solicited from trainees and from a SurveyMonkey (SurveyMonkey, Palo Alto, California) questionnaire (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A280) that was distributed electronically.
Demographic characteristics of the 140 trainees who composed the first seven CRTI cohorts are shown in Table 1. The mean number of applications received was 46 per year (range: 39–61) for the 20 available trainee positions. Table 1 shows participants’ diversity by gender, advanced degree, hematology specialty, and academic position. Selected applicants represented 52 different institutions; data from 140 applications by geographic region of the United States at the time of application included 43 East, 34 South, 35 Midwest, and 18 West; 10 were from Canada. From 2003 to 2010, trainees represented a variety of institutions. The range in number of trainees from individual successful institutions during this time period was from 1 to 10 with a median of 4. Although participation was originally limited to trainees at U.S. or Canadian institutions, 15 (10.7%) participants identified themselves as citizens and/or natives of other countries.
Beginning in 2006, CRTI applicants were asked to self-identify their gender, race/ethnicity, and disability status on their applications. Compiled information on applicants and accepted trainees from 2006 to 2010 is presented in Table 2.
Pre- and post-summer-workshop questionnaires included a self-assessment of trainees’ competence and confidence in their ability to pursue POCR. In addition, between 2011 and 2012, we collected 126 responses from a questionnaire that asked former CRTI trainees to reflect on the impact of CRTI on their career. Not all questions were answered by all respondents. When asked whether the CRTI program provided them with an opportunity to develop their career as an independent researcher, 56.6% of respondents (69/122) strongly agreed that it did, 38.5% (47/122) agreed, and 4.9% (6/122) were neutral. Further, in reporting agreement with the statement that the CRTI experience was instrumental in retaining trainees in a hematology research career, 40.3% (50/124) strongly agreed, 40.3% (50/124) agreed, 13.7% (17/124) were neutral, 3.3% (4/124) disagreed, and 2.4% (3/124) strongly disagreed.
POCR project and career success
A wide range of nonmalignant and malignant hematology areas were represented in the research projects developed at the summer workshop. Trainees had a high degree of success establishing a POCR study (either their CRTI project or another study) at some time following the CRTI summer workshop. This included 42.1% (53/126) who described their success as “excellent,” 44.4% (56/126) who described it as “satisfactory,” 7.9% (10/126) who described it as “poor,” and 5.6% (7/126) who had not tried to establish a POCR study. At the time of assessment, 29.3% (37/126) of CRTI research projects were completed and 25.4% (32/126) were ongoing. Four percent (5/126) of projects were closed at some point after trainees initiated them, and 37.3% (47/126) of proposed projects were never opened. The primary reasons that projects never opened were the lack of availability of a drug from a pharmaceutical company, feasibility concerns that were identified by CRTI biostatisticians, and trainee relocation to another institution.
Interinstitutional collaborations were promoted both during and following the CRTI program. These efforts resulted in 14 trainees establishing collaborations with other trainees, 29 with faculty, and 22 with both trainees and faculty. These 65 collaborations are not likely to have occurred without planned opportunities to network and build relationships with the other CRTI trainees or faculty.
Ninety-four percent (119/126) of participants continued to participate in POCR activities, and 87% (109/126) considered themselves to be “clinical investigators” at the time of assessment. The majority (83%; 107/129) of trainees were in academic positions, whereas others were in government (1%; 1/129), industry (2%; 3/129), academic private practice (1%; 1/129), academic/industry (2%; 3/129), or private practice (5%; 7/129) environments. The vast majority (84%; 109/129) of former CRTI trainees continuing to conduct POCR were pursuing hematology-related projects.
Publications and grants
The 133 trainees for whom data have been collected have published 1,035 peer-reviewed articles (range: 0–94), 173 chapters (range: 0–16), 115 review articles (range: 0–9), and 69 other publications (e.g., editorials, letters to the editor; range: 0–21) since the calendar year in which the individual trainees attended the CRTI summer workshop (Table 1). Although individual trainees had accomplished a wide range of publications, the average publication productivity was 7.8 peer-reviewed articles, 1.3 chapters, and 0.9 review articles per trainee over the 1- to 7-year experience.
CRTI trainees received a total of at least 262 noncorporate, extramural financial awards (Table 1 and Figure 1); only 13% (17/126) of trainees reported receiving no research funding. This latter group included those working in non-research-related careers. Trainees also received many internal awards from their academic institutions (data not shown), and there were over 44 grants provided from corporate sources for the conduct of individual clinical trials.
Review of 14 ongoing mentor–mentee relationships revealed that CRTI trainees and their CRTI mentors jointly published an average of 13 papers and submitted a total of 4 grant applications. Not all data on mentoring were objective; subjective comments were also informative. One CRTI graduate recognized the impact that ongoing mentoring relationships had on his career, stating:
One of the most powerful benefits of CRTI is the doors that it opens. Many of the speaking, writing, and research opportunities that I have been afforded over the last four years can be traced directly back to my participation in this program.4
A second graduate reflected,
It has been gratifying, beyond my expectations, to see how powerfully catalytic the ASH CRTI has been in bestowing the skills, knowledge, and confidence to push something as small as an idea into an exciting, international, and important research project….5
The ASH CRTI mentoring and career development model has had an exceptionally positive impact on promoting the hematology POCR careers of participants by both objective and subjective measures. For objective criteria, we examined those that are generally used by academic promotions committees to assess academic achievement and success, including publications and the acquisition of funding to support the individual’s academic activities. Subjective criteria included trainee assessment of mentoring and networking experiences that provided them with increased knowledge, confidence, and sense of preparedness for pursuing a POCR career. As there was a high level of retention in hematology-based careers, this model may be extrapolated to other specialties in need of larger cadres of clinical investigators.
The lack of an adequate “control” group for comparison limits the ability to establish a causal effect of CRTI on the participants’ research successes. Although a small sample was assessed, another limitation of this report is the lack of verification that all publication and funding claims actually occurred as stated in CVs and survey responses. Self-assessment of an increase in knowledge is also a subjective measurement and may be inaccurate.
As a result of CRTI program evaluations, several changes have been implemented. An initial goal and one measure of program success was to have former CRTI trainees make up an increasing number of the faculty. For 2014–2015, nine former CRTI trainees will be faculty, and one is the chair of the CRTI Oversight Subcommittee. In recognition of the impact that more continuous mentoring had on trainees’ academic success, more formal trainee–CRTI mentor relationships with prescribed continued communications have been introduced into the program. To increase the likelihood of project implementation, drugs to be studied must be available at the time of application. Recognizing the value of global collaborations, CRTI now accepts qualified international trainees. Follow-up evaluation will reveal whether these interventions provide added benefit.
The CRTI program is resource-intensive in terms of trainee and faculty time and financial resources, but the return on these investments is high. Program costs are estimated at approximately $15,000 per trainee, and in the initial years all costs were borne by ASH with some supplemental funding from the Wallace Coulter Foundation and the National Institutes of Health. In 2013, ASH was awarded a training grant (T25E) from the National Cancer Institute that has offset a substantial portion of the cost. Challenges to sustain and improve the program include acquiring the necessary financial resources for the program and identifying mechanisms to make an equivalent type of experience available to greater numbers of trainees.
Another challenge is increasing the diversity of both the CRTI trainees and faculty. In 2004, demographic information of hematology/oncology fellows in the United States revealed that 41% were female, 3.4% were black, 59.6% were white, 4.0% were Hispanic, and 30.4% were Asian.6,7 Although the CRTI applicant and trainee pools have been roughly equivalent to the possible catchment pool (by chance), ASH has initiated new efforts to encourage applications from underrepresented groups.
The ASH CRTI model has been very successful at preparing early-career clinical investigators to be significant contributors to POCR. A focus on mentoring and training for these careers appears to be important in increasing the numbers of academically oriented investigators committed to finding, evaluating, and delivering effective therapies for human diseases. Although already very successful, ongoing evaluation of the program and its outcomes will identify areas for continued improvement.
Acknowledgments: The authors thank Karen Kayoumi, Joseph Basso, Kathleen Komarinski, and the American Society of Hematology (ASH) for their assistance and support in the creation and distribution of the SurveyMonkey instrument and the collection of data from the former Clinical Research Training Institute (CRTI) trainees. The authors also thank the former CRTI trainees for completing our survey and for providing their curricula vitae and other requested information.