In the past decade, the number of physicians and psychologists pursuing clinical research careers in the field of mental health has substantially decreased,1 possibly reflecting the increased time required to prepare for a research career and the growing competition for extramural funding and faculty positions.2–4 Postdoctoral fellowships aim to offset this decrease by providing the training necessary to develop, grow, and maintain successful clinical research careers.
Traditionally, postdoctoral fellowships use an apprenticeship model of individualized training whereby a mentor imparts specialized scientific knowledge and expertise to a fellow. Although this approach is necessary for academic training, investigators have increasingly underscored the need for postdoctoral training to also expose fellows to a broader range of interdisciplinary research experiences and to provide them with essential skills for successful academic and clinical research career development, including expertise in writing grant proposals, preparing manuscripts for publication, managing time, and planning for professional advancement.5,6 All too often trainees who are recipients of significant monetary, personal, and professional investment throughout their postdoctoral training experience are not provided with the information and skills necessary for effective competitive transition into, and success within, academic clinical research careers.7,8
Programs such as the National Institute of Aging's Summer Institute on Aging Research and the Summer Research Institute in Geriatric Psychiatry and the Career Development Institute in Psychiatry, both funded by the National Institute of Mental Health (NIMH), provide information on a broad range of academic career skills, although these are typically shorter in duration, some held over the course of a few days. Further, although many postdoctoral programs increasingly incorporate teaching these skills as part of their core didactic, not all fellows have maximal exposure to the best possible information on these topics because approaches and views on academic career training typically vary from site to site. One way to address this is to implement a uniform model of training across multiple postdoctoral sites.
The Veterans Affairs (VA) Special Fellowship Program in Advanced Psychiatry and Psychology employs a multisite model specifically to provide a large number of entry-level mental health investigators with the skills and knowledge they need to effectively develop and sustain independent clinical research careers.
Multisite training has many benefits. First, a consensus-driven core curriculum, developed and agreed on by experts in the field, can be widely disseminated and accessed by far greater numbers of fellows. Second, expertise that is not available at a given site can be accessed through other participating sites. Third, it provides a mechanism for ongoing feedback from experts at all sites, exposing fellows to a range of interdisciplinary research, clinical, and career development perspectives. Finally, fellows have more opportunities to network, find mentors, and collaborate with peers, all of which are integral to effective career development.
In this article, we describe our model for a multisite training program, including the (1) history, mission, and administrative structure, (2) development of objectives and core curriculum, (3) structure of the core curriculum, (4) recruitment strategies and eligibility requirements, (5) use of key technologies for implementing the multisite nature of the program, (6) measures of success, and (7) advantages and challenges of our multisite approach.
The Multisite Training Program
History, mission, and administrative structure
Being responsible for the educational mission of the VA, the Office of Academic Affiliations (OAA) established the VA Special Fellowship Programs to train new health care professionals to meet increasing patient care needs within and outside of the VA system. In 2000, the OAA established the postdoctoral VA Special Fellowship Program in Advanced Psychiatry and Psychology.
Fellowship sites were selected in response to an OAA issued request for applications (RFA). A key requirement was that applying institutions demonstrate the existence of excellent local mentors for both the PhD and MD components of the program. The assessment of local mentorship was based not only on academic productivity of the mentors but also on a demonstrated track record of mentoring experience. Mentors were assigned to fellows on the basis of common research interests. Each site also had to have the resources to appoint two program directors—one each for the postresidency MD and postdoctoral psychology components—who would be responsible for local implementation and interfacing with the national coordinating site. One PhD and one MD could enter the program at each site, each year, with funding for two years, and a potential third year of funding available through competitive application. Thus, each site could have up to six fellows in any given year.
Each of the 10 chosen fellowship sites is affiliated with one of the VA's 10 Mental Illness Research, Education and Clinical Centers (MIRECCs) nationwide. MIRECCs were developed to improve the provision of mental health care to veterans and all Americans through state-of-the-art interdisciplinary research, education, and clinical models.9,10 These centers serve over 23 states, with each focusing on select areas of mental health. MIRECCs present excellent learning environments for postresidency and postdoctoral training and offer fellows the opportunity to develop expertise in such broad and diverse areas as pharmacological and behavioral treatments for alcohol and cocaine abuse, neuroimaging and neuroscience, translational processes, treatment effectiveness related to posttraumatic stress disorder, and treatment of behavioral, cognitive, or functional problems associated with dementia.
Essential to the successful implementation of the multisite program is the hub site, which handles the core programmatic, administrative, planning, and organizational functions. The VA Sierra-Pacific MIRECC/Stanford University site serves as the hub of our program (see Figure 1 for the structure of the program).
Development of objectives and core curriculum
In 2001, during the one-year planning period before the first class of fellows was enrolled, a committee composed of the directors of the hub site, the 20 fellowship directors, the staff of the OAA, and members of MIRECC leadership at the VA Central Office conducted regular monthly teleconferences dedicated to developing the core objectives and a two-year curriculum, targeted to both psychiatry and psychology trainees (Figure 1).
With the overarching goal of developing outstanding academic and health care system leaders who can effectively translate research findings into clinical practice and educational activities, the vision of OAA and MIRECC leadership was that the program would be dominantly dedicated to research training but would combine individualized, mentored research and clinical training with a multisite, didactic curriculum. The emphasis would be on translational research, in which research, education, and clinical program activities simultaneously inform each other. It was expected that fellows would dedicate 75% of their time to research and educational activities and 25% to clinical activities.
The development of the curriculum was guided by the RFA, which proposed that the following components be incorporated into the program: research methodology; statistics; epidemiology; developing, leading, and financing mental health systems; quality improvement methods and practice; current legal and ethical issues; databases and information management; cutting-edge educational experiences; and advanced clinical training. To further develop the core curriculum and didactics, the committee examined several excellent models, including existing local fellowship models, as well as a broad range of postdoctoral and career development institute models, such as other VA advanced fellowship programs, National Institutes of Health (NIH) T-32 programs, the Robert Wood Johnson Scholars Program, the NIMH-funded Summer Research Institute in Geriatric Psychiatry, and the NIMH-funded Career Development Institute in Psychiatry.
After the initial planning meetings, during which several components and proposals were discussed, the hub site distributed a core curriculum document to all directors and parties for comment and input. This six-month, iterative process ended with a consensus on the program's key objectives and components.
Structure of the core curriculum
The core curriculum is designed to provide fellows with state-of-the-art research training, cutting edge educational experiences, and advanced clinical training. It is also meant to provide fellows with the skills, experience, and knowledge necessary for a successful academic clinical research career, with a focus on submitting a career development grant to the VA (CDA series) or NIH (K series awards) by the end of the fellowship. The curriculum covers five main domains: research, clinical, education, mentorship, and didactics.
Fellows are required to design and conduct a pilot research investigation, forging the first steps in their clinical research domain of focus, generating data needed to obtain a career development award, and resulting in a peer-reviewed publication. The core curriculum teaches fellows to choose the best methodological and statistical approaches, to conduct in-depth literature reviews, to effectively design and conduct a research investigation, and to be knowledgeable about the complexities of IRB roles and responsibilities and the legal and ethical issues involved in human subjects research. Through this component, fellows also gain knowledge about accessing and using VA and other mental health electronic databases for research purposes, increasing skills in advanced statistical methodologies, selecting models and analytic approaches, and preparing and submitting for publication reports on research findings. Additionally, fellows gain increased knowledge of funding mechanisms and career development research awards as well as experience in preparing grant applications.
The curriculum gives fellows the tools that are crucial to skilled mental health clinical care, such as knowledge of advanced clinical approaches in mental health and the identification, evaluation, and selection of the best clinical assessment measures. Through specialized seminars and mentorship, fellows learn to recognize the legal and ethical issues involved in clinical practice in mental health, to apply principles of evidence-based medicine and psychological intervention to day-to-day clinical practice, and to direct mental health research through their clinical experiences. They also gain experience in providing clinical supervision and advice to trainees to increase their understanding of the principles of comorbidity and how these principles affect clinical practice.
Educational goals include acquiring knowledge and skills in the most recent advances in education and teaching through observing and participating in mentored teaching activities using current technology at their respective MIRECC site, including online lectures, video presentations, and chat rooms. Fellows obtain experience in the procedures involved in telemedicine, video streaming, and video conferencing. They also enhance their abilities to effectively disseminate their research findings in a public forum and to a wide range of audiences. One key opportunity is the bienniel National Veterans Health Affairs Mental Health Conference, at which fellows present posters detailing the current status and findings of their research projects. During the most recent conference, MIRECC investigators and other senior VA researchers, along with program officers from NIH, exchanged ideas and provided career development guidance to fellows. The conference also allowed the program directors to meet face-to-face to discuss program achievement and pending issues.
The program also emphasizes a committee mentorship model composed of a primary local research mentor(s), one to two additional local mentors for the clinical and education components, as well as mentorship from the coordinating and other fellowship sites. Although fellows establish individual goals and timelines with their local mentors at the beginning of their fellowships, under the direction of local and national mentors they follow the consensus-driven curriculum of specific research, educational, and clinical goals.
The multisite nature of the program increases the ability to provide mentorship on aspects of career development, scientific domains, or biostatistics which may not be available at the local site. Even when the research foci of sites are identical, the multisite structure provides the fellows with different perspectives and approaches to the same area of specialty. They can also consult with the directors of the hub site, who hold weekly virtual office hours to address questions from fellows across the country.
Finally, a key component of the fellowship program is a mandated twice montly seminar, broadcast through the VA video teleconference (V-Tel) system to all sites, creating a virtual classroom in which fellows from all 10 sites can participate. The seminars cover a broad range of topics including state-of-the-art research methodologies, biostatistics, basic science, genetics, epidemiology, mental health systems, intervention and services research, quality improvement methods, education, and service delivery. Further, experts also share strategies for obtaining federal funding, grant writing, publishing and reviewing papers, and a range of career development issues from negotiation strategies to balancing life and work.
Eligibility requirements and recruitment strategies
Eligibility requirements for the VA Special Fellowship Program in Advanced Psychiatry and Psychology were established for fellow applicants in both postresidency psychiatry and postdoctoral psychology. Physicians must have completed ACGME-accredited training, be board eligible or certified, and have an active, unrestricted state license within the U.S. to practice. Psychologists must have graduated from an American Psychological Association-accredited doctoral program in clinical or counseling psychology and have completed an American Psychological Association-accredited internship in clinical counseling psychology.
Recruitment and advertising are managed at both local and national levels by the individual site directors and those at the hub site. The site directors target local organizations for recruitment that have expertise related to that VA site's specialty research (e.g., dementia, traumatic brain injury, severe mental illness, rehabilitation, substance abuse). They make presentations on the program to local third- and fourth-year psychiatry residents and psychology interns. They also attend regional and national conferences relevant to their local specialty to find qualified psychologists, psychiatrists, and nonpsychiatry MDs either through informal networking or by setting up formal information tables. National recruitment strategies, coordinated by the hub site, include a presence at national conferences (e.g., American Psychiatric Association, American Psychological Association), mass mailings to all national residency training programs in psychiatry, and distribution of information on our overarching program and 10 fellowship sites to all residency training directors.
The use of innovative technologies
A central feature of the program is the use of technologies, including the Web, e-mail, teleconferences, and video streaming, to coordinate the fellowship sites and facilitate the dissemination of the core curriculum, professional development, and interaction among fellows spread across the country.
Although consensus-driven approaches to curricula are not new,11,12 an innovative component of our fellowship program is that the core curriculum is broadcast through the VA's nationwide V-Tel system, which allows parties in different places to communicate through audio and video. During the twice monthly seminar series, the 10 sites connect through a prearranged number, with the hub site designated as the “host.” An integrated service digital network allows for high-speed data transfer over a telephone system, creating a visual and audio clarity that effectively simulates an in-person discussion.
The use of the V-Tel system facilitates learning and mentoring opportunities that would simply not be otherwise possible. During the twice monthly seminars, nationally recognized research specialists, invited by the hub site, present topics relevant to the core curriculum, with each seminar accompanied by supporting materials including PowerPoint presentations and relevant scientific articles. These experts also attend fellows' presentations, providing them with valuable feedback on their research projects. V-Tel's interactive format allows fellows to ask specific questions and gives them the chance to foster relationships with renowned clinical researchers and with the directors and fellows at the other sites. Each site contributes by presenting during one or more of the V-Tel seminars and the associated modules in which they have expertise. This series is open to all interested trainees at the postresidency/postdoctoral level, as well as interested junior faculty and staff. Thus, it is an educational resource that reaches beyond the MIRECC itself.
This cutting-edge technology also offers organizational efficiency. The 20 fellowship program directors hold teleconference sessions every two months to evaluate the curriculum and its implementation, and to discuss and, when necessary, adapt the core curriculum in response to shifts in academic, scientific, or pragmatic emphases.
Materials from the seminars and other local didactics are uploaded to the hub site's Web site for subsequent viewing and access. The Web site regularly disseminates materials on relevant conferences, grant mechanisms, career development institutes, and academic career job postings.
Measures of success
We feel that the productivity of our fellows and their successful transition into clinical research positions reflect the many benefits of a multisite approach. In the first five years of our program, 58 fellows have graduated, 50% of whom are women (25 MDs [19 men; 6 women] and 33 PhDs [10 men; 23 women]). Over 70% of graduated fellows have entered academic and/or clinical research positions. Since the program began, only six fellows have dropped out of the program to pursue private practice. Eight fellows advanced out of the fellowship program early: two who took positions in academia on receiving sizeable grants, two who took joint academic-VA positions, and four who took full-time VA positions. Our fellows have also obtained over 25 extramural VA or NIH grants as principal or coinvestigators (see Table 1), and they have published several hundred peer-reviewed scientific papers.
Prime indicators of success include the completion of the core curriculum, including human subjects training and ethics, the design, implementation, and completion of a successful study investigation, numbers of manuscripts submitted and published by peer review, numbers of grants submitted and obtained, and clinical credentials obtained during fellowship. Additional factors include posters and presentations at national conferences and supervisory and educational roles played by the fellows during their programs, meant to augment the individual training goals of the fellows. Presentations at national conferences are encouraged, to benefit networking and presentation experience; however, peer-reviewed publications are given the highest priority. In particular, the fellowship program emphasizes publications from the fellows' own investigations and those that contribute to the field of evidence-based medicine and mental health. The seminar series and individual mentorship focus on these endeavors. We stress that central to the success of our program is the excellence of the individual sites and local mentorship.
We are in the process of planning qualitative interviews to assess which aspects of the program the fellows and their mentors find most effective, but these data, not yet collected, are not included in this article. We have not yet compared the productivity of our fellows against that of a control group.
Advantages and challenges of a multisite approach
This multisite program has successfully exposed our fellows to an inter- and cross-disciplinary network of experts, research mentors, role models, and other collaborators who may not be available at an individual site. Exposure to national experts in diverse areas of mental health research is particularly important given the current emphasis on interdisciplinary and translational research. The multisite approach provides fellows with additional feedback and perspectives from other clinical research sites and approaches regarding all domains of scientific, clinical, and career development. The program offers a competitive stipend and benefits package, innovative, structured, and comprehensive training opportunities, and, at a majority of the sites, an office of postdoctoral affairs to help fellows throughout their fellowship years. The high rates of success, cutting across gender lines, demonstrate that multisite training, with its unique learning and networking opportunities, results in greater transfer of knowledge to greater numbers of fellows.
Implementation of a multisite training program, however, is not without its challenges. Implementation issues included reaching consensus among the 20 nationwide directors on (1) program goals and core curriculum, (2) maintaining an appropriate balance between the research focus, clinical training, and educational experiences, (3) uniform standards for assessing performance, (4) ensuring uniform quality and availability of local mentorship and resources, and (5) ensuring that technology and support systems work for the program.
To reach consensus on program goals and core curriculum, the fellowship program directors meet at least every two months and more often when necessary. These teleconference meetings allow the directors to discuss the implementation and progress of the program and current issues or problems. The directors also reevaluate the core curriculum to ensure that the appropriate balance between the research focus, clinical training, and educational experiences is maintained at all fellowship sites. They also modify the curriculum each year to accommodate the differing research emphases of the incoming fellows, as well as changing scientific methods, information, and emphases. Other modifications to the curriculum, which are approved by the consensus of the fellowship directors, the OAA, and the hub site, have included expanding the program to include physicians with residency training in areas other than psychiatry but who have a defined focus on mental health research. Issues regularly discussed in this forum also include enhancement of recruitment, deadlines and timelines for achievement of core program goals, mentorship issues, and working closely with IT departments at each of the 10 fellowship sites to ensure that the necessary technologies are available and functioning.
To ensure uniformity of performance across sites, all fellows undergo a common evaluation process that entails a semiannual review with their mentor(s) of their goals and objectives. On entry into the program, fellows collaborate with local mentors to develop individualized plans to achieve the goals and objectives of the fellowship. They are rated quarterly on their ability to integrate science and practice, that is, their ability to identify and use empirically supported assessment methods and treatments in service delivery. They must demonstrate clinical exit competencies in their specialty area, including sensitivity to ethical, legal, and diversity issues. Finally, they must show professional activity in research in their specialty by publishing, or submitting for publication, at least two coauthored papers, or by applying for a career development award or investigator-level grant during their training.
The OAA's RFA for fellowship site selection paved the way to uniform quality and availability of local mentorship and resources. However, fellows and fellowship directors must routinely address issues of local availability to ensure that fellows at all sites have maximal access to the same quality of technological, scientific, clinical, and practical resources and support. Given the extensive contact the hub site has with individual fellows, it can effectively and rapidly address any concerns raised regarding local support. The hub site director also participates in monthly meetings of VA central office and national MIRECC center directors and has used this forum for addressing concerns regarding local resource issues, with great success. The result is a large investment at all levels in the success of the fellowship program.
Despite some challenges, we feel that a multisite approach enhances the successful transition of trainees into clinical research careers, particularly in cases in which the feasibility of such training is limited by either geographic or financial constraints. The VA Special Fellowship Program in Advanced Psychiatry and Psychology has played a valuable role in preparing a significant number of well-trained clinical research scientists for a range of academic clinical research and leadership positions in the field of mental health. A uniform multisite model that combines the traditional merits of scientific apprenticeship with the modern merits of technology for multisite structure and implementation seems to be an effective and efficient approach to integrating multiple sites into a larger fellowship program.
The authors thank Emily Gere and Nichole Kryla for their administrative support. They also extend thanks to MIRECC fellowship directors and affiliates, past and present, for their commitment to the success of the program.
The writing of this manuscript was supported by the Veterans Health Administration Office of Academic Affiliations, VA Special Fellowship Program in Advanced Psychiatry and Psychology, Department of Veterans Affairs; by the Sierra-Pacific Mental Illness Research, Education and Clinical Research Center; and by National Institutes of Mental Health grants R13MH069748 and R13MH069748.