Implementing a Resident Research Program to Overcome Barriers to Resident Research

Abstract

Internal medicine residents are required to participate in scholarly activity, but conducting original research during residency is challenging. Following a poor Match at Baystate Medical Center, the authors implemented a resident research program to overcome known barriers to resident research. The multifaceted program addressed the following barriers: lack of interest, lack of time, insufficient technical support, and paucity of mentors. The program consisted of evidence-based medicine training to stimulate residents’ interest in research and structural changes to support their conduct of research, including protected time for research during ambulatory blocks, a research assistant to help with tasks such as institutional review board applications and data entry, a research nurse to help with data collection, easily accessible biostatistical support, and a resident research director to provide mentorship. Following implementation in the fall of 2005, there was a steady rise in the number of resident presentations at national meetings, then in the number of resident publications. From 2001 to 2006, the department saw 3 resident publications. From 2006 to 2012, that number increased to 39 (P< .001). The department also saw more original research (29 publications) and resident first authors (12 publications) after program implementation. The percentage of residents accepted into fellowships rose from 33% before program implementation to 49% after (P = .04). This comprehensive resident research program, which focused on evidence-based medicine and was tailored to overcome specific barriers, led to a significant increase in the number of resident Medline publications and improved the reputation of the residency program.

The Accreditation Council for Graduate Medical Education (ACGME) requires that residency training programs in internal medicine ensure that all residents participate in scholarly activity. The requirements for scholarly activity are purposefully vague—the current iteration states simply that “residents should participate in scholarly activity”—allowing programs to fulfill the requirement in a variety of ways.¹ The benefits of scholarly activity include improving resident education, promoting quality patient care, providing skills for lifelong learning, enhancing analytic skills, and developing critical thinking.²

Most program directors agree that original hypothesis-driven research fulfills the ACGME scholarly activity requirement.³ Pursuing such research in particular is important for residents’ future work in academia, and the completion of a research project has been associated with the successful application to subspecialty fellowship⁴ and higher satisfaction with residency.⁵

For many programs, however, limited resources make it impractical if not impossible for residents to perform original research. These same programs often have trouble attracting high-quality residents, many of whom desire subspecialty fellowships. This article describes one program’s approach to overcoming the barriers to resident research by reengineering the program’s culture and implementing specific changes to facilitate the successful completion of resident research projects.

The Problem and Ways to Approach It

Most programs struggle to provide residents with opportunities to conduct original research. A number of barriers have been identified by program directors³ and residents.⁶ These include limited resident and faculty time, lack of resident research skills, absence of a research curriculum, insufficient resident interest, inadequate funding, and a paucity of mentors.⁷ Other programs have implemented research rotations aimed at fostering hypothesis-driven research,^8–11 as well as structural changes such as the introduction of a resident research director (RRD).¹²

The addition of an RRD can be a powerful catalyst to resident research. They provide mentorship, oversee programmatic structure, and can match faculty researchers with interested residents, although their specific activities vary by program. In the 1990s, 42% of residency programs reported having an RRD.¹³ In 80% of those programs, the time dedicated to such a position was ≤ 0.2 full-time equivalent (FTE), and less than 5% of those programs reported devoting ≥ 0.4 FTE to the position. Durning and colleagues¹² reported that having an RRD for approximately seven hours per week almost tripled the number of resident publications per five-year period and that the number of national presentations also increased markedly.

Research electives help to overcome the most substantial barrier to resident research—limited resident time. Electives provide residents time away from patient care to focus on research. Byrnes and colleagues⁸ found that a four-month research elective increased the number of residents performing research and obtaining fellowships; they did not report on publications. A two-week structured research rotation overseen by an RRD was also successful in increasing the number of resident publications, mostly letters to the editor.¹⁰ Finally, a four-week research rotation during internship increased the number of case reports and research abstracts that residents presented.⁹ However, no program of which we are aware emphasized changing their culture or adding auxiliary personnel to assist residents with their research projects.

The Resident Research Program at Baystate Medical Center

Baystate Medical Center is an independent academic medical center affiliated with Tufts University and located in Springfield, Massachusetts. Today, the internal medicine residency program has approximately 54 categorical positions, and residents are primarily graduates of international or osteopathic medical schools. Approximately 60% apply for subspecialty fellowship training, with the remainder pursuing careers in hospital medicine or primary care.

In 2003, the residency program experienced a disastrous Match, filling only 8 of 16 positions. At the time, residents performed little or no research, evidence-based medicine (EBM) was rarely discussed, in-training exam scores were low, and board pass rates hovered in the mid-80s. Many residents who desired fellowships were unable to attain them. The Match that year served as a wake-up call for the new department chair, who together with the program director began a series of interventions to improve the outcomes and reputation of the program. As part of that effort, in 2005, Baystate applied to the Association of American Medical Colleges Educational Innovations Program and was approved to join the first round of participants in 2006.¹⁴

In October 2005, with the backing of the department chair, the program director appointed an RRD, who was a junior faculty member with fellowship training in research and approximately 15 publications. This new position included 0.25 FTE of protected time dedicated to implementing the resident research program. The RRD was tasked with increasing the number of original research projects resulting in publication and with developing an EBM curriculum. The goals of this new program were to teach residents the skills needed to critically review the literature and to conduct research, with the expectation that these changes would enhance the reputation of the program and allow residents to obtain desirable fellowships. Those not pursuing a fellowship also would benefit from learning EBM and presentation skills, and some might later engage in research as part of practice-based research networks. Because of the chair’s involvement in the process, substantial resources were made available, and the RRD was given considerable latitude in implementing the program.

At the time, EBM was not taught in a systematic way, and residents were conducting little original research, even those who were eager to include publications on their fellowship applications. Among those residents not seeking fellowship training, there was almost no interest in research. Finally, presentations at regional and national meetings consisted primarily of case vignettes.

The program director and RRD implemented the resident research program in stepwise fashion to overcome the perceived barriers to resident research, identified in the literature and through program participants’ experiences, and to build on past successes (see Table 1). Both the program and the role of the RRD have evolved over time.

Table 1: Timeline, Elements, and Outcomes of the Resident Research Program at Baystate Medical Center, 2005 to 2012

Overcoming Barriers to Resident Research

Interest

To stimulate residents’ interest, the RRD attempted to create an atmosphere of inquiry¹⁵ through EBM. He implemented an EBM curriculum, based on the Users’ Guides to the Medical Literature,¹⁶ that focused on building residents’ skills in searching the literature and critically appraising published articles; it consisted of 4 two-hour interactive workshops over two years. A daily 30-minute conference during ambulatory blocks reinforced these skills with an emphasis on asking and answering structured clinical questions. Each morning, one resident presented the answer to a structured clinical question based on a case seen in the clinic. Most questions exposed gaps in the literature, and some spawned research projects. The daily conference reinforced the regular use of EBM in practice. Residents began to look for answers in research articles rather than text books. Over the next five years, residents brought these evidence-based approaches to the hospital—their morning report was reformatted to include a daily EBM question, and they added a weekly journal club. Faculty development workshops were offered to help faculty keep pace.

At the same time, the RRD began to work one-on-one with a few residents to demonstrate that residents could complete research projects leading to publication. Initially, residents served as second authors on existing projects and were encouraged to author case reports and to present their work at national meetings, with expenses paid by the institution. Some presentations won prizes. The success of these national presentations, coupled with the resulting publications, spurred further interest in research among residents. Consequently, more residents engaged in original projects for which they qualified as first author. Interest in research was further enhanced by the creation of a research track, which met monthly for two hours. At these monthly meetings, residents received advanced instruction on topics such as working with a statistician, navigating peer review, presenting at a national meeting, and applying for grants. Residents also had opportunities to present their work to senior researchers. Track members were expected to present their work at a national meeting and submit a manuscript for publication before graduation. Finally, in the medical education office, we posted on a bulletin board the residents’ publications with their names highlighted. Across the bottom of the board, we listed as a timeline the steps for a research project and identified each resident’s project with a pin and his or her name, to allow easy tracking of all projects.

When the program was implemented, residents were required to complete a scholarly project, but the requirement was vague, no enforcement mechanism existed, and no resident was held back from graduating because of a lack of scholarly activities. Over time, the requirement became stricter. At a minimum, residents were required to present a case vignette at a regional meeting, and, by 2010, residents entering their third year without a research project were instructed to submit a case vignette to a regional meeting. Finally, during our institution’s research week, we established a resident abstract competition judged by researchers using a rubric from the Society for General Internal Medicine. Each year, the two highest-rated entrants presented their abstracts orally during grand rounds and received cash prizes.

Time

Like all residency programs, ours put numerous demands on residents and required that they complete their work within the allotted hours. Adding a research component created a competing demand for their limited time. We addressed this challenge from two directions. Like other resident research programs, we created “protected time” for residents to work on their research projects. During each of the three monthlong ambulatory blocks in their second year, all residents were eligible for one day per week dedicated to scholarly activity. To qualify, residents had to have a protocol approved by the institutional review board (IRB); otherwise, the time reverted to clinical sessions. By giving the residents longitudinal protected time, we hoped to lessen the effect of unexpected delays that could ruin a one-month research elective. Residents with IRB-approved projects also could take a separate research elective, which freed them from all clinical responsibilities except continuity clinic, to allow time for data collection or manuscript preparation.

In addition, we hired additional staff with specific skills to enhance the residents’ efficiency. For example, one time-consuming obstacle we identified was the IRB process, which was complex and often where projects stalled. To overcome this barrier, in 2007, we hired a research assistant for 20 hours per week to assist with IRB applications and data collection. She soon became an IRB expert, helping residents to complete forms correctly and make necessary changes expeditiously. She also was able to help with survey creation, data collection, data entry, and other tasks. Later, we noted that residents often failed to complete chart reviews before graduation, resulting in unfinished projects. Thus, in 2011, we hired a retired nurse for 10 hours per week to perform chart abstraction on resident projects.

Finally, residents were encouraged to design projects that could feasibly be performed given the time constraints. Preferred methodologies included surveys, chart reviews, case reports, review articles, and observational studies using administrative claims data.

Technical support

Technical support for the program was provided by Baystate to facilitate research throughout the hospital. In 2008, Baystate introduced a biostatistical core, consisting of a PhD epidemiologist and two (later three) master’s-level biostatisticians, to provide free statistical support. All residents had access to this resource, and biostatisticians were incorporated into research teams from project inception. As a result, statistical issues related to study design were addressed before data collection began. Fewer projects failed to be published because of an inadequate sample size, and data collection was cleaner, which streamlined the analyses. In addition, no projects had to be abandoned because of an inability to analyze the data effectively. Had the hospital not provided these resources, the resident research program probably could have hired a master’s-level statistician for 20 to 40 hours per week to perform these same functions.

Mentorship

Because Baystate is not a large research institution, when the resident research program was implemented, the hospital had few experienced research faculty, and most were committed to mentoring subspecialty fellows. At first, the RRD worked directly with many of the residents, although clinical faculty members were encouraged to collaborate on case reports and review articles. Later, the RRD mentored junior faculty members, and some became independent mentors for the program. As more faculty members had positive experiences, others requested to work with residents as well. We instituted a mentor dinner at which mentors pitched project ideas to interested residents and compiled an electronic project list that included each mentor’s name, proposed research question, a one-paragraph description of the study, and the resident’s role. The RRD helped match residents with mentors and mediated conflicts. Mentors frequently approached the RRD with a specific project looking for an appropriate mentee. Apart from the RRD, no faculty members had protected project time. Nevertheless, the department chair set expectations and reminded faculty that publications were required for promotion. In addition, the department offered annual academic bonuses to faculty members based on publications.

Program Assessment

Since October 2005, when the resident research program was implemented, we have seen a sharp rise in the number of abstracts presented, followed by a slow and steady rise in the number of total and first-author publications (see Figure 1). Interestingly, as the number of presentations at national meetings (e.g., the Society of General Internal Medicine meeting) rose, the number of presentations at regional meetings (e.g., regional American Colleges of Physicians meetings) declined, perhaps because residents came to value more highly national meetings. See Sup plemental Digital Appendix 1 (https://links.lww.com/ACADMED/A204) for a complete list of publications.

Figure 1: Number of regional and national presentations and total and first-author peer-reviewed publications by residents in the Baystate Medical Center resident research program. For consistency, publications are listed according to the year they appeared in print. National presentation data were not available for 2001 to 2003.

We compared the publication rate per resident per year in the five years before and six years after program implementation using piecewise Poisson regression with robust standard errors. Before implementation, the publication rate did not change significantly over time (incidence rate ratio [IRR] 0.83; 95% confidence interval [CI] 0.50–1.36); after implementation, it increased each year (IRR 1.39; 95% CI 1.28–1.51), from an average of 0.03 publications per resident before to 0.67 after implementation (P < .0001). We also found an increase in the number of original research and first-author publications. Overall, 29 (50%) of the publications involved original, hypothesis-driven research; residents contributed to 12 (21%) as the first author and 11 (19%) as the second author (see Table 2). The RRD participated in 18 of 58 projects, 7 (39%) as the first author and the rest as the senior author.

Table 2: Resident Publications by Type and Year Before (2001–2006) and After (2006–2012) Implementation of the Resident Research Program, Baystate Medical Center

Finally, the composition of the residents evolved during the study period. Consistent with the decline in U.S. medical students’ interest in internal medicine, we saw a decreasing number of residents from U.S. medical schools (see Table 3). To assess residents’ performance, we used Fisher exact test and Student t test and found no significant change in the in-training exam scores or in the percentage of residents who passed the internal medicine board exam (86% before versus 78% after, P = .14). However, after implementation, the percentage of all residents accepted to subspecialty fellowships increased from 33% (28/84) to 49% (50/103) (P = .04). In exit interviews conducted between 2007 and 2012, 80% of program applicants rated the scholarly activity requirement as positive or very positive, while less than 3% rated it as negative; 10% identified the research program as their reason for applying.

Table 3: Characteristics of Residents Before (2001–2006) and After (2006–2012) Implementation of the Resident Research Program, Baystate Medical Center

Lessons Learned

Although we describe numerous interventions in this article, we believe that three elements of the program—leadership, culture change, and excitement—stand above the others.

Leadership

One key element to the success of the program was choosing the right RRD. A successful RRD should have formal research training and broad interests to be able to collaborate with residents on the projects that interest them. Before implementing our program, collaborating with residents on research had been the responsibility of another associate program director, who was primarily an educator with no first-author publications. Although he understood the steps required to perform research, he was unable to successfully lead residents through projects and produced almost no publications. In addition, the support of the department chair was critical to protecting the RRD’s time and to funding the various time-saving initiatives. Because we successfully achieved our initial goals, additional resources were allocated in subsequent years.

Culture change

Without a doubt, our constant focus on EBM created the atmosphere that allowed the program to flourish. Culture change takes time. Initial investments in the EBM curriculum took years to come to fruition. We started with just one conference, then incrementally added elements until the entire program was suffused with EBM. The case conference was not immediately popular with all residents, many of whom did not perceive value in EBM. The conference also required a substantial time commitment from the RRD, who was personally present for at least half of the daily sessions. Still, the faculty persisted, and as new residents arrived, successively less resistance and eventually enthusiasm for the new culture resulted. Similarly, as the research requirement became more stringent, new residents readily accepted these higher expectations. Over time, residents embraced these changes and helped carry them forward and improve on them.

Excitement

Creating excitement around EBM and research encouraged residents to participate. The RRD focused his initial efforts on the projects of the most promising residents. Their presentations and publications helped generate interest among others. In addition, offers of paid travel expenses and cash prizes were helpful in generating interest. Later efforts aimed to remind residents about research from early in their internship. With all the competing demands on their time and attention and the slow pace of research, without constant reminders, residents were at risk for letting projects languish (and many did). The research track, which at first was open to anyone, was poorly attended. In 2011, we restricted membership to four residents per year and added an arduous application process. That year, half the interns applied, and attendance soared.

Value of the Program

The resident research program achieved a number of goals. It increased the residency program’s scholarly output, raising the profile of Baystate nationally. More important, it transformed the culture of the residency program and, in some ways, the hospital. By concentrating on EBM, we stimulated residents’ interest in answering questions not only for research but also for patient care. Most resident projects were related to patient care, and some affected practice. Also, filling residency slots ceased to be a concern. As the research program matured, the residency program attracted trainees with more interest in EBM, and graduating residents obtained competitive subspecialty fellowships. Finally, applicants to the residency program cited the research component as a positive aspect, and some cited it as their reason for applying. In many ways, the problems that we faced initially were solved.

Future Directions

As the resident research program matures, our focus has shifted to institutionalizing past gains while adding innovative changes. The original RRD has left Baystate, and the program continues with new leadership. Efforts are focused on improving the quality of the projects, as well as getting residents involved in research earlier in their training. A resident-initiated curriculum for writing case reports will be incorporated into attending rounds, and presentations of works-in-progress are now part of noon conferences. Possible additional enhancements include a three-month research block and increased expectations for research track members.

In conclusion, research is an important component of internal medicine training, but it is difficult for residents to conduct research successfully because of specific barriers inherent in residency training. A resident research program focused on EBM and tailored to overcoming these specific obstacles can lead to a significant increase in residents’ research activity, including peer-reviewed publications and presentations at national meetings.