Coates, Wendy C. MD; Love, Jeffrey N. MD; Santen, Sally A. MD; Hobgood, Cherri D. MD; Mavis, Brian E. PhD; Maggio, Lauren A.; Farrell, Susan E. MD, EdM
Dr. Coates is chair, Acute Care College, and professor of clinical medicine, David Geffen School of Medicine, University of California–Los Angeles (UCLA), Los Angeles, California, director of medical education, Harbor–UCLA Department of Emergency Medicine, Los Angeles, California, and investigator, Los Angeles Biomedical Research Institute, Harbor–UCLA, Torrance, California.
Dr. Love is residency program director and associate chair of academic affairs, Department of Emergency Medicine, Georgetown University Hospital/Washington Hospital Center, Washington, DC, and professor of emergency medicine, Georgetown University School of Medicine, Washington, DC.
Dr. Santen is associate professor, Office of Medical Education and Student Affairs and Department of Emergency Medicine, Emory School of Medicine, Atlanta, Georgia.
Dr. Hobgood is associate professor and vice chair, Department of Emergency Medicine, and chief, Division of Education, Department of Emergency Medicine, University of North Carolina School of Medicine, Chapel Hill, North Carolina.
Dr. Mavis is director, Office of Medical Education Research and Development, Michigan State University College of Human Medicine, East Lansing, Michigan, and chair, Medical Education Research Certificate Program Steering Committee, Association of American Medical Colleges, Washington, DC.
Ms. Maggio is medical education librarian, Stanford University School of Medicine, Stanford, California.
Dr. Farrell is assistant professor of medicine, Harvard Medical School, Boston, Massachusetts, education specialist, Partners Healthcare Office of Graduate Medical Education, Boston, Massachusetts, and faculty consultant, Center for Teaching and Learning, Harvard Medical School, Boston, Massachusetts.
Please see the end of this article for information about the authors.
Correspondence should be addressed to Dr. Coates, Department of Emergency Medicine, Harbor–UCLA Medical Center, 2000 West Carson Street, Box 21, Torrance, CA 90509; telephone: (310) 222-3501; fax: (310) 212-6101; e-mail: firstname.lastname@example.org.
Medical educators face the task of providing their learners with innovative and high-quality curricula for implementation both in the classroom and at the bedside, in an effort to inspire and promote self-directed learning. Many universities have created an academic advancement and promotion series for clinician–educators whose primary function is grounded in the education of learners. This advance is based on the work of Boyer,1 Glassick and colleagues,2 and Glassick,3 who categorized academic scholarship into four categories that were worthy of academic advancement (application, discovery, integration, and teaching), and on the subsequent work of others.4–8 Faculty members whose primary role falls into Boyer's classification as the “scholarship of teaching” have a unique skill set that is worthy of consideration for academic advancement and promotion on the clinician–educator pathway.1 For faculty on this pathway, academic advancement includes scholarship grounded in their role as educators. To ensure that the curricula being developed are effective and to assess learner characteristics and opinions, it is important to study medical education interventions systematically. By using standard principles of medical education research, educators can design and test programs to optimize them and can assess outcomes. Training educators to conduct high-quality medical education research can be accomplished in a number of ways. Some institutions have designed specialized programs in faculty development to train educational leaders who are proficient in teaching, curriculum design, feedback and evaluation, and techniques of medical education research.9–12 Institutional support of researchers promotes the development and advancement of faculty with these skills. For example, the University of Michigan has a Department of Medical Education13 and a formal Medical Education Scholars Program. Other universities offer teaching scholars programs14,15 in which faculty receive specialized instruction and support for their academic endeavors.9–12 Long-term benefits result from these programs, as measured by increases in academic productivity, grant funding, and promotion.9,11,12,15,16
Formation of a National Program to Address Medical Education Research Skills
As part of the strategic planning process, the Association of American Medical Colleges (AAMC) Group on Educational Affairs (GEA) identified “advancing the careers of medical educators through skill development and mentoring” as one of its three major areas of focus.17 The AAMC's section on Research in Medical Education (RIME) developed the Medical Education Research Certificate (MERC) program in response to this need for research skills training. MERC's goal was to provide a curriculum to help medical educators acquire and enhance their medical education skills so that they could effectively collaborate with more experienced researchers and could be better consumers of medical education scholarship. The target audience for the MERC program included medical educators, many of whom were also clinicians and were less familiar with the techniques and approaches common to educational research. In November 2002, at the RIME Conference at the AAMC annual meeting, the Professional Development Committee reported on its initial MERC program planning. The members of the committee envisioned a curriculum delivered as a series of workshops at the beginner, intermediate, and advanced levels, to be offered at regional and national medical education meetings. By the summer of 2004, the identity of MERC came into being as a six-workshop series leading to a certificate. MERC was included as a specific initiative under the GEA strategic planning document, called “Agenda for Action.”17 Three workshops were offered under the MERC name at the AAMC annual meeting in November 2004, and they were followed by additional workshops at each of the regional meetings in the spring of 2005. Since then, MERC has offered four to five workshops at the national meeting each year, as well as two workshops at each of the four regional meetings. To qualify for a certificate, participants had to complete six workshops of their choice from the eight available. MERC has expanded to meet a growing demand for educational programs to enhance the research skills of educators involved in medical student and resident education. Several medical schools have incorporated MERC workshops into their faculty development and educational leadership programming. In addition, medical specialty organizations and health care systems have integrated the MERC workshops into their infrastructure to support educational scholarship. As of 2009, there are more than 60 graduates of the MERC program.18
Typically, the participant who enrolls in a series of MERC workshops works independently through the seminar series. During standard breakout sessions, each MERC participant is challenged to apply what he or she is learning by creating a plausible solution to each query. In theory, the student can choose a topic of interest and receive input from the workshop instructor and other participants to refine his or her research project. After the workshops, the MERC participant returns home to continue independent work on the project.
The Need for Collaboration
Two frequent criticisms of research in medical education are that many studies are limited by a small sample size due to their being sited at a single institution and that they often are inadequately funded.19 When published medical education research was measured objectively on the Medical Education Research Study Quality Instrument, studies that received funding of $20,000 or more were of higher quality than were unfunded or underfunded studies.20 Factors that Reed and colleagues20 cited for the latter studies included a lack of academic rigor, a noncontrolled design, and conduct of the study at a single institution. Thus, it is important to find ways to promote collaborative research networks in medical education.
Small programs within a given institution can provide useful training in increasing collaboration in medical education research. For example, at the Mayo Clinic, an ongoing medical education research group was formed.21 Members collaborated on group projects and also provided peer support and advice for independent projects. They noted increased productivity after they initiated the program. Collaborative training in research also takes place during an infectious diseases fellowship program at the John H. Stroger Hospital of Cook County and Rush University Medical Center (Chicago, Illinois), where first-year fellows work together to learn medical education research techniques and to design and carry out a single project. Faculty members noted that the collaboration played a strong role in fostering peer learning and that its benefit extended into improved success in clinical activities.22
A small sample size is frequently cited as the reason that a particular study cannot be generalized to other populations. Training several scholars of teaching within a given specialty in medical education research techniques could facilitate multiinstitutional collaborations that would allow the study of large, diverse populations. Fostering such a collaborative system would enable educators with various skill sets to work together to design, test, refine, and share educational research ideas and programs. With this collaborative project, we aimed to invest in the development of scholars of teaching across a single discipline and to promote multiinstitutional collaboration through the design of the MERC at the Council of Emergency Medicine Residency Directors (CORD) program.
Development of a Mentored, Collaborative Approach to the MERC Program
Medical educators in the specialty of emergency medicine (EM) recognized that there were faculty members who excelled in the scholarship of teaching, but they had a relatively low rate of peer-reviewed publications of medical education research. In response, the steering committee (which consisted of all of us authors and one additional person who had to reduce her participation and who could not be listed as an author) developed a novel application of the MERC program in which our specialty organization, CORD, sponsored a MERC workshop series for established medical educators in EM, called the MERC at CORD program (Figure 1). Our mentored workshop focused on teamwork in learning and in the application of the concepts presented by the MERC workshop directors. The members of CORD who participated in our course have substantial experience in teaching, and many hold positions of national prominence in academic EM, but most lack formal training or experience in medical education research.
To address the need for developing multiinstitutional collaboration and establishing a long-lasting network of medical education researchers, the steering committee created a mentor network comprising EM faculty physicians with advanced training or substantial experience in medical education research to design this application of MERC. The reinforcing principle for this method was to create a forum in which groups of medical educators could learn and collaborate on a single project under the guidance of a mentor. Initially, these mentors, all members of the planning committee, examined the existing MERC curriculum and the perceived needs of EM educators to determine how best to relate the two. For the initial workshops, we identified key MERC topics that would meet essential learning needs of all participants. An integral component of the workshops was that each small group focused on the learning objectives of the MERC lecture under the guidance of a group mentor. We offered our workshop as a unique approach to MERC certification that would allow each participant in the program to affiliate with a collaborative group according to his or her individual expertise and similar interests.
Group topics were identified on the basis of the preferences of the participants in a preworkshop e-mail that asked about their areas of interest for research topics. Responses featured a large number of narrow topics that were of specific interest to individuals, but themes emerged, and we were able to identify four broad topics: (1) learning styles and settings, (2) simulation, (3) evaluation, and (4) clinical performance. Participants self-selected to one of the topic-defined groups before the seminar, which was held in March 2009 as an offering before the CORD Academic Assembly. The first three MERC workshops were titled “Getting started in medical education research/formulating research questions and designing studies,” “Searching and evaluating the medical education literature,” and “Hypothesis-driven research” (Figure 2). In the fall of 2009, we offered the next three workshops to the same participants: “Overview of qualitative data collection methods,” “Questionnaire design and survey research,” and “Measuring educational outcomes with reliability and validity” (List 1). Successfully taking all six workshops will allow the participants to complete the formal process for MERC certification while also completing a multicentered, collaborative, mentored medical education research project.
Report on the Initial Program
The program was made available to a maximum of 35 participants. The course was advertised in conjunction with the 2009 CORD Academic Assembly Annual Meeting, and it received many more inquiries for registrations than space would allow. Two MERC workshops were scheduled on the day before the Academic Assembly in the same venue, and the final workshop took place on the first morning of the scheduled meeting.
The format of the workshop
On the first morning of the MERC workshop series, but before the start of the formal seminar and lectures, participants in each topic group gathered with their mentors to introduce themselves and to describe their educational settings, roles, and levels of experience in medical education and research. They began to brainstorm ideas for further investigation within their topic area. The formal MERC session began with the module “Getting started in medical education research/formulating research questions and designing studies.” During this workshop, several breaks in the program provide participants with opportunities to formulate questions and apply the principles being taught. Rather than each participant working in isolation, the workshop participants worked within the group and with their mentor during these blocks of time to solve each task in a collaborative manner. The second workshop was “Searching and evaluating the medical education literature,” during which the participants in each group searched together for specific content that was pertinent to their group's research question. The mentors and the session facilitators were available to assist participants in finding appropriate background materials. All of the groups benefited from hearing other groups' solutions to the task, as related by those groups' facilitators.
The final workshop, “Hypothesis-driven research,” took place on the second morning. By this time, each group had participated in meaningful conversations about their proposed research question, and participants had an informal understanding of their groups' goals. This particular session provided groups with the structure they needed to formalize their study hypothesis and set a preliminary research plan into action. During the session, the facilitator invited each working group to present its study hypothesis to the entire workshop class for comments and suggestions. The facilitator noted that the groups in this workshop took longer to respond to standard questions such as “What is your research objective?” than he had noted in his earlier experience of teaching the MERC course to individuals (Ernest Yoder, MD, PhD, St. John Health, Southfield, Michigan, personal communication, March 7, 2009). We postulate that, because the questions being posed applied to a real research project, the participants in each group invested much more time in perfecting their theoretical study hypothesis, null hypothesis, etc. It is possible that this difficulty in answering the questions differs from the response of the individual who participates in the MERC workshops in the standard format. Those participants generally attend the MERC workshops to gain the knowledge delivered in the workshops, and they subsequently return to their institutions to develop and test their research hypothesis. Therefore, they do not have the need to finalize the components of their research project on the day of the lecture.
After the conclusion of the first three MERC workshops, the participants left with an active project. In addition, the groups assigned specific research duties to their members. A leader was chosen within each group to manage the progress of the project; the leader would theoretically be identified as the first author of any resulting manuscript. The group leaders did not all choose the same approach to ensure the progress of their projects. For example, some groups decided to continue their collaboration by using e-mail, conference calls, or file-sharing programs such as PBwiki (http://pbwiki.com). At the time of this writing, all of the groups and their mentors are actively working on their long-term projects. Funding for these projects would have been desirable, but none was available.
The role of the mentors
Mentors were actively involved in the design of the MERC at CORD program. Through a series of conference calls and collaborative, creative work, the mentors identified important goals and objectives for the program. As previously mentioned, we hoped to create a network of medical education researchers within our specialty, and we focused on skills that would benefit our target audience of experienced clinician–educators. Emergency physicians are trained to work well in teams and to effectively synthesize new information. We hypothesized that the participants would benefit most from a comprehensive overview of medical education research topics and would be amenable to working under the guidance of colleague–mentors who would provide advice and information on areas that were yet to be covered in this educational process. For example, after the four broad topic categories were determined, mentors considered ways in which each topic could be addressed. Each mentor held research ideas in reserve in the event that his or her individual group was unable to agree on a project. We felt that it was important for each participant to benefit from the process of generating a research question as well as a workable project from which to learn the basic tenets of medical education research.
All of the mentors met together, both before and after each of the MERC workshop sessions, to share information on the progress of their research groups. The agenda for these meetings included a debriefing session in which each mentor reported on his or her group's activities, progress toward meeting the session's objectives, and interactive dynamics. One benefit of this format was the ability of the mentors to provide support to one another in addressing any challenges that arose in their individual research groups. A series of “best practices” began to take shape, and mentors could choose to apply effective strategies that others had discovered to ensure forward momentum within their groups.
On the morning of the first workshop, each mentor met with his or her group to facilitate personal introductions and to explain the process of this application of MERC. Participants were informed of the nature of the mentored groups at the time they registered for the MERC at CORD program and were told that this experience would be different from that of pursuing MERC certification independently at the national AAMC meeting or at regional GEA meetings.18 The members of each group sat together in the lecture hall to facilitate communication during breakout sessions. During these sessions, each mentor participated with his or her working group to facilitate discussion and to give specific input, as needed. For example, as a group discussed the various aspects of the intended study's design, the mentor could offer the group directed guidance on methodological processes relevant to their questions of interest. Because the participants were embarking on the MERC certification process, they lacked knowledge in some research methods that were critical to the designing of effective medical education studies. Mentors were able to provide necessary information and advice. In addition, mentors were able to advise the groups on the practical implementation of the group research in areas such as the creation of a reasonable timeline, the assignment of duties to carry out the study, and the conduct of a preliminary discussion of authorship criteria.
Some of the mentors may have felt that the research design their group proposed could have been improved, but we were careful to allow the groups to make their own decisions, after suggesting strategies that would optimize their studies. In the end, mentors deemed each project feasible and then sought to channel each group's energy into conducting the study that would answer their research question. Once the project was set, the mentor's role was to follow its progress and make helpful suggestions, answer questions, and ensure that the project was moving forward smoothly. During the implementation phase, the mentors were not active participants who enrolled their own subjects at their institutions; rather, they continued to serve in an advisory capacity to solve problems and provide answers as the research progressed. Mentors provided useful articles and resources for the groups to consider when performing background research for their projects. In addition, the mentors were able to assist the identified principal investigator in managing the research group. Mentors were also available to work with individual members of the research group who needed assistance in carrying out their role in the project. For the writing phase, the mentors provided insight into data interpretation, served as a resource on the writing of the manuscripts, and provided intense editorial support.
The ability of the mentors to continue their own collaborative networking as their individual groups progress enables them to provide support to one another and to objectively analyze how each individual group's progress can be supported or improved.
Expected Benefits of the MERC at CORD Program
Medical educators in EM are clearly interested in improving their skills in the area of medical education research. Educators are engaged in didactic and clinical teaching at their own institutions, and they develop curricula to meet the needs of their learners. Yet, it is likely that most of these curricular elements are not tested in a rigorous manner. It is possible that improved inquiry through the use of medical education research techniques could help optimize these teaching materials.
As previously noted, the highest-quality medical education research receives funding and takes place across multiple institutions.19,20 Intuitively recognizing these principles, EM educators identified the need for training in educational research methodology and have allied themselves with the AAMC MERC program experts to offer a unique course that is both educational and collaborative. In addition, if these researchers are to contend for the limited funding for education research, it is imperative that their skills are sound and that they demonstrate success in implementing their projects and publishing their reports. Critical evaluation of educational strategies in EM makes it more likely that high-quality medical education research will take place.
This collaborative method of forming a research network of medical educators in one specialty will, we hope, extend beyond the confines of this individual workshop by identifying persons with similar interests and establishing a consortium within which to conduct more-robust studies that will contribute to the medical education literature. The fact that the demand for the course far exceeded the available space indicates a perceived need and desire on the part of EM educators to improve their skills in medical education research. No matter whether each individual goes on to become an active researcher, the MERC advisory panel's objective of creating better consumers of medical education scholarship will be achieved. We expect that the graduates of the MERC at CORD program will need continued mentorship as they gain experience in conducting medical education research and that this course will provide a foundation for further learning, as well as links to mentors and collaborators.
Our program can serve as a model for other groups who would like to engender collaboration among their members. The principles set forth in each of the MERC workshops are generalizable to medical education research across all medical and administrative specialties. Therefore, several possibilities exist that would benefit from this model. Some groups who might choose to collaborate are undergraduate medical educators from a medical school, residency directors from all disciplines at one training site, providers of continuing medical education courses, fellows at a given medical center, or medical educators in any of the medical specialties. Groups could be formed through national collaboratives, as in this study, or in the more intimate setting of a single institution.
One limitation of our program, which is not typically an issue with the standard MERC program, is that individual participants were not able to conduct their own, individually designed projects as part of the workshop series. We feel that this is not a serious limitation, because each participant will be able to experience the full scope of the process from hypothesis generation to publication and will take part in a mentored project in their area of interest. Participants still have the freedom to complete independent projects at their own institutions by using the principles that they learned in the mentored MERC at CORD program. Thus, although each participant may not have been able to perform an independent research project at the MERC workshop, the main objectives—to learn medical education research skills and to develop a network of colleagues with similar interests, with whom they will be able to collaborate in the future—were met.
Just as individual institutions have noticed an improvement in scholarship when resources are dedicated to training faculty in educational scholarship,9–12,15,21 we hope to promote a rise in the quality of scholarship and in the future ability of clinical educators to secure funding to ensure the success of their research. The development of alliances among educators interested in medical education research and the identification of qualified mentors who can serve as consultants or members of future collaborative groups can foster a network of programs for learners and can expand the pool of qualified educational scholars for individual institutions and the specialty.
We intend to monitor longitudinally the individual academic progress of each of the participants in the MERC at CORD workshop by tracking their peer-reviewed research publications in the literature before and after they attended the workshop and completed the mentored research project. We hope to solicit feedback from the participants on the potential benefits of the program and on its effect on their ability to conduct and publish medical education research. We also plan to ascertain whether the experience led to increased collaboration among researchers at various institutions. In addition, the mentors hope to refine the best practices that were identified during the planning, implementation, research, and writing phases of the program so that they can be applied at future MERC at CORD workshops or by any organization that hopes to implement a similar mentored research network at their institution or within their specialty. It would be interesting to compare the opinions of our mentored, specialty-specific group participants in the MERC workshop series with those of independent registrants from multiple specialties who attained MERC certification via the standard route.
One of us (B.E.M.), who also serves on the MERC steering committee and who was involved in the development of the MERC workshop series, noted that, in the initial discussions of the ideal MERC workshop certification process, a mentored research project was intended as a capstone experience. However, the practical constraints associated with launching the MERC program and meeting the overwhelming demand to offer workshops to medical educators precluded the implementation of this requirement for certification. At the time of the program's development, the steering committee perceived that the task of recruiting potential mentors for each participant would be overwhelming. Moreover, practical obstacles were associated with coordinating and monitoring that proposed requirement. Rather than delay the implementation of the MERC program altogether, the steering committee decided to postpone requiring mentored research for MERC certification.
We used the well-established MERC program devised by the AAMC GEA, whose mission was to improve medical education research skills and to enhance scholarship among medical educators, as the basis for the development of a specialty-specific, research-focused curriculum for faculty development. We facilitated the creation of a consortium of potential researchers within a specialty organization of established medical educators who previously had little research experience. We provided these educators with mentorship by medical education researchers who also were board-certified practitioners within the specialty. The mentors facilitated collaborative research projects among participants. This curriculum can advance knowledge and increase the overall quality of medical education research and scholarship while fostering a network of potential researchers in EM.
The authors wish to thank Sharon Muret-Wagstaff, PhD, for her contributions to the design of the MERC at CORD program.
The Council of Emergency Medicine Residency Directors provided administrative support for the planning phase and workshops.
The authors all were members of the steering committee that developed the MERC at CORD program. There was one additional member of the steering committee who is not an author.
1Boyer EL. Scholarship Reconsidered: Priorities of the Professoriate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching; 1990.
2Glassick CE, Huber MT, Maeroff GI. Scholarship Assessed: Evaluation of the Professoriate. San Francisco, Calif: Jossey-Bass; 1997.
3Glassick CE. Boyer's expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med. 2000;75:877–880.
4Fincher RM, Simpson DE, Mennin SP, et al. Scholarship in teaching: An imperative for the 21st century. Acad Med. 2000;75:887–894.
5Beattie DS. Expanding the view of scholarship: Introduction. Acad Med. 2000;75:871–876.
6Dauphinee D, Martin JB. Breaking down the walls: Thoughts on the scholarship of integration. Acad Med. 2000;75:881–886.
7Shapiro ED, Coleman DL. The scholarship of application. Acad Med. 2000;75:895–898.
8Coates WC, Hobgood CD, Birnbaum A, et al. Faculty development: Academic opportunities for emergency medicine faculty on education career tracks. Acad Emerg Med. 2003;10:1113–1117.
9Armstrong EG, Doyle J, Bennett NL. Transformative professional development of physicians as educators: Assessment of a model. Acad Med. 2003;78:702–708.
10Muller JH, Irby DM. Developing educational leaders: The teaching scholars program at the University of California, San Francisco, School of Medicine. Acad Med. 2006;81:959–964.
11Gruppen LD, Frohna AZ, Anderson RM, Lowe KD. Faculty development for educational leadership and scholarship. Acad Med. 2003;78:137–141.
12Frohna AZ, Hamstra SJ, Mullan PB, et al. Teaching medical education principles and method to faculty using an active learning approach: The University of Michigan Medical Education Scholars Program. Acad Med. 2006;81:975–978.
13Gruppen LD. The Department of Medical Education at the University of Michigan Medical School: A case study in medical education research productivity. Acad Med. 2004;79:997–1002.
14Irby DM, Hodgson CS, Muller JH. Promoting research in medical education at the University of California, San Francisco, School of Medicine. Acad Med. 2004;79:981–984.
15Wilkerson L, Uijtdehaage S, Relan A. Increasing the pool of educational leaders for UCLA. Acad Med. 2006;81:954–958.
16Irby DM, Hekelman FP. Future directions for research on faculty development. Fam Med. 1997;29:287–289.
19Reed DA, Kern DE, Levine RB, Wright SM. Costs and funding for published medical education research. JAMA. 2005;294:1052–1057.
20Reed DA, Cook DA, Beckman TJ, et al. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–1009.
21Beckman TJ, Lee MC, Ficalora RD. Experience with a medical education research group at the Mayo Clinic. Med Teach. 2009;31:518–521.
22Madariaga MG, Evans AT, Brobbey W, et al. Learning by doing: Developing fellows' academic skills through collaborative research. Med Teach. 2006;28:77–80.