The Office of Research in Medical Education (ORME) at the University of Washington (UW) School of Medicine was founded in 1967, becoming the ninth such unit serving American medical schools eight years after the first offices were founded in 1959.1–3 From the start, ORME was established to evaluate educational projects supported by grants and contracts, initially as part of a federally supported regional medical education and professional development program for the Pacific Northwest. This was a prelude to our current five-state WWAMI educational program, establishing a single medical school for the states of Washington, Wyoming, Alaska, Montana, and Idaho.4,5 Based on its initial successes, ORME attained formal “academic status” in 1980, becoming the Division of Research in Medical Education (DORME), and then in 1989, becoming the Department of Medical Education (DME), all under the direction of it's founding leader, Charles (Chuck) Dohner.2,6 Becoming a formal division allowed our unit to have and control its own budget and faculty appointments, and to offer official university-approved courses. Department status allows us to offer our own degree programs. In 2002 the unit's name changed once again, becoming the Department of Medical Education and Biomedical Informatics (MEBI). This change reflected the continuing and expanding role the department plays as an academic home for biomedical and health informatics at the University of Washington, creating increased synergies between medical education and information science and technology that began with early distance learning experiments using satellite technology as part of the WWAMI program.7
In this case study, we provide some of the history and background of the evolution of the Department's structure and its role in providing leadership in medical education research at our institution, as well as regionally, nationally, and internationally. We also offer our observations and reflections on what has helped and hindered the Department's success, and we end with some predictions on medical education research in the future. In so doing, we hope to explicitly address the invitation to describe how our “program has emerged to address the challenges, barriers, and solutions to producing high-quality medical education research,” an ideal to which we all aspire and continually strive to attain.
Background and Institutional Structure
MEBI is one of 29 academic departments within the UW School Medicine, enjoying the same privileges, responsibilities, and challenges as its sibling departments. The role of medical education research at the university is best framed within UW School of Medicine's overall mission statement, “to improve the health of the public by advancing medical knowledge, providing outstanding primary and specialty medical care to people of the region, and preparing tomorrow's physicians, scientists and other health professionals.”8 One of the institution's unique characteristics has been its ability to excel in both primary care training and in biomedical research, ranking “first among public medical schools and second among all medical schools in federal research funding,” while at the same time being an innovator in education, as evidenced by being “ranked as [the] top medical school for eleven consecutive years in training primary care physicians, and [having] top-ranked academic programs in family medicine and rural health.” The WWAMI program “is a model of community-based training of medical students and residents and of interstate collaboration in medical education.”8 This relatively unique emphasis on both high quality science and generalist primary care education serves to provide both tension and opportunity for “natural experiments,” and provides abundant “grist for the educational research mill.” While WWAMI still may be considered an experiment in progress after 30 years,5 and the institution still struggles with issues of identity and faculty rewards (and dedicated time) for teaching, it continues to not only work but also to provide a nurturing environment that increasingly rewards scholarship based on teaching and learning. Even the clinical departments that have not adopted our formal clinician–educator faculty track have found ways to reward excellence in teaching, which must be documented through scholarship as part of the university's annual review and appointment and promotion processes.
The educational mission of ORME, then DORME and DME, and now MEBI, as mentioned earlier, has been organized around meeting the data driven evolution needs of the School of Medicine and the WWAMI program. The need to demonstrate “educational equivalency” across our four- and now five-state educational program was a challenge, and a requirement for accreditation by the Liaison Committee on Medical Education (LCME), and served to create institutional need and thus resources to support educational evaluation (and thus research).9 The fact that this was done cooperatively and collaboratively with the dean's office, individual departments, and WWAMI schools contributed to the success of the early efforts,10–12 and led to PhD-educators being viewed as true colleagues and peers within the institution. Indeed, selected faculty in clinical departments request faculty appointments in MEBI, as do some of our WWAMI faculty. This exemplifies not only the importance of education as part of their personal careers, but also the value they place on collaborating with MEBI core faculty. Similarly, many of our faculty members have faculty appointments in other departments in the school.
Medical education faculty in the department played key roles in early education and training grants and contracts,13,14 collaborating with their clinical colleagues, some of whom also developed medical education research careers. Faculty and professional staff positions were created within the department to support test item writing and scoring; course, program, and teaching evaluation systems; curriculum development; faculty development; and a longitudinal database for institutional research and policy purposes. Early emphases included assisting the basic science and clinical departments in developing organ-based courses with better-organized syllabi, instructional objectives, and a system for evaluation and assessment of teaching and learning. To the credit of the faculty, they collaborated with their clinical colleagues to leverage these needed activities into scholarly work (i.e., educational research). Faculty in MEBI are appointed and serve on key curriculum and educationally related School of Medicine committees, thereby enhancing collaboration and the opportunities for scholarly projects. Indeed, the institution has an “ethic of closely monitored innovation” in which faculty in MEBI continue to play a significant role.5 In 1992, the Northwest Consortium for Clinical Assessment was established with support from the Josiah Macy Jr. Foundation, which led to the development of collaborative multi-institutional standardized patient cases to enrich students’ learning experiences.15
One additional organizational characteristic of our environment that is germane to this discussion is the policy that all centers and institutes must have department homes and not be “free standing” or reside in purely administrative units (like the dean's office or Health Sciences Administration). Thus education-related centers that were created by competing successfully for government and private foundation grants and contracts have been located within the department (as discussed below), allowing MEBI to be the educational research focal point for the School of Medicine, while at the same time supporting, and not limiting, education-based research within other departments, most of which have their own education sections.
Current Organizational Approach to Medical Education Research
Medical education research at the UW School of Medicine is decentralized, like almost all elements of our academic medical center. Thus while MEBI provides a formal academic home for medical education research, most of the clinical and basic science departments have educational programs, many in collaboration with our sister WWAMI universities. Within MEBI, medical education research is organized within the Division of General Medical Education (DGME) in general and within the Center for Medical Education Research (CMER) in particular. All in-house, non–externally funded educational research, including that around course and faculty development, competency assessment and evaluation, standardized patient and computer simulations, longitudinal tracking of our graduates, and so on, lie within the purview of DGME. CMER, an official UW center approved by the Board of Regents, began in 1994 as one of two and then three centers of excellence funded by the U.S. Health Services Research Administration. CMER has focused on policy-related medical education research16–20 and has provided the home for projects led by department faculty who are supported by the National Library of Medicine (NLM),21,22 the Agency for Healthcare Policy and Research,23 the National Institutes of Health (NIH) Fogarty International Center,24–27 and the China Medical Board.28 Department faculty also offer their educational research expertise and serve as coinvestigators on grants directed by colleagues in other departments. Some of these are academic career awards or curriculum development projects and include support from NIH and the National Board of Medical Examiners Stemmler Research Fund.29,30 Direct external grant and contract support for which department faculty are the principal investigators has grown from over $135,000 per year in 1997 to over $680,000 per year in 2002, and topping $1.2 million per year in 2003.
The department's medical education faculty has grown in number from a total of 13 faculty members in 1995 to 33 in 2004. Of these, nine are core, regular faculty members and three are either retired or emeritus who had been regular faculty, while 21 are either adjunct or affiliate faculty members, including those whose primary appointments are either in UW clinical departments or in our WWAMI-affiliated universities. The chair of the department, the head of the DGME, and the director of the CMER provide leadership at the departmental level. At the school level, our dean and the Appointments and Promotions Council value medical education and scholarship, particularly in the clinician–teacher faculty track. It is expected that individual faculty members in the department will provide leadership in medical education research and evaluation issues when serving on individual projects, committees, and general school-related activities. While research projects tend to be driven by institutional needs and timely opportunities, much of the research agenda is faculty initiated and self-directed. Our faculty's interests and activities focus on the following content domains: physician performance assessment; evidence-based medicine; physician–patient communication skills; systems for clinical decision support; case-based instruction; knowledge representation, retrieval, and analysis; health professions diversity; public health informatics; teaching and program evaluation; and competencies for primary care.31–37
Historically, the department has been able to support graduate students as research assistants on these projects, some of whom have chosen to take our graduate course offerings in medical education research, evaluation methods, teaching methods, leadership in medical education, and evidence-based health care. One of the legacies of the original goals of CMER, which included the training of medical education researchers, was the enhancement of our local medical education research infrastructure through the establishment of our Teaching Scholars Program (TSP) in 1995. The TSP continues to be widely popular and has graduated over 70 faculty in its certificate program, many of whom have gone on to leadership roles in their departments or in the dean's office. Scholars are expected to complete at least one scholarly research project during the yearlong program.38–42
Mandate for Medical Education Research at the University of Washington
As summarized in the school's report in the supplement to Academic Medicine entitled “A Snapshot of Medical Students’ Education at the Beginning of the 21st Century,” the department has a clear institutional mandate for research: “Today, the department's mandate has been broadened to include: providing instruction to faculty and students through courses, faculty development workshops, performance-based teaching, and training medical education researchers and educators in the Teaching Scholars Program; conducting research into teaching and learning in a variety of settings; providing professional services to health sciences faculty, including course evaluation and test design and scoring; [and] working in the medical school to develop innovations in biomedical and health informatics.”1 As a formal academic department, MEBI's metrics of success include traditional measures of scholarly and research productivity: grant and contract support, scholarly publications and presentations, chapters and books, awards, and representation on national and international efforts of importance.
The School of Medicine, the other health sciences schools, and the university at large have been supportive of the department becoming the academic home for biomedical and health informatics on campus. Our new multidisciplinary masters and PhD degree programs, supported by state funds, are organized in collaboration with the schools of public health and community medicine, nursing, pharmacy, and dentistry, as well as the colleges of engineering and arts and sciences, and the information school. Research, including that related to informatics and education, is a key component of this program, and graduate students and fellows, many of whom are supported on our NLM Training Grant, provide a rich source of research energy and creativity. In addition, all of our medical students have an Independent Investigative Inquiry requirement in which they need to complete a research scholarly project before graduation. This includes survey research and systematic evidence reviews of the literature, in addition to bench and clinical research projects, and have included educationally related projects. Our dean provides institutional leadership for this program and takes pride in the fact that we have the largest annual representation of medical student research presentations at the American Federation for Medical Research Western Regional Meeting (〈http:www.afmr.org/〉), with many of our students receiving awards for outstanding oral and poster presentations.
Finally, one of the original historical missions of the department has been to maintain the institutional database for our students, residents, and graduates. Studies are done both for institutional and policy-related purposes, and include legislative and LCME reports, as well as to produce scholarly publications. In summary, specific measures of success include research related to the problems and issues associated with teaching and learning in the training of physicians and related health care workers; research funding, especially that related to the large educational WWAMI experiment that was initially provided by the Commonwealth Fund; and generating data to assist administrators in making significant program decisions and in the LCME accreditation process to document the significance, success, and validity of the WWAMI experiment.
Among the barriers to success are the usual problems of capturing sufficient support to protect (or “buy”) dedicated faculty time and space for research, as resources for medical education research are limited. Another limitation is the difficulty in conducting high-quality educational research in general. This includes difficulty in asking the most significant questions at the right time, using rigorous research methods, including the identification of appropriate control groups, defining and measuring important outcomes, especially patient-related care outcomes, and attributing effects and outcomes to “educational” causes given the complexities in causal attribution across the medical education continuum from premed through continuing medical education.26
Forecasting Future Issues for Medical Education Research
There will continue to be a need for higher-quality research and evaluation as the medical education curriculum evolves and adapts to incorporate changes in demography and health care delivery (including multidisciplinary care, patient empowerment, and shared decision making), advances in science, and innovations in technology. New areas of discovery involving the new biology of genomics and proteomics may motivate innovation in teaching and learning, as will innovation in the information sciences and informatics. Increasing globalization in all aspects of life, including health care, medicine, and education, will likely affect all our institutions and the field as a whole. For example, efforts are being made to develop global minimal standards for medical education and “virtual” medical education using distance-learning approaches.28 Demographic changes, both within the health professions and in the population as a whole, will affect the workforce and delivery of care as more women and ethnic minorities enter the profession and the cultural makeup of the population becomes more diverse. Physician–patient interactions and “professionalism” will become important educational and research issues as a result of demographic changes and stresses in health care delivery (e.g., pressures for reduced time per patient encounter to meet productivity goals). These will present new educational and training issues that will necessitate careful monitoring and provide opportunities for careful analysis.
Methodologically, continuing efforts to advance the quality of research in medical education will be needed, including calls to use more rigorous qualitative and quantitative research methods, to incorporate continuous quality-improvement methods in education, to make education more “evidence based,” and to develop a national agenda and support for medical education outcomes research.43,44
1.Hunt DD, Ferguson B, Ketchell DD, Wolf FM, Ramsey PG. University of Washington School of Medicine. In: Anderson MB (ed). A snapshot of medical students’ education at the beginning of the 21st century. Acad Med. 2000;75(9 suppl):S395–S397.
2.Department of Medical Education and Biomedical Informatics, University of Washington Web site 〈http://www.dme.washington.edu
/〉. Accessed 15 May 2004.
3.Rosinski E. The Society of Directors of Research in Medical Education: A Brief History. San Francisco: University of California, San Francisco, 1988.
4.Schwarz MR. Medical education and medical care - underserved areas. WAMI: a concept of regionalized medical education. J Med Educ. 1973;48:116–8.
5.Ramsey PG, Coombs JB, Hunt DD, Marshall SG, Wenrich MD. From concept to culture: the WWAMI program at the University of Washington School of Medicine. Acad Med. 2001;76:765–75.
6.Irby DM, Wilkerson L. Charles W. Dohner, PhD: an evaluator and mentor in medical education. Adv Health Sci Educ. 2003;8:63–73.
7.Schwarz MR, Schaad DC, Evans FW, Dohner CW. Communications satellites in health education and health care provision. JAMA. 1983;250:636–9.
9.Cullen TJ, Dohner CW, Striker GE, Schwarz MR. Evaluating student performance in a decentralized basic science program. J Med Educ. 1976;51:473–7.
10.Ramsey PG, Carline JD, Inui TS, Larson EB, Logerfo JP, Wenrich MD. Predictive validity of the American Board of Internal Medicine certification process. Ann Intern Med. 1989;110:719–26.
11.Carline JD, Inui TS, Larson EB, Logerfo JP, Ramsey PG. The knowledge base of certified internists: relationships to training, practice type, and other physician characteristics. Arch Intern Med. 1989;149:2311–3.
12.Hunt DD, MacLaren CF, Scott CS, Chu J, Leiden LI. Characteristics of dean's letters in 1981 and 1992. Acad Med. 1993;68:905–11.
13.Scott CS, Leaf D, Neighbor WE, Schaad DC, Brock DM, Van Citters RL. Preventive cardiology education and practice in residency training: residents’ attitudes, perceptions and practice. Am J Prev Med. 1990;6(suppl):60–9.
14.Phillips TJ, Rosenblatt RA, Schaad DC, Cullen TJ. The long term effect of an innovative family physician curricular pathway on the specialty and location of graduates of the University of Washington. Acad Med. 1999;4:75–8.
15.Scott CS, Barrows HS, Brock DM, Hunt DD. Clinical behaviors and skills that faculty from 12 institutions judged were essential for medical students to acquire. Acad Med. 1991;66:106–11.
16.Ambrozy DM, Irby DM, Bowen JL, Burack JH, Carline JD, Stritter FT. Role models’ perceptions of themselves and their influence on students’ specialty choices. Acad Med. 1997;72:1119–21.
17.Brock D, Schaad DC, Guo J, Irby DM, Hunt DD, Sampson W. The Washington primary care interest inventory: reliability, validity, and predictive capabilities. Acad Med. 1998;73:1299–1304.
18.Carline JD, Patterson D, Davis L. Enrichment programs for undergraduate college students intended to increase the representation of minorities in medicine. Acad Med. 1998;73:299–312.
19.Smith CS, Irby D. The roles of experience and reflection on ambulatory care education. Acad Med. 1997;72:32–5.
20.Ellsbury KE, Burack JH, Irby DM, et al. The shift to primary care: emerging influences on specialty choice. Acad Med. 1996;71(10 suppl):S16–S18.
21.Wolf FM, Friedman CP, Elstein AS, et al. Changes in diagnostic decision-making after a computerized decision support system consultation based on perceptions of need and helpfulness: a preliminary report. J Am Med Inform Assoc. 1997;4(suppl):263–7.
22.Friedman CP, Elstein AS, Wolf FM, et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multi-site study of two systems. JAMA. 1999;282:1851–6.
23.Mauksch LB, Hillenburg L, Robins L. Establishing focus: a training protocol for collaborative agenda setting in the medical interview. Fam Sys Health. 2001;19:147–57.
24.Davis DA, O'Brien-Thompson MA, Freemantle N, et al. The impact of formal CME: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867–74.
25.Guevara JP, Wolf FM, Grum CM, Clark NM. Effects of asthma self management educational interventions in children and adolescents: a systematic review and meta-analysis. BMJ. 2003;326:1308–9.
26.Wolf FM. Lessons to be learned from evidence-based medicine: practice and promise of evidence-based medicine and evidence-based education. Med Teach. 2000;22:251–9.
27.Wolf FM, Shea JA, Albanese MA. Toward setting a research agenda for systematic reviews of evidence of the effects of medical education. Teach Learn Med. 2001;13:54–60.
28.Zhang X, Li X, Wan X, Zeng C, Scott CS, Schaad DC. Attitudes of Chinese medical students toward the IIME Global Minimum Essential Requirements. Teach Learn Med. 2004;16:139–44.
29.Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol. 2001;184:1462–8.
30.Mandel LP, Lentz GM, Goff BA. Teaching and evaluating surgical skills. Obstet Gynecol. 2000;95:783–5.
31.Schaad DC, Walker EA, Wolf FM, Brock DM, Thielke S, Oberg L. Evaluating the serial migration of an existing required course to the World Wide Web. Acad Med. 1999;74(suppl):S84–S86.
32.Brock DM, Ballweg R, Jenkins L, Wick KH. Web-based collaborative exercises: lessons learned in conducting on-line discussion forums. Perspect Phys Assist Educ. 2001;12:227–31.
33.Scott CS, Schaad DC, Mandel LS, Brock DM, Kim S. Information and informatics literacy: skills, timing and estimates of competence. Teach Learn Med. 2000;12:85–90.
34.Kim S, Schaad DC, Scott CS, Robins L, Astion ML. A longitudinal evaluation of an educational software program: a case study of Urinalysis-Tutor. Acad Med. 2001;76:1136–43.
35.Robins L, Braddock CH III, Fryer-Edwards KA. Using the American Board of Internal Medicine's “Elements of Professionalism” for undergraduate ethics education. Acad Med. 2002;77:523–31.
36.Curtis JR, Engelberg RA, Wenrich MD, et al. Studying communication about end-of-life care during the ICU family conference: development of a framework. J Crit Care. 2002;17:147–60.
37.Ralston JD, Revere D, Robins LS, Goldberg HI. Patient experience with a diabetes support program based on an interactive electronic medical record: a qualitative study. BMJ. 2004;328:1159–62.
38.Pinsky LE, Irby DM. “If at first you don't succeed”: using failure to improve teaching. Acad Med. 1997;72:973–6.
39.Mao C. Teaching residents humanistic skills in a colposcopy clinic. Acad Med. 2002;77:742.
40.Mann T, Colven R. A picture is worth more than a thousand words: enhancement of a pre-exam telephone consultation in dermatology with digital images. Acad Med. 2002;77:742–3.
41.Dee KE. MammoEd: digital interactive breast imaging education. Med Educ. 2002;36:1103–4.
42.Kempainen RR, Migeon MB, Wolf FM. Understanding our mistakes: a primer on errors in clinical reasoning. Med Teach. 2003;22:177–81.
43.Wolf FM. Methodological quality, evidence, and research in medical education (RIME). Acad Med. 2004;79(10 suppl):S68–S69.
44.Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med. 2004;79:955–60.