Medical education research is underfunded, and support for medical education reform has been sporadic at best. This observation has been made many times in print,1–4 and perhaps even more frequently by those seeking support. This problem is not limited to medical education research. Other health professions5 and the entire field of education itself6 suffer from the same lack of support. Electronic, engineering, and clinical medicine companies and organizations spend 5–15% of their capital turnover on research and development.6 About 20% of these expenditures are in basic areas, and 80% are used in systematic design and development. In contrast, expenditures for education research in the United States are dramatically lower. In 1998, less than 0.01% of the overall education budget in the United States of $300 billion a year was spent on research.7 In 1994, the amount of federal spending on health professions education research was less than 0.001% of the total amount of direct federal spending on graduate medical education.8 While the recent situation of medical education research has not been substantiated by a similar report, researchers in the field would assert that medical education research continues to experience the same low level of funding.
The status and future of medical education research were the subjects of an invitational conference jointly sponsored by the Bureau of Health Professions and the Association of American Medical Colleges (AAMC) in 1993.9 Recommendations from this conference recognized the problems in funding medical education research and proposed the development of a study section in the National Institutes of Health devoted to medical education research and the establishment of centers for medical education research in several medical schools. Creation of these organizations would bring much needed attention to the area of medical education research, as well as additional funding resources. Several centers were established in 1994 by the Bureau of Health Professions and continued through 1998, at which point funding was curtailed. A study section was not initiated, and the national center for medical education research proposed by Wartman and O'Sullivan1 in 1989 (and discussed by Wartman in his article in this issue of Academic Medicine) was never established.
Unfortunately, there has not been a recent increase in the level of funding for medical education research, either in federal or private funding sources. In this article I offer a commentary on how medical education research is typically supported, methods by which medical school faculty and staff may find support for their work, and the issues in funding and research that we medical education researchers still face.
Prior to identifying the sources of support for medical education research, it is important to answer the question, What constitutes medical education research? A simple answer is that it includes any topic presented in journals or at conferences deemed to be about educating physicians. This definition would take in any topic related to the education of medical professionals, including teaching methods, the development and evaluation of teachers, and the assessment of learner performance. Basic medical education research studies focus on exactly how physicians function in their profession, and the educational experiences needed to learn the knowledge, skills, and attitudes required to be an expert practitioner. Research in the ethical and humane behavior of physicians is an increasingly important area of study, as is the use of technology in the education and medical practice of physicians. Medical education research also includes issues of admission and preparation of candidates for medical training, career choice, and professional performance after completion of training. Research methods are also a topic but are rarely reported in the literature. Recently, connecting educational interventions to patient outcomes has become an important topic in the field, although the call for connecting medical education and outcomes was voiced by Geoff Norman10 as early as 1985. (See the article on outcomes research by Chen and colleagues in this issue of Academic Medicine.) The boundaries of medical education research could include other areas of research—for example, the evaluation of physician competence, or the preparation of students in sciences and mathematics required for entry into medical careers.
How Has Medical Education Research Been Supported?
Many of the topics germane to medical education research focus on the activities of faculty and staff in selecting, teaching, and evaluating medical students and residents, and the preparation and evaluation of materials and faculty for teaching activities. The need of schools and departments to monitor and improve these activities and the requirements of accrediting organizations have been the impetus for many of the evaluation and research reports published or presented at meetings. The majority of research reports appearing in medical education journals seem to come from the efforts of faculty and staff to understand and evaluate the outcomes of their own educational programs. Often the resources of the institution support these research efforts. Employment of educators within a school is a major source of support that goes unreported. Offices of medical education or other research services in public health and education provide additional support. Faculty development activities at a number of schools and the development of clinician–educator tracks also foster medical education research. Faculty development programs or career awards from federal or private foundation grants also give support to faculty interested in educational research. Some research activities require resources that go beyond what is usually available in the normal course of faculty activity, and external funding must be sought.
Grants in support of educational innovation, such as the Edward J. Stemmler, MD Medical Education Research Fund Awards from the National Board of Medical Examiners or awards from the Fund for Improvement of Post Secondary Education, can be sources of funding for medical education research. Large grants aimed at major reform of medical education, such as the Undergraduate Medical Education for the 21st Century project, sometime provide funds to support individual and multi-institution studies of their outcomes. Training grants from the Bureau of Health Professions, the National Institutes of Health, or other federal agencies may provide support for evaluation and research, but these usually represents a small portion of the total funding, and are often too small to provide the types of activities needed to monitor the impact of the programs beyond the lifetime of the innovation itself. A small number of medical education research infrastructure grants have also been awarded to primary care departments from the Bureau of Health Professions, but their effectiveness in promoting research has not yet been documented.
In order to confirm these perceptions of the state of support of medical education research, I reviewed a sample of research reports in two major journals that place a strong emphasis on medical education, Academic Medicine and Teaching and Learning in Medicine. I attempted to identify the source of funding, which is usually in the acknowledgments section, for all research articles printed in the May issues of Academic Medicine from 2000 through 2004, and for those printed in the spring issues of Teaching and Learning in Medicine from 1999 through 2003. Of the 70 articles, 45 did not include an acknowledgment of funding. Most of the 70 articles reported activities in one department or medical school. A number focused on results from national data sources, and were authored by employees of the agencies responsible for the data. It may be assumed, particularly if the nature of these articles is taken into account, that the research reported was performed in the course of the employment of the authors with supporting funds coming from operational budgets. Of the 25 articles that did acknowledge funding, five indicated support from departmental or institutional awards, including research grants or faculty development funds.
The remaining 20 articles listed external funding; five of these included several sources. A total of 30 individual agencies were acknowledged. Eight of the citations (27%) included federal sources such as academic and career awards from the National Institutes of Health, grants from the Health Resources and Services Administration, the Fund for Improvement of Postsecondary Education, the Centers for Disease Control and Prevention, or the Canadian Institutes of Health. Seven of the citations (13%) acknowledged funding from national institutions such as the AAMC, National Board of Medical Examiners, Society of General Internal Medicine, American Academy of Family Physicians, or the Royal College of Physicians and Surgeons of Canada. Four articles (13% of the citations) acknowledged funding from the Robert Wood Johnson Foundation or the W.K. Kellogg Foundation. The remaining 11 citations acknowledged funding by a number of private foundations, each mentioned once.
Medical education research in North America is reported in many other journals, including international journals of medical education as well as journals from basic science disciplines and clinical specialties. Articles in these journals, particularly those associated with clinical specialties or basic science disciplines, may be supported by funding specific to the field. While the relatively small number of articles and types of journals included in this sample limit any conclusion from this informal analysis, it is likely that this general pattern is found for medical education research articles and reports in other types of journals. In short, it appears that the majority of medical education research articles stem from activities not funded by external sources
How Can One Get Started When Funding Seems Unlikely?
If external funding is unusual, how can a faculty member get started in medical education research? The environment in which medical education research can be fostered has several characteristics that are needed to support this type of work. It is critical that the institution, more specifically the leadership of the school or department, values research as part of the process of the education of physicians. Without recognition of the importance of this type of work, and the publications that may proceed from it, there is little impetus for a faculty member to carve out the time and commit the effort to undertake this type of research.
Second, the faculty member should have had some explicit training or experience in educational development and research in medicine or in similar social science research fields. While formal instruction in learning theory and educational statistics may not be necessary to conduct a course or manage a clerkship, access to this knowledge can clearly help the researcher in developing meaningful educational interventions and posing sensible research questions based in educational theory and substantiated by a sound logic model. Training in educational research can take several forms, including workshops and faculty development programs sponsored by individual schools or professional societies. Other support can often be found from the faculty or staff of offices of medical education or from educators hired specifically to practice within medical school departments. The collaboration between professionals trained in education and basic scientists and physicians benefits both professions, combining expertise in educational theory and research methods, access to trainees and patients, and the strength of content and professional knowledge.
Often, the methods needed to conduct medical education research require little in the way of personnel or equipment. Many research tools, once they have been developed and piloted, require relatively little support for implementation; the researcher simply collects the data and then analyzes the results. A great deal of educational research can be conducted using resources available to most medical school faculty, such as surveys of medical students of faculty within the medical school setting. Observational and qualitative methods, including ethnographic approaches, interviews, and focus groups, require more resources to collect data and to code and interpret results. Interventions requiring equipment or development support, such as investigations of the use of hand-held computers or information systems in education, implementation of electronic curriculum, or use of human or machine based simulations, may require substantial funds. In many schools, resources for these types of educational innovations, such as standardized patient programs or requirements for student computer purchases, have already been established and the research task becomes one of developing a reasonable research proposal that can utilize these resources and technologies.
What About Funding Small or Modest Amounts of Support?
Although a great deal of medical education research is conducted without external funding, funding is available if one searches for it. “A lot can be done with a little money!” according to Judy Shea, PhD, associate professor of medicine at the University of Pennsylvania, in a recent discussion of research funding in the Research in Medical Education Steering Committee of the AAMC Group on Educational Affairs. Small amounts of support can help a researcher implement a simple study or develop pilot data on which to base a more extensive proposal for additional funding. Institutions can help support this process (and often see benefits from it beyond the value of their investment). For example, in 1995, the University of Wisconsin School of Medicine4 initiated an internally funded grant program for medical education research with awards of no more than $5,000 for any one project. The initial $20,000 allocation from the school was matched by an additional $40,000 from other sources in 1997. Based on pilot work sponsored by these internal grants, two projects received external funding of over $500,000. During the initial two years of the project, one faculty member received a national award for the quality of research undertaken with a university grant, and several presentations and at least one publication resulted from the program.
The value of small or modest investments in medical education research is replicated in the granting practices of a number of professional associations. The American College of Emergency Physicians, the Emergency Medicine Foundation, and the Society for Academic Emergency Medicine jointly award annual grants of $5,000 to support research related to educational activities in emergency medicine. These grants generally focus on students, residents, and fellows across a wide range of education topics including procedural skills and improving clinical problem solving.
The Medical Education Foundation of the Association of Professors of Gynecology and Obstetrics, in conjunction with Ortho-McNeil Pharmaceutical, makes annual grants of $15,000 to support research in medical student education in obstetrics and gynecology. Fundable topics include clerkship evaluation, community-based education, computer-based education, ethics in obstetrics and gynecology, evaluating students, gynecologic teaching associates, student recruitment in obstetrics and gynecology, standards and competence, teaching techniques, and general women's health projects. A second grant program, supported by the foundation and Pfizer Women's Health, funds projects in curriculum development in women's health for undergraduate medical students. Although a faculty member in a department of obstetrics and gynecology must lead these projects, applicants are encouraged to propose multidisciplinary projects.
The Foundation for Anesthesia Education and Research awards slightly larger grants of $25,000 for research in anesthesia education. Funded proposals have included the evaluation of the outcomes of a curriculum innovation of student learning and performance, evaluation of methods for faculty development, and research in methods to prevent stress-related problems and physician impairment. The American Sleep Medicine Foundation has developed a new award specifically for education research in sleep medicine, providing funding of $30,000 a year for two-year periods. The Association for Surgical Education awards grants of up to $50,000 a year for two years in educational development and research in surgical education. The current top priority of the foundation is to develop innovations in teaching in a changing educational environment. Other specialty associations may have similar grant programs, and it is worthwhile to investigate them as sources of funding.
Somewhat larger funding amounts are available through the Edward J. Stemmler, MD Medical Education Research Fund, which provides up to $70,000 over a two-year period; they foster significant research in medical education on relatively modest budgets. The fund was established in 1995 by the National Board of Medical Examiners with the intent of providing support for research or development of assessment approaches to improve the evaluation of individuals preparing for the practice of medicine. The most recent grants included those with the following titles: Assessment and Prediction of Teamwork; Development and Testing of an Innovative Admissions OSCE for Assessing Non-Cognitive Key Competencies in Medical School Candidates; Systems-Based Practice: Development of a Measure to Assess Competency; Assessing Clinical Teaching with Standardized Students: A Feasibility and Validity Study; Towards Assessing Professional Behaviors of Medical Students Through Peer Observation: A Multi-institutional Study; and Expanding a Model and Assessment of Professionalism in Medical Students. While grants of $70,000 are much larger than those of $5,000, they still do not represent significant support for the salary of an investigator and the staff support and materials needed to conduct the research.
Other foundations also provide grants in support of developing and evaluating medical education projects. For example, the Aetna Foundation supports training in culturally appropriate care; bioethics education is supported by the Greenwall Foundation; the Tamkin Foundation of Los Angeles has funded innovations in the training of residents as teachers; and the Hartford Foundation supports improvements in geriatrics education. This list is not exhaustive! There are hundreds of foundations that have supported medical education innovation and research. Some foundations are small, and give grants only to organizations within a specific region. For example, the California Endowment has supported cultural competency education in California. Finding a potential funding agency takes persistence; frequently, the help of grants information specialists from your library or grants office will be useful.
Funding can also come from unexpected nontraditional philanthropic sources, in one instance a local funeral director. Many families reported to the director how poorly physicians conveyed the news that their loved ones were dying or had died. This led the director to fund a project to develop and test a curriculum to teach physicians to break bad news to families more effectively.
What Resources Are Available for Support of Larger Projects?
The major source of funds for medical education development and research is the federal government, through a variety of training programs across a number of agencies. Training grants, career awards, fellowships, and scholarships are available from many federal agencies. The Bureau of Health Professions, through its Academic Administrative Units in Primary Care grants, has supported the development of medical education research infrastructure at a number of medical schools. While many grants do not fund medical education research per se, they may provide opportunities to develop and evaluate educational programs as well as to learn and develop skills in research methods. The federal government provides links to all its assistance programs in the Catalog of Federal Domestic Assistance (see 〈http://188.8.131.52/cfda/cfda.html〉). The National Institutes of Health makes available a description of all funded projects at 〈http://crisp.cit.nih.gov/〉 and a weekly newsletter for all announcements of funding opportunities at 〈http://grants.nih.gov/grants/guide/index.html〉. And the Health Resources and Services Administration offers grant information at 〈http://www.hrsa.gov/grants/default.htm〉.
Private foundations also have supported major programs in medical education and health care. Gunzburger3 identified five foundations in 1994 as the major sources of funding support for medical education and health care. Eight years later, a list of major foundations supporting medical education and health care today would still include four of these organizations: the Josiah Macy, Jr. Foundation, the Robert Wood Johnson Foundation, the Pew Charitable Trusts, and the W. K. Kellogg Foundation. Major funders in health care have been joined by a new player in the international field of support to health care, the Bill and Melinda Gates Foundation. While these organizations provide millions of dollars in grants for health care each year, their main focus is on improving health care in the United States and the rest of the world, and not on investigating the process of medical education.
However, two of these foundations remain actively interested in supporting medical education innovation and research. The Macy Foundation is interested in educational projects that will (1) improve medical and health professions education in the context of the changing health care system, (2) increase diversity among health care professionals, (3) demonstrate or encourage ways to increase teamwork between and among health care professionals, and (4) devise educational strategies to increase care for underserved populations.11 The Robert Wood Johnson Foundation supports “training, education, research (excluding biomedical research), and projects that demonstrate the effective delivery of health care services.”12
How Can a Researcher Identify Potential Funding?
Inquire about pilot funding from your department or university. Small amounts of money are frequently available, particularly if the investigator can use these resources to develop preliminary results that may be leveraged into large grant proposals for external funding. Talk with colleagues, both at your own institution and from other schools. Colleagues may be able to quickly identify sources that are interested in your particular research or development topic, and may tell you about small or local foundations that are not widely known. Contact program officers at likely foundations or federal agencies. They can quickly tell you if your interest fits within their programs, or they may have suggestions about other agencies that might be interested in your project.
Search funding databases: there are a variety of grants databases that are available on the Web with free access, or available for a fee usually paid for by the researcher's institution. GrantsNet, found at 〈http://www.grantsnet.org〉 and funded by the American Academy of Science, is particularly helpful or identifying research in basic sciences. The Community of Science Funding Opportunities Database, at 〈http://fundingopps.cos.com/〉, allows users to specify searches for a wide variety of topics and disciplines, and delivers announcements of grant opportunities to subscribers via e-mail. Unfortunately, many of these databases do not include search terms appropriate for medical education research. It may take persistence and great inventiveness to develop searches that identify appropriate foundations or agencies. Other organizations, such as the University of Illinois, provide publicly accessible compendiums of research funding opportunities (see the Illinois Research Information Service at 〈http://www.library.uiuc.edu/iris/〉). Many universities and medical schools have developed their own resources, and a researcher may do best by seeking help from a local research information service, usually located in the library or research support office.
Search the Internet on your own. Small or regional foundations not included in large databases can be identified by simple searches. But be careful in limiting your search terms. A search in Google, using the search terms + “medical education” + research + grant + foundation, resulted in nearly 90,000 documents. The interests of foundations change over time; you cannot be sure that an agency will continue to fund programs in your particular area of research. On the other hand, new funding sources may become available for your research.
Sometimes, a match between your ideas and a funding agency will not be found. But recasting an idea within contemporary issues may allow a fit. For example, currently there is a great deal of interest and funding around patient safety. If an educational innovation could be tied to improvements in patient safety, your chances of funding would be greatly enhanced. The impacts of changes in health care practice, including the reduction of residents’ duty hours and decreasing availability of faculty time for teaching, are areas of concern for educators and researchers as well. The current emphasis on connecting education and practice to patient outcomes may lead to additional sources willing to fund educational research. Collaboration with health services practitioners could also be a strategy to develop fundable medical education research.
What's Missing in This Picture?
Historically, a significant factor for medical education researchers using funding from private foundations or the federal government has been the willingness of these groups to adequately support evaluation activities within training or educational innovation grants. Most granting agencies have resources earmarked for evaluation, but they usually represent less than 5–10% of the total budget. This level of funding makes it difficult to mount reasonable evaluation or research efforts, particularly if the focus is on outcomes for students progressing though an educational program that requires a number of years to complete. Funding agencies are under pressure to show that their efforts are successful, and there is a significant hope that the programs they have chosen to fund will be effective: negative or inconclusive outcomes are not welcome findings. Recent reviews of medical education research, much of which has been funded by these agencies, have concluded that there is a need for greatly improved rigor and quality of the evaluation of these programs.2,13,14 Part of this improvement will depend on increased funding for evaluation and research, as well as an increase in the sophistication with which proposals are written and reviewed.
More important, perhaps, is that the call for increased attention to the larger issues of research in medical education voiced over the last ten years has not been heeded. For a brief time, the Bureau of Health Professions funded centers for research in medical education, but that funding ended. The status of medical education research seems to parallel that of education in general.6 There is a need to fund research into the basic processes of learning and teaching in medicine. Overall, we medical education researchers are strong in our beliefs about what works best in medical education, but are weak in basic theory and models of learning medicine that would support these beliefs, or suggest focused innovations to improve medical education. We have little evidence that our understanding of learning in medicine has grown from insights gained from prior work. Too few studies are conducted across institutions. An innovation at one school may be totally impractical at any other institution. Multi-institutional studies of innovations would permit experimental control as well as careful documentation of the feasibility and outcomes of innovations. Medical education research suffers from the lack of teams whose members represent the specific training and expertise needed to complete large and complex research tasks. We lack the resources to take sound educational innovations from the research stage to implementation beyond our own institutions.
The issues of underfunded evaluation and research of innovative programs, the lack of emphasis on theory building, and the failure to prove the usefulness of educational innovations across schools cannot be overcome at the current levels of attention paid and funding provided for medical education research. A researcher can usually find funding to study his or her own question. The task of finding funding to undertake larger research studies in medical education that envision a long-term commitment to the task and the intent to prove the generalizability of the findings across institutions is much more difficult. As professionals in medical education, we all have a responsibility to be advocates for the research support needed to move medical education research from a field with limited influence on practice to one that is able to connect sound insights and ideas with actual practices in education and improved patient health.
What Can We Conclude?
Many medical educational researchers have made important contributions with the scarce resources that are available. Funding does indicate that a value has been given to the proposed ideas, and there are effective strategies that a faculty member can employ to engage in funded research. Yet we still need to make major strides to overcome deficits in theory and generalizable knowledge about learning and teaching in medical education. This will require enactment of a substantive, federal commitment to professions education research.
The author wishes to acknowledge the contributions of Patricia O'Sullivan, PhD, for her advice and critical review of the manuscript, and the members of the Research in Medical Education Steering Committee for their early brainstorming of issues and concerns incorporated in this article.