Patel, Mitesh S. MD, MBA; Lypson, Monica L. MD; Davis, Matthew M. MD, MAPP
In 2002, the Association of American Medical Colleges (AAMC) established the Institute for Improving Medical Education (IIME) to respond to growing concerns regarding the quality of medical education in the United States. The 2004 AAMC report entitled Educating Doctors to Provide High Quality Medical Care1 created an action plan to address specific concerns that the design, content, and conduct of undergraduate medical programs had not kept pace with the changes in medicine and the increasing complexity of the health care system. The IIME was responding to recent evidence showing that medical care was often less than optimal2,3 and that an improved system of training (i.e., an undergraduate medical education curriculum) was needed to improve the quality of care in the United States.4
Previous investigators had identified several areas for improvement within medical education that would help physicians become better prepared for the practice of medicine, specifically in the areas of population health,5 managed care,6–9 and the changing environment of the U.S. health care system.10–14 In 2004, the AAMC established standards and recommendations to improve the training of future physicians in clinical decision making, clinical care, and the practice of medicine.1 Since that time, few have assessed the impact of these new standards and recommendations on medical school education in the United States.
To assess training in medical school in the domains of clinical decision making, clinical care, and the practice of medicine, we compare trends of medical student responses to the AAMC Medical School Graduation Questionnaire (GQ) from 2003 to 2007. To examine the degree to which education in health care systems affects medical student perceptions of training, we compare responses to the GQ of students from two similar medicals schools—one with higher- and one with lower-intensity curricula in health care systems.
The institutional review board at the University of Michigan Medical School approved this study.
National data sources and analysis
The AAMC’s annual GQ provides national estimates of students’ perceptions of training quantity and adequacy while in medical school.15 The GQ asks graduating U.S. seniors to respond to the following question: “Do you believe that the time devoted to your instruction in the following areas was inadequate, appropriate, or excessive?” The questionnaire lists topics to address a variety of components. For this study we excluded components if they were not listed in each year of the survey from 2003 to 2007. We selected five components to represent training in each of the following GQ-defined domains: clinical decision making, clinical care, and the practice of medicine (List 1). The national GQ provided the percentage of students reporting “appropriate” training in each component of the domains, and we estimated mean domain response rates by averaging the response rates among the five components within each domain, with each component weighted equally.
School-specific data sources and analysis
We compared GQ responses from graduates of two similar medical schools—both routinely rated among the top 10 schools nationally by the news media16—to assess the association between student satisfaction and the intensity of the health care systems curricula. We selected the two schools before the study analysis on the basis of their overall similar curricula but differing intensity in the health care systems curricula. We requested that faculty and administrative staff at each school provide the following information about the health care systems curricula at their school:
* the total number of courses offered,
* the format (lecture or discussion) of the courses,
* the topic or focus of the courses, and
* the year of medical school in which the courses were offered.
We classified the medical school with a more exhaustive listing of course lectures and discussions as having the higher-intensity curriculum in the content area of health care systems. The higher-intensity curriculum included courses specifically addressing components within the practice of medicine domain including medical economics and health care systems, among a variety of other related topics. We classified the other school as having the lower-intensity curriculum in health care systems. Among the courses within the lower-intensity curriculum, only one lecture directly addressed any of the components within the practice of medicine domain (Table 1).
We obtained individual GQ reports through a request to the dean of student programs and/or the dean of medical education at each of the two medical schools. We corresponded with the deans via e-mail, describing the study objective, and we conducted phone conversations and meetings with faculty involved in the administration of the health systems curricula. We chose these two schools because of the significant difference in curricular intensity. We have kept the schools anonymous in an attempt to emphasize curricular content issues and to avoid focusing on either school’s historical rationale for curricular balance. We obtained the characteristics of each of the two medical schools from published literature that provided 2007 statistics regarding medical schools’ average incoming class size, average acceptance rate, matriculants’ average academic scores (GPA and MCAT), and average total NIH research-based grant funding (Table 1).16 We displayed each of these statistics as falling within a range of the top 10-ranked schools, excluding the highest and lowest values as outliers, to allow comparison of the two schools while still maintaining anonymity.
Each school’s GQ provided the percentage of students reporting appropriate training in each component. We performed statistical analysis to estimate mean domain response rates by equally weighting the response rates among the five components within each domain. We conducted Pearson chi-square tests to determine statistical significance of differences between responses of students from the school with the higher-intensity curriculum and those of students from the school with the lower-intensity curriculum. We reviewed data from each school over five years (2003–2007). Because only 16 students from the school with the higher-intensity curriculum responded to the 2005 GQ, we did not consider these data to be an accurate representation of that graduating class, and therefore we excluded from the study the 2005 data for the school with higher-intensity curriculum.
To determine the odds that students from the school with the higher-intensity curriculum reported appropriate training compared with students from the school with the lower-intensity curriculum, we estimated odds ratios and 95% confidence intervals across the entire study period using two-by-two contingency tables consisting of the total number of respondents in the study that reported either appropriate or not appropriate (i.e., inadequate or excessive) training within each of the components of the practice of medicine.
GQ sample characteristics
The number of national respondents to the GQ from 2003 to 2007 reached a high of 13,764 in 2003 and a low of 9,592 in 2005. In 2007, the number of respondents increased (from the number in 2005) to 12,574. The decreased response rate in 2005 has partially been attributed to a change to a two-part survey rather than a single longer survey.17
National trends in GQ responses
Among responses regarding appropriately devoted instruction time, students reported the highest level of satisfaction with instruction in clinical decision making, with 90% to 92% of students consistently reporting appropriate training during the study period (Figure 1). The majority of students also rated clinical care highly, with 80% to 82% of responses consistently reporting appropriate training. In contrast, fewer than 40% of students reported appropriate training in the practice of medicine in 2003. This rate remained at or below 50% for the remainder of the study period.
Curriculum in health care systems at two medical schools
The two medical schools in this study had similar characteristics in terms of incoming class size, acceptance rate, academic standards, and research-based NIH funding, while their curricula in health care systems varied significantly (Table 1).
The medical school classified as having a higher-intensity curriculum in health care systems offered 13 required lectures and 13 required discussion sections in a four-month period during the first-year of medical school. Each one-hour class lecture was followed by a two-hour small-group session. Students were also required to complete a health-care-systems-related project during that first year. Examples of past projects include comparing presidential candidates’ plans for insurance coverage within the United States and comparing current health care systems among countries abroad. The school with the higher-intensity curriculum also offered an optional structured clinical elective course in the fourth year that allowed students to gain firsthand experience in hospital operations, quality assessment, and health care delivery that reflected the principles learned in the preclinical courses.
The school with the lower-intensity curriculum in health care systems offered three required two-hour small-group sessions during the first year of medical school and three required two-hour lectures during the third year of medical school. While students were assigned readings and problem sets for discussion sections, they were not required to conduct a project related to these courses during the study period. While a fourth-year elective that allowed students to shadow hospital administrators and conduct independent projects for four weeks was available, no fourth-year elective was structured to specifically reflect the principles taught in previous health care systems sessions.
For both the school with higher- and the school with lower-intensity curricula in health care systems, trends in the mean percentage of students reporting appropriate instruction time within the domains of clinical decision making and clinical care were similar to national trends. For clinical decision making, satisfaction in training was similar to national trends (90%–92%) with 89% to 96% of students from the school with higher-intensity curriculum reporting appropriate training compared with 89% to 93% of students from the school with lower-intensity curriculum. For clinical care, satisfaction in training was slightly lower than national trends (80%–82%) with 70% to 77% of students from the school with higher-intensity curriculum reporting appropriate training compared with 71% to 80% of students from the school with lower-intensity curriculum.
For the practice of medicine overall, student responses from each of the two schools varied significantly from each other (Figure 2). Students from the school with the higher-intensity curriculum consistently reported significantly higher satisfaction than students from the school with the lower-intensity curriculum: 62% versus 39% for appropriate training in 2003 (P < .01), and 57% versus 48% in 2006 (P < .01).
For each of the five components within the practice of medicine, students from the school with the higher-intensity curriculum consistently reported appropriate training at rates greater than students from the school with the lower-intensity curriculum, with the exception of the medical-record-keeping component, for which both groups reported rates similar to national trends.
Among all 15 components addressed in each of the three domains, students were least satisfied with training in medical economics. Throughout each year of the study period, students from the school with the higher-intensity curriculum in health care systems consistently reported substantially greater satisfaction rates in medical economics (above 50% in each year) than students nationally (below 40% in each year) and students from the school with the lower-intensity curriculum (below 30% in each year).
Students from the school with the higher-intensity curriculum were three times more likely to report appropriate training in medical economics and health care systems than students from the school with a lower-intensity curriculum (Figure 3). They also had greater odds of reporting appropriate training in managed care and practice management. We observed no statistical difference in the odds between the two groups for medical record keeping.
U.S. medical educators have been called on to continually improve the undergraduate medical curriculum to keep pace with a complex and changing health care environment.1,18 We believe this is the first study to examine medical student perceptions of training in clinical decision making, clinical care, and the practice of medicine from 2003 to 2007—that is, since the institution of the IIME. In this study, we assessed student perceptions both at the national level and at two individual medical schools with health care systems curricula of contrasting intensity.
We found that a large majority of graduating U.S. medical students from 2003 to 2007 were satisfied with medical school training in the domains of clinical decision making and clinical care. In stark contrast, fewer than half the students felt that appropriate instructional time was devoted to the practice of medicine, especially the component of medical economics. Moreover, students exposed to a higher-intensity curriculum in health care systems reported statistically greater satisfaction rates for training in the practice of medicine—specifically medical economics—than students exposed to a lower-intensity curriculum. In other words, the higher-intensity curriculum in health care systems—which differed from the lower-intensity curriculum both in structure and in class time devoted to the subject—seems to have translated into a greater sense of appropriate instruction on the subject among graduating students. Importantly, the commitment in time and resources to health care systems sessions at the school with the higher-intensity curriculum did not lead to lower perceived levels of adequate training in other domains of instruction compared with the school with the lower-intensity curriculum.
These findings have direct implications for the training of medical students. Physicians have expressed concern that the scope of care provided in their practice has been affected by health economics,19 managed care,20 and other financial and administrative aspects.19–21 In this study, the practice of medicine was specifically defined using the AAMC component areas of medical economics, health care systems, managed care, practice management, and medical record keeping. The odds of students from the school with the higher-intensity curriculum reporting appropriate training in medical economics and health care systems were more than three times higher than the odds of students at the school with the lower-intensity curriculum reporting appropriate training in these same two components. In fact, they had higher odds of reporting appropriate training in all components, with the exception of medical record keeping. It is important to note that both of the individual schools assessed for this study (and most of their associated teaching hospitals) had electronic medical records throughout the study period, which is not true for all students nationally. Therefore, while it is difficult to appreciate a differential effect of health care systems curriculum intensity on satisfaction in training regarding the component of medical record keeping, each of the other four components was perceived more positively by students exposed to the higher-intensity health care systems curriculum.
A secular trend concurrent (1994–2004) to our study period was the rapid growth of dual-degree programs; the number of joint MD/MBA programs jumped by 223%, and the number of joint MD/MPH programs jumped by 83%.22 These dual-degree programs typically offer additional training in health economics and health care systems. Possibly, these programs represent some medical schools’ attempts to address concerns in health care systems education. However, because nationally each class typically enrolls fewer than 5% of its students in these programs,23 these programs are not likely to significantly impact the overall education of medical students without the implementation of a more intensive curriculum in health care systems at the undergraduate medical school level.
Cox and colleagues19 studied the effect at 16 medical schools of implementing a curriculum to address health care economics, financing, organization, and delivery by comparing student responses from the GQ regarding training in cost-effective clinical practice. They found that implementing curricula that address the component of health care economics had a positive impact at these schools compared with other schools nationally. Since then, the AAMC has removed “cost-effective medical practice” from the GQ and replaced it with “medical economics.” Our findings in this study regarding responses of students exposed to higher- and lower-intensity curricula in health care systems echo those of Cox and coinvestigators. Simply put, more exposure to the concepts of medical economics and related subjects translates into a greater sense of appropriate instruction on these topics among medical students.
At the national level, despite the AAMC’s 2004 recommendations to address concerns in undergraduate medical education regarding health care systems, we found that among three domains of training (clinical decision making, clinical care, and the practice of medicine), students continue to be least satisfied with training in the practice of medicine, specifically in medical economics. These findings raise questions about whether a significant proportion of medical school curricula have yet to follow AAMC recommendations regarding this arena of instruction.
There are several limitations to our study. First, our study addresses medical student perceptions of training and does not reflect actual measurements of competency. Second, we cannot assess whether students answered each question in the GQ on an absolute basis or relative to training in another component. Third, as students continued into residency or clinical practice, we could not obtain data from them to ascertain the long-term effects of their perceptions of medical school training. Fourth, we could not account for any temporal changes in trends of students’ perceptions in the general components studied in our analysis. Finally, while the varying intensity of health systems curricula allows comparison of two different educational environments, we cannot fully assume a causal effect from curriculum intensity and training perceptions because of a variety of unmeasured factors (e.g., baseline characteristics of the medical students) that could not be assessed with the GQ data.
In summary, graduating medical students report consistently high satisfaction rates with training in clinical decision making and clinical care. In contrast, despite national recommendations for improvement in undergraduate medical training in the context of health care systems, students consistently report that not enough instructional time is devoted to the practice of medicine, specifically medical economics. A higher-intensity curriculum in health care systems may hold the key to addressing these continued perceptions of training inadequacy. Future studies are needed to determine the long-term effects of medical school graduates’ perceptions of inadequate training and to develop methods to continually improve medical education regarding the practice of medicine.
2Bodenheimer T. The American health care system. The movement for improved quality in health care. N Engl J Med. 1999;340:488–492.
3McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645.
4Institute of Medicine. Committee on Quality of Health Care in America. Crossing the Quality Chasm. A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
5Greenlick MR. Educating physicians for population-based clinical practice. JAMA. 1992;267:1645–1648.
6Blumenthal D, Thier SO. Managed care and medical education. The new fundamentals. JAMA. 1996;276:725–727.
7Friedman E. Managed care and medical education: Hard cases and hard choices. Acad Med. 1997;72:325–331.
8Kuttner R. Managed care and medical education. N Engl J Med. 1999;341:1092–1096.
9Lurie N. Preparing physicians for practice in managed care environments. Acad Med. 1996;71:1044–1049.
10Council on Graduate Medical Education, U.S. Department of Health and Human Services, Public Health Service, Health Resources and Services Administration. Thirteenth Report. Physician Education for a Changing Health Care Environment. Available at: (http://www.cogme.gov/13.pdf
). Accessed May 19, 2009.
11Yedidia MJ, Gillespie CS, Moore GT. Specific clinical competencies for managing care: Views of residency directors and managed care medical directors. JAMA. 2000;284:1093–1098.
12Halpern R, Lee MY, Boulter PR, Philips RR. A synthesis of nine major reports on physicians’ competencies for the emerging practice environment. Acad Med. 2001;76:606–615.
13Wood DL. Educating physicians for the 21st century. Acad Med. 1998:73;1280–1281.
14Rabinowitz HK, Babbott D, Bastacky S, et al. Innovative approaches to educating medical students for practice in a changing health care environment: The national UME-21 project. Acad Med. 2001;76:587–597.
16Best Graduate Schools: Schools of Medicine: The Top Schools—Research. US News & World Report. April 9, 2007:90.
17Howell WLJ. AAMC examines low response rate for 2005 GQ. AAMC Reporter. September 2005.
19Cox M, Pacala JT, Vercellotti GM, Shea JA. Health care economics, financing, organization and delivery. Fam Med. 2004;36:S20–S30.
20St. Peter RF, Reed MC, Kemper P, Blumenthal D. Changes in the scope of care provided by primary care physicians. N Engl J Med. 1999;341:1980–1985.
21Mayo WJ. The practice of medicine as a business. JAMA. 2006;295:2672.
22Barzanksy B, Etzel SI. Educational programs in US medical schools, 2004–2005. JAMA. 2005;294:1068–1074.
23Larsen DB, Chandler M, Forman HP. MD/MBA programs in the United States: Evidence of a change in health care leadership. Acad Med. 2003;78:335–341.