In the years since the release of the Institute of Medicine (IOM)1 report, To Err is Human: Building a Safer Health System, in November 1999, there has been a dramatic increase in the number of articles in the medical literature on patient safety and medical error. There has not been the same explosion of publications on educating medical students about these issues although the education of physicians beginning in medical school has been increasingly advocated.1–13 From the early 1980s, the literature has included observations of trainees in relation to medical errors,14–20 anecdotes involving medical students and residents,21–28 and a few interviews and surveys.29–33 However, this literature rarely offers specifics on how this teaching should be done. In fact, commenting on progress made and improvements still needed in the years since the IOM report, Timothy Flaherty, MD, chairman of the Board of the National Patient Safety Foundation, noted in an interview that medical education is an area where patient safety has not made any dramatic improvements.34
In addition to the rapidly expanding literature on safe practice, there has been much activity by the federal and state governments, industry pressure (e.g., The Leapfrog Group35), and new regulation (e.g., the National Patient Safety Goals of the Joint Commission on Accreditation of Health care Organizations [JCAHO]36) for actively addressing medical errors and unsafe practices. We found it surprising that the literature much less frequently addresses the education of medical students on patient safety, even training on the basics of the JCAHO's National Patient Safety Goals. With the exception of anecdotal reports of teaching activity37,38 we found very few published reports of specific curricula on medical error in undergraduate medical education.39–41
There are a multitude of reasons to encourage explicit teaching about patient safety, most importantly the prevention of error and improved quality of care for the patient. Five other motivations point to the need for medical schools and residency programs to integrate education about patient safety and medical error into their training. First, the health effect on society of medical errors is huge and merits dedicated time in the curriculum.42,43 Second, academic medicine lags far behind other health care and regulatory bodies such as the JCAHO, the National Quality Forum, and many state governments44 and should be leading efforts to address patient safety problems.45,46 Third, medical schools should address the concerns of patients and the public, many of whom want physicians to handle errors and disclosure differently.47–49 Fourth, physicians report that more training in how to handle errors is necessary, including ways to constructively heal themselves and colleagues after making an error.50 Finally, there is a need to decrease the emotional and cultural barriers in medicine, to address the “hidden curriculum” in medicine, and to facilitate a change in the culture. Such motivations raise many obstacles, not the least of which is the physician's fear of malpractice,51,52 threats to the autonomy of the profession, reluctance to apply systems theory,53 the lack of expert faculty,54 and the competing demands on today's practitioners and medical educators.
In July 2000, the Department of Family Medicine at New York Medical College (NYMC) in Valhalla, New York, integrated into its required third-year clerkship a curriculum to introduce students to avenues for communication about medical errors, to expose them to the prevalence and origins of errors, and to increase awareness of the physician's responsibility for patient safety. Standard methods of evaluation were incorporated from the start of the program to evaluate its effectiveness in raising students' awareness of patient safety and to determine the curriculum's value. In this article, we present an evaluation of the first three years of this curriculum and discuss patient safety education.
In 2000–01, 2001–02, and 2002–03, a total of 572 third-year medical students at NYMC were required to complete and evaluate the curriculum during their four-week family medicine clerkship. We sought and were exempted from IRB approval.
In early 2000, we developed a curriculum to raise awareness about medical errors and patient safety and provide students with practice in an essential skill (i.e., communicating an error to a patient and/or family). We chose to focus on third-year students because we assumed some clinical experience was needed to understand and incorporate this awareness. With only limited time available in our four-week family medicine clerkship, we sought to create a curriculum that might improve the attitudes and skills of medical students more than their knowledge base.
The four-hour curriculum had three parts: an introductory lecture/discussion, brief required readings, and a videotaped simulation with a standardized patient. The students received verbal and written feedback, but were not graded.
A family physician presented the one-hour didactic in an interactive format to a group of 12–24 third-year students during the orientation to the family medicine clerkship. The didactic included a discussion of a medical error, physicians' reactions to an error, and the epidemiology of medical errors. We used the definition of error proposed by Wu et al: “a commission or an omission with potentially negative consequences for the patient that would have been judged wrong by skilled and knowledgeable peers at the time it occurred, independent of whether there were negative consequences.”55, p. 770 In the third year of the curriculum, we added two components to the didactic: the National Patient Safety Goals from the JCAHO36 and the Agency for Healthcare Research and Quality Morbidity and Mortality Rounds on the Web.56
We distributed three to six articles from a list that was updated annually and that most recently included a one-page commentary on the IOM's To Err is Human53; an article about how errors happen, written by a medical student26; and an article that included specific considerations when disclosing an error to a patient.55
Each student participated in a videotaped simulation where an outpatient error was acknowledged and discussed with a standardized patient. The three-hour exercise accommodated four students at a time with two standardized patients and one family medicine faculty member. Beginning with a 20-minute orientation to the case material, the faculty member also provided a review of basic communication skills, techniques for delivering “bad news,”57 and suggestions for discussing an error with a patient or a patient's parent. Two different cases were used each session; the scenarios were outpatient errors that could plausibly be made by a medical student. Each student then had a ten- to 15-minute videotaped encounter with the standardized patient. The student was asked to incorporate basic interviewing skills as well as specific tasks related to discussing the error, including apologizing for the error, taking responsibility for the error, admitting they did not know something, and making attempts to reestablish trust with the patient. This encounter was followed by a small-group feedback session lasting about two hours that included the four students, the two standardized patients, the family physician, and the behavioral medicine faculty. As each video was reviewed, students were encouraged to give feedback to each other as well as to offer commentary on their own interaction with the patient. The standardized patient gave both verbal and written feedback to each student. In both this session and in the initial didactic, the family medicine faculty member frequently told of his or her own experiences in making and disclosing errors.
All 572 students were asked to complete the same seven-item questionnaire (see Table 1 for items) on awareness of strengths and weaknesses both at the start of the clerkship and after the videotape feedback session. Students gave responses to the statements using a five-point scale (1 = extremely aware; 5 = not at all aware). We asked the students to complete the questionnaire on the first day of the clerkship just prior to the didactic presentation and discussion of medical errors and again after completion of the videotape and feedback session that is always part of the four-week clerkship. We matched all responses using the last four numbers of the student's social security number.
In addition, we asked each student to complete a 13-item evaluation of the curriculum at the end of the videotape feedback session (see Table 2 for evaluation statements). Finally, we forwarded a 12-item anonymous, follow-up questionnaire to all students approximately two to eight months after their family medicine clerkship that asked about the students' experience with medical errors since their clerkship. We also followed the results of a single question on the Graduation Questionnaire (GQ) of the Association of American Medical Colleges (AAMC) that relates to a prescription error.58
We compared responses to each statement in the before and after questionnaires using the Wilcoxon signed-rank test for matched data. We present the data from students' evaluations and follow-up questionnaires as numbers and percentages.
For the analysis of the before and after questionnaires we removed respondents who had missing data on either questionnaire. The remaining 535 respondents were distributed among the study period as follows: 2000–01 (n = 180), 2001–02 (n = 178), and 2002–03 (n = 177). A comparison of the mean scores from the before and after questionnaires by academic year is shown in Table 1.
We used responses from all 572 students' forms in the analysis of the evaluations; nonresponses are noted for each item (see Table 2). In the students' evaluations of the patient safety curriculum after the videotape exercise, 511 (89%) students agreed or strongly agreed that “the opportunity to present an error to a patient increases my confidence about discussing this issue with patients.” On the same evaluation, the didactic session “provided a good introduction to the issue” according to 470 (82%) students; however, only 331 (58%) agreed or strongly agreed that “the readings provided on this issue were helpful.” Finally, 537 (94%) students reported the standardized patient exercise to be “a valuable LEARNING experience.”
The response rates for the follow-up questionnaire were 42% (82/193) for 2000–01, 36% (68/188) for 2001–2002, and 82% (157/191) for 2002–03. We do not have an explanation for the markedly high response rate in the third year. A total of 307 students, out of 572 students who participated in the curriculum, responded to the follow-up questionnaire. Eighty-four percent (259/307) of respondents who completed the follow-up questionnaire reported that they strongly agreed or agreed that they had an increased awareness of errors in medicine, with 67% (207/307) reporting strong agreement or agreement that they were more aware of patient safety issues. Only 9% (29/307) noted “the issue of medical errors has been formally addressed in my other third year clerkships.” Twenty-eight percent (85/307) had “witnessed a colleague make a medical error” often or very often; 17% (53/307) had themselves “made a medical error” often or very often in the course of other clerkships. Seven percent (21/307) reported having “discussed an error with a patient or a patient's family.” Ninety-seven percent (297/307) agreed or strongly agreed that “it is important to teach students about medical errors,” with 87% (267/307) agreeing that “the third year is the more appropriate time to discuss medical errors.”
Finally, we examined another potential measurement of the change in students' confidence regarding error in medicine. Every spring, most fourth-year medical students in the United States answer the AAMC's GQ.59 The results are provided to each school with a comparison to the national average. In 2001, all graduating students began responding for the first time to the statement, “I am confident that I have the appropriate knowledge and skills to discuss a prescription error I made with a patient,” using a five-point Likert scale (1 = strongly agree, 5 = strongly disagree). Because the 2001 graduating class at NYMC did not participate in the curriculum, but all subsequent classes did, we chose to follow the responses of the students in our study and the national average.
The national average remained unchanged at 1.9 during 2001–04, but the first NYMC student class to participate in our curriculum improved in their assessment of their confidence to discuss a prescription error with the patient. The average response of the NYMC students to this statement in 2001 was 2.1, and the averages in 2002, 2003, and 2004 were 1.7, 1.8, and 1.7, respectively.58 We also examined trends in the other interviewing questions on the GQ to try to determine if there was some larger unknown change happening to NYMC students, but we found no trend. While this is only one question, and a shift of 0.3–0.4 on a five-point Likert scale is of unclear clinical significance, it does suggest that NYMC students who participated in the brief curriculum expressed increased confidence in discussing a prescription error with a patient when asked 12–24 months after the curriculum. It should be noted that fourth-year students nationally expressed a surprisingly high level of confidence in this area and this confidence remained unchanged over a period in which so much changed in the patient safety and error disclosure movement.
The limitations of our study include the single institution focus, the before and after questionnaire design, and the self-report follow-up. For several reasons, we believe our findings can be generalized to other medical schools that do not have formal patient safety curricula: undergraduate medical education tends to have a similar structure across the United States, our class size is large (190 per year) and includes students from around the country; and finally, our students' clinical work takes place in multiple and varied teaching settings, making it less likely that our students are unique or uniquely trained. The findings from our before and after questionnaires would have been strengthened with the inclusion of a control group of students from either our own or one or more other institutions. This was not financially feasible; however, a randomized controlled trial of an educational intervention should be considered in future research. The self-report follow-up suggests an effect among respondents but does not account for students who did not respond.
In spite of these limitations, our findings suggest that education about patient safety and medical errors can be successfully implemented and maintained in undergraduate medical education. While a relatively small number of students reported that they themselves discussed an error with a patient, it is important to note that we told them that it was not a medical student's responsibility to do so. A brief, experiential educational intervention was shown to increase and sustain awareness of patient safety issues and medical error disclosure to patients.
We believe that this brief curriculum may work because it allows for instruction on many levels. First, students were required to actively practice and then review their own performances in the disclosure of an error with a patient. Most students were successful in honestly disclosing the error, taking responsibility, and apologizing for the error. Many students were relieved after participating in the curriculum, commenting that it was not as bad as they had expected. Second, faculty members modeled discussing medical errors as an integral part of practicing medicine. The students hear the faculty member's own experiences in the matter, and they prepare how they may intervene with the patient. Third, the students' witnessed a forum where faculty encouraged disclosure of individual and system vulnerabilities and gave a human context to the issues. Fourth, professional responsibility and ethics were reinforced and demonstrated in the faculty feedback to the videotapes. Finally, a small-group process of reviewing the videotapes and offering feedback allowed students to begin a dialogue with colleagues about how they managed the disclosure of an error.
Much of the content of a curriculum about medical errors and patient safety is best suited to experiential teaching methods such as standardized patients and simulators. In the literature, simulations have been advocated for potentially improving safety performance or patient safety.59,60 An additional factor in the success of our curriculum was the participation of a stable cadre of committed faculty who were willing to disclose their own experiences with medical errors in practice and openly and frankly discuss the issue with medical students. Such faculty role models discuss not only the knowledge and skills required for safe practice, but also demonstrate the attitudes required.
In addition to an initial curriculum like ours, other useful methods for teaching about patient safety and medical errors include case-based conferences like Grand Rounds and Morbidity and Mortality review improved to incorporate explicit discussion of error and systems,61–63 or even a Patient Safety Conference.64 However, curricula on patient safety should not be primarily delivered through lecture-discussion and large-group formats. Methods must be used which facilitate self-reflection and mindfulness, discussion of feelings,65 acquisition of specific skills in many specialties, and improvement in communication skills not only with patients but also with nurses, other health care professionals, and physicians throughout the hierarchy. Specific team-training skills2 are essential in settings like intensive care units, operating rooms, and emergency departments; improved skills in teamwork may also become essential in many other settings and specialties, including primary care.66 The ideal educational strategy with regard to patient safety would be to view the entire undergraduate curriculum as an opportunity to teach students how to prevent mistakes in clinical practice. Many in medical education believe this teaching already occurs; however, the standard approaches to error prevention and response to errors in medicine are not consistent with the methods used in high-reliability organizations. Some authors advocate a more fundamental restructuring of medical education to improve quality of care and to establish a safety culture in medical education.45,67,68 Short of these systemic endeavors, “one small but very visible step would be for educational leaders to introduce the subject of error prevention and patient safety into undergraduate and postgraduate medical curriculums and examinations.”4, p. 583 As Timothy Flaherty, MD, has noted, “until you change the tests, you don't change the curriculum.”34, p. 3 Beyond that initial step, the best methods of incorporating specifics about safe practice into medical education have not been clearly identified although some groups have delineated goals and objectives, the content, domains, and the competencies needed.10,68–70
The authors gratefully acknowledge the contribution of Patricia Patrick, MPH, research associate, Primary Care Research Unit, in the preparation of this manuscript; and the commitment and contribution to this effort of the following faculty members: Rhea Barton, MD; Paul Gross, MD; Ellen Miller, MD; Sonia Velez, MD; and Lori Weir, MD.
1 Kohn LT, Corrigan JM, and Donaldson MS (eds). To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press, 1999.
2 Health Resources and Services Administration. Council on Graduate Medical Education and National Advisory Council on Nurse Education and Practice. Collaborative Education to Ensure Patient Safety: Report to the Secretary of U.S. Department of Health and Human Services and Congress, September 13–14, 2000, Washington, DC.
3 Ferner RE, Aronson JK. Medication errors, worse than a crime. Lancet. 2000;355:947–8.
4 Wilson T, Pringle M, Sheekh A. Promoting patient safety in primary care: research, action and leadership are required. BMJ. 2000;323:583–4.
5 Agenda for Research and Development in Patient Safety. The National Patient Safety Foundation, 2000 〈http://www.npsf.org
〉 Accessed 3 March 2005.
6 Barach P, Moss F. Delivering safe health care: safety is a patient's right and the obligation of all health professionals. BMJ. 2001;323:585–6.
7 Dickey NW. Patient safety and medical education: a partnership for the future. Texas Med. 2002;98:55–9.
8 Maxwell S, Walley T, Ferner RE. Using drugs safely: undergraduates must be proficient in basic prescribing. BMJ. 2002;324:930–1.
10 Greiner AC, Knebel E (eds). Health Professions Education: A Bridge to Quality. Washington, DC: National Academies Press, 2003.
12 Pilpel D, Schor R, Benbassat J. Barriers to acceptance of medical error: the case for a teaching programme. Med Educ. 1998;32:3–7.
13 Holmes JH, Balas A, Boren SA. A guide for developing patient safety curricula for undergraduate medical education. J Am Med Inform Assoc. 2002;9(Nov–Dec suppl):S124–7.
14 Dubovsky SL, Schrier RW. The mystique of medical training: is teaching perfection in medical house-staff training a reasonable goal or a precursor of low self-esteem? JAMA. 1983;250:3057–8.
15 Mizrahi T. Managing medical mistakes: ideology, insularity and accountability among internists-in-training. Soc Sci Med. 1984;19:135–46.
16 Christensen JF, Levinson W, Dunn PM. The heart of darkness: the impact of perceived mistakes on physicians. J Gen Intern Med. 1992;7:424–31.
17 Leape LL. Error in medicine. JAMA. 1994;272:1851–7.
18 Lesar TS, Briceland LL, Delcoure K, Parmalee JC, Masta-Gornic V, Pohl H. Medication prescribing errors in a teaching hospital. JAMA. 1990;263:2329–34.
19 Bosk CL. Forgive and remember: managing medical failure. Chicago/London: University of Chicago Press, 1979.
20 Carmichael DH. Learning medical fallibility [editorial]. South Med J. 1985;78:1–3.
21 McClure MJ. To err is human. West J Med. 1994;160:492.
22 Calman NS. No one needs to know: a physician recalls taking part in his first cover-up. Health Aff (Millwood). 2001;20:243–9.
23 Eikelboom J. Uses of error: the tip of the iceberg? Lancet. 2002;359:2194.
24 Volpp KGM, Grande D. Residents' suggestions for reducing errors in teaching hospitals. N Engl J Med. 2003;348:851–5.
25 Sutcliffe AG. Uses of error: death of a child. Lancet. 2002;359:2104.
26 Madsen LE. Diesel gas, rice, and medical errors. Pharos. 2002;65:4–10.
27 Levinson W, Dunn PM. Coping with fallibility. JAMA. 1989;261:2252.
28 Wu AW. A major medical error. Am Fam Physician. 2001;63:985–8.
29 Wu AW, Folkman S, McPhee SJ, Lo B. Do house officers learn from their mistakes? JAMA. 1991;265:2089–94.
30 Baldwin PJ, Dodd M, Wrate RM. Junior doctors making mistakes. Lancet. 1998;351:804.
31 Estrada CA, Carter J, Brooks C, Jobe AC. Medical errors must be discussed during medical education. BMJ. 2000;321:507.
32 Schenkel SM, Khare RK, Rosenthal MM, Sutcliffe KM, Lewton EL. Resident perceptions of medical errors in the emergency department. Acad Emerg Med. 2003;10:1318–24.
33 Rosebraugh CJ, Honig PK, Yasuda SU, Pezzullo JC, Flockhart DA, Woosley RL. Formal education about medication errors in internal medicine clerkships. JAMA. 2001;286:1019–20.
34 Raef S. Five years after “To Err is Human”: A Look at the Patient Safety Landscape. Focus on Patient Safety. A Newsletter from the National Patient Safety Foundation. 2004;7(3):1-3.
37 Stevens S. When good doctors err. Young Physicians. 1997;August:27–31.
38 Meyer B. A student teaching module: physician errors. Fam Med. 1989;21:299–300.
39 Lowery DW, Heilpern KL, Otsuki JA. An interdisciplinary curriculum for reducing medical errors in the academic medical center [abstract]. Acad Emerg Med. 2001;8:586.
40 Scobie SD, Lawson M, Cavell G, Taylor K, Jackson SHD, Roberts TE. Meeting the challenge of prescribing and administering medicines safely: structured teaching and assessment for final year medical students. Med Educ. 2003;37:434–47.
41 Langford NJ, Martin U, Kendall MJ, Ferner RE. Medical schools can teach safe drug prescribing and administration. BMJ. 2001;322:1424.
42 Meyer G, Lewin DI, Eisenberg J. To err is preventable: medical errors and academic medicine. Am J Med. 2001;110:602–3.
43 Rosebraugh CJ, Honig PK, Yasuda SU, Pezzullo JC, Woosley RL. Centers for education and research on therapeutics report: survey of medical errors education during undergraduate and graduate medical education in the United States. Clin Pharm Ther. 2002;71:4–10.
45 Stevens DP. Finding safety in medical education. Qual Saf Health care. 2002;11:109–10.
47 Robinson AR, Hohmann KB, Rifkin JI, et al. Physician and public opinions on quality of health care and the problem of medical errors. Arch Intern Med. 2002;162:2186–90.
48 Blendon RJ, DesRoches CM, Brodie M, et al. Views of practicing physicians and the public on medical errors. N Engl J Med 2002;347:1933–40.
49 Gallagher TH, Waterman AD, Ebers AG, Fraser VJ, Levinson W. Patients' and physicians' attitudes regarding the disclosure of medical errors. JAMA. 2003;289:1001–7.
50 Wears RL, Wu AW. Dealing with failure: the aftermath of errors and adverse events [editorial]. Ann Emerg Med. 2002;39:344–6.
51 Studdert DM, Mellom M, Brennan TA. Medical malpractice. N Engl J Med. 2004;350:283–92.
52 Kapp MB. Legal anxieties and medical mistakes: barriers and pitfalls. J Gen Intern Med. 1997;12:787–8.
53 Cohen JJ. Behold the patient safety genie. Acad Med. 2000;75:105.
54 Wilcock P, Carr E. Improving teaching about improving practice. Qual Health Care. 2001;10:201–2.
55 Wu AW, Cavanaugh TA, McPhee SJ, Lo B, Micco GP. To tell the truth. J Gen Intern Med. 1997;12:770–5.
56 Agency for Healthcare Research and Quality. Morbidity and mortality rounds on the Web 〈http://www.webmm.ahrq.gov
〉. Accessed 3 March 2005.
57 Fallowfield L, Jenkins V. Communicating sad, bad, and difficult news in medicine. Lancet. 2004;363:312–9.
58 Medical School Graduation Questionnaire Individual School Report: New York Medical College. Washington, DC: Association of American Medical Colleges, 2001–2004.
59 Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78:783–8.
60 Flanagan B, Nestel D, Joseph M. Making patient safety the focus: crisis resource management in the undergraduate curriculum. Med Educ. 2004;38:56–66.
61 Pierluissi E, Fischer MA, Campbell AR, Landefeld CS. Discussion of medical errors in morbidity and mortality conferences. JAMA. 2003;290:2838–42.
62 Hirsch D, Gliatto P, Meah Y. Quality grand rounds. Ann Intern Med. 2003;138:W-W50.
63 Orlander JD, Barber TW, Fincke BG. The morbidity and mortality conference: the delicate nature of learning from error. Acad Med. 2002;77:1001–6.
64 Sorokin R, Claves JL, Kane GC, Gottlieb JE. The near miss resident conference: understanding the barriers to confronting medical errors. Semin Med Pract. 2002;5:12–20.
65 Penson RT, Svendsen SS, Chabner Lynch TJ, Levinson W. Medical mistakes: a workshop on personal perspectives. Oncologist. 2001;6:92–9.
66 Grumbach K, Bodeheimer T. Can health care teams improve primary care practice? JAMA. 2004;291:1246–51.
67 Lester H, Tritter JQ. Medical error: a discussion of the medical construction of error and suggestions for reforms of medical education to decrease error. Med Educ. 2001;35:855–61.
68 Aron DC, Headrick LA. Educating physicians prepared to improve care and safety is no accident: it requires a systemic approach. Qual Saf Health Care. 2002;11:168–73.
69 Halbach JH, Sullivan LL. Medical Errors and Patient Safety: A Curriculum Guide for Teaching Medical Students and Family Practice Residents. 3rd ed. Valhalla, NY: Department of Family Medicine, New York Medical College, 2003.
70 The Society of Emergency Medicine Patient Safety Task Force. Curriculum for Patient Safety. Lansing, MI: The Society of Emergency Medicine, 2001.