The 2000 Institute of Medicine report “To Err Is Human” cited as many as 98,000 annual deaths linked to preventable medical errors,1 and recent literature estimates an even higher degree of burden leading to patient harm.2 These staggering figures have fostered a serious discussion on the subject of error disclosure in the medical community. Are patients being made aware of the errors that may have occurred in their care? How are they made aware? What are the appropriate ways to disclose medical mistakes?
In response to these questions, the Agency for Healthcare Research and Quality (AHRQ) proposed guidelines for medical error disclosure based on the 2010 National Quality Forum report supporting disclosure as one of 30 safe practices for better health care.3 These include (1) disclosure of all harmful errors, (2) an explanation as to why the error occurred, (3) how the error's effects will be minimized, and (4) steps the physician (and organization) will take to prevent recurrences.3
Although formal instruction in disclosure exists in the literature, published studies of error disclosure curricula4–15 demonstrate wide variability in the way training programs incorporate medical error disclosure into their respective curricula. In our experience, providers are often deficient in their ability to disclose errors because of inconsistent training, and they often learn disclosure skills through direct observation of supervising physicians as a result. Trainees may even be exposed to negative role modeling, which increases likelihood of nontransparent behavior in response to an error in the future.16 In a recent scoping review of error disclosure training,17 only 5 of 21 studies performed a structured assessment of learners' performance.5,6,15,18,19 Sixteen studies relied only on self-evaluation to assess learner performance during the teaching and learning process, rather than on objective measures.
Healthcare simulation aims to recreate elements of physical, situational, and psychological realism in a safe and controlled environment20 and offers a unique opportunity that is particularly well suited to assessment of patient-physician conversations regarding medical errors.21 Recently, educators have begun to use simulation technology to address error disclosure for trainees22–26 and practicing clinicians.27 This study aimed to assess error disclosure of health care providers at different levels of experience based on adherence to AHRQ disclosure safe practice guidelines during a simulation scenario. We hypothesized that our participants will be suboptimal in their performance across a range of clinical experience. Secondarily, we hypothesized that immersive simulation using a high-fidelity mannequin simulator can be used to assess clinicians' overall error disclosure performance.
This was a prospective cohort study conducted at a US medical school with an affiliated 4-year emergency medicine (EM) residency from October 2012 to November 2013. This study received a waiver of consent from our institutional review board.
Health care provider groups included third-year medical students (MSs), junior EM residents (PGY 1–2), senior EM residents (PGY 3–4), and attending EM physicians. At our institution, third-year MSs participated in a mandatory 12-week simulation course in which the current research scenario was incorporated into the existing curriculum. All participants were informed that performance in the research simulation scenario would have no effect on evaluations. Resident physicians received 1 hour of didactic credit as an incentive to participate in the study. Attending faculty received credit for 1 hour of required academic participation. Residents and faculty physicians were excluded from participation if they had previous exposure to the research scenario during the MS course. Neither the medical school nor the residency program had formal didactics or curricula in error disclosure before the implementation of this study.
Scenario Design and Execution
The research scenario (Appendix 1) was created based on an actual patient case, was reviewed by the authors, and rehearsed in the simulation laboratory before the initiation of the study. The same four simulation-trained faculty members served as the patient voice and case facilitator operating the computer program throughout the study. The facilitator voiced similarly scripted lines during all cases. All simulation scenarios were conducted in a simulation laboratory using a SimMan 3G (Laerdal, Stavanger, Norway) high-fidelity mannequin. The simulation laboratory treatment room replicated an emergency department patient care room with the availability of telemetry, blood pressure, and oxygen saturation monitoring. Electrocardiogram (ECG) and laboratory results were displayed to the participants on a large LCD monitor. Due to the complexity of the case and anticipated higher cognitive load, MSs completed scenarios in teams of four. One student was assigned as the team leader who was solely responsible for communication and discussion with the patient and only that student's discussion was scored for the purposes of error disclosure. All residents and attending faculty completed the scenario individually. An author (A.C.) was present in the treatment room as the confederate nurse for all scenarios. All scenarios were video recorded in their entirety.
The patient is a 40-year-old man with a history of asthma and several food allergies including anaphylaxis to peanuts. He normally carries his EpiPen but did not have it with him while he ate at a Chinese restaurant where he believes he ate a dish fried in peanut oil. The incident occurs in close proximity to the hospital (<5-minute transport). He calls 9-1-1 and emergency medical services transports him directly to the hospital with no interventions. Upon arrival to the emergency department, he complains of throat swelling and mild difficulty with breathing. His vital signs are notable for a pulse of 110 beats per minute, respiratory rate of 32 breaths per minute, blood pressure of 96/42 mm Hg, and oxygen saturation of 92%. Physical examination findings present on the mannequin include tongue swelling and expiratory wheezing.
During the course of the simulation scenario, when the participant orders epinephrine, the confederate nurse administers an intramuscular dose of epinephrine (0.3 mL 1:1,000 dilution) intravenously. Embedded in the computer program for the scenario is a pharmacological response to the medication error. The patient becomes tachycardic with ST segment elevations on the cardiac monitor. The patient immediately begins to complain of chest pain. If the participant does not immediately recognize the error in route of administration, the nurse volunteers that she gave the dose of epinephrine intravenously instead of intramuscularly. Regardless of the actions taken by the participant, the nurse response is standardized to clarify the medical error and apologize to the provider for the mistake. Subsequent to the medical error, the medical management of the participant determines the flow of the scenario. In every scenario, the patient states that he has been to the emergency department before for anaphylaxis and has never developed chest pain in response to epinephrine and asks the question, “How did this happen?” The participant is expected to immediately disclose the error to the patient before completion of the scenario.
All participants were debriefed after the scenario by an author trained in debriefing. The format and content of the debriefing were drawn from literature on “debriefing with good judgment”.28,29 All participants were given the opportunity to offer feedback on the scenario. Debriefing included discussion of clinical management and the individual participant's adherence to AHRQ guidelines regarding medical error disclosure.
Assessment Instrument Design and Use
Our primary outcome was the total overall score on a 6-component, 21-point disclosure instrument developed using the AHRQ Patient Safety Primer on error disclosure (Fig. 1). The tool was developed using the National Quality Forum's 2010 “Safe Practices for Better Healthcare” Update on medical error disclosure.3 Potential items to be included were selected from the publication. Items and their respective weighting were then refined using the Delphi method.30 The Delphi decision group consisted of EM faculty with expertise in education and simulation. Each expert received a series of open-ended questionnaires referencing the AHRQ Primer through a study co-investigator who compiled the responses. Three rounds of blinded feedback were collected to build consensus for individual items in the final instrument. Secondary outcomes consisted of scores in each of the six domains. Disclosure, apology, and explanation of why the error happened were deemed most critical in provider-patient discussions and therefore were rated on a 0- to 4-point scale; acknowledgment of responsibility, explanation of how the error's effects would be minimized, and prevention of recurrence were rated on a 0- to 3-point scale. Each video-recorded scenario was viewed and scored independently by two members of the research team who were trained by scoring three sample videos not included in the final results using the instrument. For any score differences higher than 1 point on a single component, both raters reviewed the video together and that item was rescored if the experts could come to a consensus. If the experts could not reach a consensus, the original scores remained.
Weighted κ statistics were calculated for interrater reliability. Agreement was considered moderate for κ between 0.40 and 0.60, substantial for κ from 0.61 to 0.80, and high for κ value of greater than 0.80.31 Scores were compared across level of training using Mantel-Haenszel χ2 tests of linear trend for ordinal variables. Spearman rank correlations were used to compare performance on disclosure with previous simulation exposure. We performed all analyses using SAS 9.3 (SAS Institute, Inc, Cary, NC).
There were 12 participants in each study group (N = 48). Demographics are shown in Table 1.
Interrater reliability for scoring with the evaluative instrument is shown in Table 2. A count of items rescored after reaching consensus as well as weighted κ for each component is included. Weighted κ for scores of disclosure and participant's discussion of minimizing harm reflected moderate interrater reliability (0.59, 0.50). There was high interrater reliability for scores of acknowledgement of responsibility, apology, and preventing recurrence (κ = 0.82, 0.88, 0.86, respectively). Interrater reliability for total scores was substantial (κ = 0.70).
Graphs of the distribution of scores for each category are shown in Figure 2. Notably, all participants disclosed the error, with most participants disclosing the error in language the patient could understand (score = 3). Twelve participants (25%) did not address responsibility for the error in any way. All participants discussed minimizing harm to the patient associated with the medical error. Seventy-five percent of participants did not discuss preventing the recurrence of the error with the patient.
Average scores for all domains across clinical experience are listed in Table 3. Total scores (maximum score of 21) improved significantly as experience increased (P = 0.03), from MSs [10.3 (2.7)], PGY 1–2 trainees [12.3 (6.2)], PGY 3–4 residents [13.7 (3.2)], and attending physicians [12.8 (3.7)]. Differences in scores across years of experience were not statistically significant for apology for the error (P = 0.23), disclosure of the error (P = 0.08), and acknowledgement of responsibility for the error (P = 0.053). Differences in scores across clinical experience were not statistically significant for participants' explanation of the medication error (P = 0.07). Participant scores for discussion of minimizing the harm caused by the error were higher for greater clinical experience (P = 0.02). Only 25% of participants addressed preventing the recurrence of the medication administration error. Participant scores for this component were higher for greater clinical experience (P = 0.001).
Our study demonstrated that error disclosure performance correlated with increasing clinical experience, but adherence to AHRQ guidelines was suboptimal for both providers in training and attending physicians after managing an adverse event from a medical error. In addition, we found that a simulation-based approach can be feasible to assess compliance with current standards of error disclosure using an evaluative tool.
Previous studies have employed standardized participants for the assessment of learners' performance of error disclosure.17 Standardized participants are particularly powerful to address communication skills, allowing for high degree of authenticity and provision of timely feedback from the direct recipient of the communication.32 For acute care specialties, error disclosure has unique interpersonal contexts as providers attempt to stabilize, diagnose, and treat patients while rapidly establishing rapport and developing trust and alleviating anxiety with the patient and family.33 For this purpose, we designed the scenario with a discussion between the participant and a simulated high-fidelity mannequin that occurred in a dynamic, immersive acute care environment. By juxtaposing the management of an acutely ill patient with a medical error that led to adverse outcomes, we were successful in assessing error disclosure for a unique but important clinical context through healthcare simulation.
We were particularly interested in capitalizing on the advantages of the simulation environment for assessment of error disclosure. Matos and Raemer23 pioneered the design of an assessment instrument for error disclosure tailored for use in immersive simulation. The authors employed a behaviorally anchored rating scale and an instrument tied into the five stages of grief for anesthesia trainees after the patient overhears a conversation ridiculing his body habitus due to inadequate anesthesia. Our study focused instead on the components of the disclosure and adherence to national guidelines rather than on participant stress, body language, and patient-physician interaction. In contrast to other communication-related tasks, which center primarily on delivery and word choice, content is a key determinant of a full disclosure. Our instrument combined characteristics of a checklist containing items required in six domains with a behaviorally anchored rating scale that scored performance in each domain. Previous literature has also found success with global rating scales to assess teamwork and communication skills in the simulation environment.34,35
Our results revealed significant improvement in error disclosure performance as providers gained clinical experience. Although this result may be intuitively logical, the correlation between the ability to disclose errors and experience in patient care is not clearly established in the literature.17 This is in light of the fact that a survey found only 33% of trainees reported any formal instruction in error disclosure, and 92% indicated that they felt underprepared for error disclosure encounters.36 Our study has compared error disclosure performance across a broad level of experience that included MSs and practicing physicians. One previous study compared junior and senior residents in disclosing either an iatrogenic injury or an incidental cancer finding to a patient's family member through a self-assessment and a global rating scale.15 On self-assessment, senior residents were more comfortable than junior residents with disclosure of an iatrogenic injury. Although senior residents scored higher on some individual items for disclosure of iatrogenic injury, the difference in overall performance was not statistically significant. This may have been due to small sample size (n = 15) with only trainees but reflects further work needed to measure and establish competence in error disclosure across levels of experience.
Finally, our participants underperformed overall on the adherence to guidelines for error disclosure. Although it is difficult to accurately judge performance because no benchmarks exist for any particular level of clinical experience, it is notable that scores were particularly lower for the domains related to responsibility, apology, and prevention of recurrence of the medical error. Physicians-in-training have cited multiple reasons for not fully disclosing errors, including fear of litigation, concerns regarding reprisal and punishment, discomfort with patients' emotional responses, and uncertainty with the process to disclose an error.37 For practicing physicians, error disclosure is still not universally accepted as standard practice. One study showed that 42% of surveyed physicians did not believe full disclosure is necessary, whereas partial disclosure was believed to be necessary by 56% of physicians, and no disclosure was the choice of 3% of physicians.38 It would be unsafe to presume the etiology of medical errors immediately during a patient encounter, and a review process through a root cause analysis or a follow-up investigation is needed to determine the correct course of action to prevent future occurrences. However, providers should be aware that error prevention is one of the recommended steps in best practices guidelines, and patients should be told that a medical error will be investigated thoroughly by the healthcare system. Given our results, more consistent and in-depth education regarding error disclosure best practices may be needed, even for practicing clinicians who have completed training.
This study had several limitations. Participants may have spoken more frankly with the patient in a simulated encounter than they would have in a real clinical situation where an error resulting in harm to a patient had occurred. More junior participants, particularly MSs, may have had higher cognitive load because of management uncertainty, the stress of which may have interfered with their communication with the simulated patient. Previous exposure to simulation within the faculty was much more varied than in the other participant groups. Familiarity with the simulation laboratory and capabilities of the simulator may have influenced participant performance.
Because it was difficult to estimate an effect size for clinical experience on performance on our instrument due to lack of established provider performance in error disclosure in the literature, we were unable to perform a power analysis. This may inflate error for individual domains of the instrument. Finally, to ensure that an error consistently occurred and that the participant would need to perform error disclosure as part of the case, we wrote the medication administration error into the script to be performed by the confederate nurse in the scenario rather than the physician participant. This may have caused some participants to not feel personally responsible for the error and impact their performance of the error disclosure. However, we explicitly made it clear to participants that they were solely responsible for all communication with the simulated patient. The use of a standardized participant rather than a mannequin simulator may have facilitated the realism of the case, although the confederate nurse assisted in the psychological fidelity of the scenario. Finally, the psychometric properties of our instrument and lack of additional raters may have affected error disclosure performance scores for our participants.
We have created an instrument and an immersive simulated scenario for objectively assessing adherence to AHRQ error disclosure guidelines. In addition, although improvement in disclosure of medical errors increased with physician experience, this simulation study indicated that health care providers do not comply fully with safe practice guidelines for disclosure of adverse events. This suggests that current training may be inadequate and more effort should be placed on education of error disclosure, including practicing physicians in the process. Expanded training has the potential to improve patient-physician communication with respect to disclosure of medical errors and ensure that patients are provided with the information they desire regarding unanticipated outcomes related to their care. Going forward, our consensus instrument of error disclosure compliance will require use on a larger scale and in multi-institutional settings to provide further validity for simulation-based assessment purposes. More granularity regarding individual components of the medical error discussion and potential barriers to full compliance of disclosure guidelines will be important in designing future educational interventions to address this need.
1. Kohn L, Corrigan J, Donaldson M. To Err Is Human: Building a Safer Health System
. Washington, DC: Institute of Medicine; 2000.
2. James JT. A new, evidence-based estimate of patient harms associated with hospital care. J Patient Saf
3. National Quality Forum (NQF). Safe Practices for Better Healthcare - 2010 Update: A Consensus Report
. Washington, DC: National Quality Forum; 2010.
4. Bell S, Moorman D, Delbanco T. Improving the patient, family, and clinician experience after harmful events: the “When things go wrong” curriculum. Acad Med
5. Gillies RA, Speers SH, Young SE, Fly CA. Teaching medical error
apologies: development of a multi-component intervention. Fam Med
6. Gunderson A, Smith K, Mayer D, et al. Teaching medical students the art of medical error
full disclosure: evaluation of a new curriculum. Teach Learn Med
7. Halbach J, Sullivan L. Teaching medical students about medical errors and patient safety: Evaluation of a required curriculum. Acad Med
8. Keller DR, Bell CL, Dottl SK. An effective curriculum for teaching third-year medical students about medical errors and disclosure. WMJ
9. Madigosky WS, Headrick LA, Nelson K, et al. Changing and sustaining medical students' knowledge, skills, and attitudes about patient safety and medical fallibility. Acad Med
10. Moskowitz E, Veloski JJ, Fields SK, Nash DB. Development and evaluation of a 1-day interclerkship program for medical students on medical errors and patient safety. Am J Med Qual
11. Patey R, Flin R, Cuthbertson B, et al. Patient safety: Helping medical students understand error in healthcare. Qual Saf Health Care
12. Paxton JH, Rubinfeld IS. Medical errors education: a prospective study of a new educational tool. Am J Med Qual
13. Thompson D, Cowan J, Holzmueller C, et al. Planning and implementing a systems-based patient safety curriculum in medical education. Am J Med Qual
14. Varkey P. Educating to improve patient care: integrating quality improvement into a medical school curriculum. Am J Med Qual
15. Varkey P, Natt N. The objective structured clinical examination as an educational tool in patient safety. Jt Comm J Qual Patient Saf
16. Martinez W, Hickson GB, Miller BM, et al. Role-modeling and medical error
disclosure: a national survey of trainees. Acad Med
17. Stroud L, Wong BM, Hollenberg E, Levinson W. Teaching medical error
disclosure to physicians-in-training: a scoping review. Acad Med
18. Singh R, Singh A, Fish R, et al. A patient safety objective structured clinical examination. J Patient Saf
19. Posner G, Nakajima A. Assessing residents' communication skills: disclosure of an adverse event to a standardized patient. J Obstet Gynaecol Can
20. Rudolph JW, Simon R, Raemer DB. Which reality matters? Questions on the path to high engagement in healthcare simulation. Simul Healthc
21. Breitkreuz KR, Dougal RL, Wright MC. How do simulated error experiences impact attitudes related to error prevention? Simul Healthc
22. Chumpitazi CE, Rees CA, Chumpitazi BP, et al. Creation and assessment of a bad news delivery simulation curriculum for pediatric emergency medicine fellows. Cureus
23. Matos FM, Raemer DB. Mixed-realism simulation of adverse event disclosure: an educational methodology and assessment instrument. Simul Healthc
24. Barrios L, Tsuda S, Derevianko A, et al. Framing family conversation after early diagnosis of iatrogenic injury and incidental findings. Surg Endosc
25. Brewster L, Risucci D, Joehl R, et al. Management of adverse surgical events: A structured education module for residents. Am J Surg
26. Overly F, Sudikoff S, Duffy S, et al. Three scenarios to teach difficult discussions in pediatric emergency medicine: Sudden infant death, child abuse with domestic violence, and medication error. Simul Healthc
27. Raemer DB, Locke S, Walzer TB, et al. Rapid learning of adverse medical event disclosure and apology. J Patient Saf
28. Rudolph J, Simon R, Rivard P, et al. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin
29. Rudolph J, Simon R, Dufresne RL, Raemer DB. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc
30. de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach
31. Viera A, Garrett J. Understanding interobserver agreement: the kappa statistic. Fam Med
32. Turner TR, Scerbo MW, Gliva-McConvey GA, Wallace AM. Standardized patient encounters: periodic versus postencounter evaluation of nontechnical clinical performance. Simul Healthc
33. Schinasi DA, Kolaitis IN, Nadel F. Difficult conversations in the emergency department: spotlight on the disclosure of medical errors. Clin Pediatric Emerg Med
34. Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies. Simul Healthc
35. Adler MD, Vozenilek JA, Trainor JL, et al. Comparison of checklist and anchored global rating instruments for performance rating of simulated pediatric emergencies. Simul Healthc
36. White AA, Gallagher TH, Krauss MJ, et al. The attitudes and experiences of trainees regarding disclosing medical errors to patients. Acad Med
37. Coffey M, Thomson K, Tallett S, Matlow A. Pediatric residents' decision-making around disclosing and reporting adverse events: the importance of social context. Acad Med
38. Gallagher TH, Garbutt JM, Waterman AD, et al. Choosing your words carefully: how physicians would disclose harmful medical errors to patients. Arch Intern Med
Section I: Scenario Overview
Section II: Curriculum Integration
Section III: Scenario Script
Section IV: Computer Program
Program uses 0 handlers, 0 trends, and 1 frame.
Section V: Confederate Script