The survival of critically ill patients depends on care that is prompt and error free.1 Within the United Kingdom’s health care system, deteriorating inpatients are often assessed and treated, at least initially, by teams of ward-based junior doctors (doctors within their first year of practice after attainment of a primary medical qualification). Because of competing time demands, senior doctors (specialist doctors with at least four years of postqualification experience) are often not immediately available. Consequently, junior doctors are expected to contact the appropriate specialists according to their assessment of the patient’s condition and the urgency of the situation at hand. Despite the ability to “provide immediate care in medical emergencies”2 being a General Medical Council–mandated outcome of all UK primary medical degree courses, acute care is an area in which new graduates feel consistently poorly prepared.3,4 This perception is supported by data suggesting that patients admitted on the day that junior doctors commence work in the United Kingdom have an in-hospital death rate 6% higher than those admitted a week previously.5 The combination of time pressure, dynamic conditions, and heavy information load afforded by acute situations provides fertile ground for error.6,7
The causes of medical error are diverse and complex, involving both individual and systems factors.8 As the contribution of human error to suboptimal health care outcomes is increasingly understood, a plethora of error-modeling frameworks and taxonomies have been developed which attempt to facilitate deeper exploration and understanding.9–11 However, much of the contemporary discourse within the medical education literature in relation to medical error emphasizes diagnostic error,12–17 despite the fact that “diagnostic reasoning is only one part of the equation.”18 One of the unique challenges of acute care is the necessity to instigate generic resuscitative measures whilst concurrently collating clinical information to aid diagnosis and guide specific management. In the context of hospital inpatients, diagnosis formation may be either aided or hindered by prior knowledge of a patient’s condition, which may not necessarily be relevant to the acute deterioration. Consequently, the exploration of errors made in acute care contexts should not start with diagnosis but, rather, should explore all of the actions undertaken during initial assessment and treatment. The cognitive processes underlying other decisions such as seeking help, judgment of illness severity, and initial investigation choice may provide new insights into the causes of clinical error in the context of acute care. An additional challenge in the care of acutely unwell patients is the fact that doctors rarely make decisions in isolation but, instead, tend to work in teams to plan and provide initial resuscitation and ongoing care.
The conceptual framework we used for this study was the generic error-modeling system (GEMS) devised by James Reason19 but heavily influenced by Rasmussen’s20 skill-rule-knowledge classification of human performance. GEMS was chosen for this study because it provides a practical and logical framework which recognizes the importance of both observed behavior and cognitive processing. Since its inception, GEMS has been developed in a variety of ways, including subdivision of the categories21 and amalgamation with other conceptual frameworks.11 Such modifications have been particularly useful in the context of systems improvement10,22 but seem less applicable to error exploration as a means of driving educational innovation at the level of the individual, where theoretical detail can dilute the potential for practical application. Consequently, we chose to use the original broad version of GEMS for this study. Recent work by Dornan and colleagues23 employed the same broad framework to categorize prescribing errors using information obtained during critical incident debriefing. Other previous studies have also identified the value of retaining broad classifications, although such work has thus far been restricted to the field of prescribing,22,24 an activity which is often undertaken alone and rarely involves the complex, multimodal interactions observable in team-based acute care.
Definitions, explanations, and examples of the four error types described by Reason are given in Table 1. Skill-based slips and lapses, rule-based mistakes (RBMs), and knowledge-based mistakes (KBMs) are all types of unintentional error.19 Violations are intentional aberrant behaviors which, unlike the other error types, are judged against the social and organizational context within which actions occur and not merely against one’s own intentions.19
In this constructivist study, we aimed to answer the following questions: Can GEMS be used to classify the errors made by junior doctors working in small teams, using simulated acute care scenarios to provide the contextualized data? And how can the framework be amplified to accurately reflect the range of errors made by junior doctors working in small teams?
Setting and population
The study was conducted in NHS Lothian, one of the 14 district National Health Service (NHS) boards in Scotland. Newly qualified doctors employed by NHS Lothian undertake a three-day induction program immediately before commencing work. Participation in this study was an optional component of the induction program delivered in August 2010 at the Western General Hospital in Edinburgh. With the prior agreement of the associate dean for foundation training in the South East Scotland deanery and the director of medical education in NHS Lothian, we e-mailed all junior doctors due to commence work at the Western General Hospital and invited them to take part in the study.
Whereas the observation of authentic clinical practice is limited by both practical difficulties and ethically unjustifiable patient safety implications, simulated scenarios allow the observation of clinical skills, behaviors, and responses in an environment that does not expose patients to harm. We thus employed high-fidelity simulation to provide the contextualized data for this study, rather than in its more usual role as an educational tool. Between January and July 2010, eight simulated scenarios involving acutely unwell patients were designed and electronically programmed by three clinicians (V.R.T. and two anesthetic consultant colleagues with particular interests in simulation education). We repeatedly piloted all scenarios using a total of 16 junior doctors who were not study participants. The junior doctors provided feedback on the difficulty and clinical credibility of the scenarios, and the programming was refined to create scenarios that were reproducible and realistic. The four ward-based scenarios used for the study were those which performed most consistently and received the most positive feedback in the pilots: postoperative hemorrhage, severe sepsis, postoperative respiratory distress, and hypoglycemic coma.
The simulated environment consisted of a single, full-body, adult mannequin simulator (Emergency Care Simulator, Medical Education Technologies, Inc., Sarasota, Florida) accompanied by monitoring equipment, drugs, and other supplies as available on a general medical or surgical ward. Three ceiling-mounted cameras allowed each scenario to be filmed from a variety of perspectives and relayed real-time to the control room. The fidelity of the simulated patient was enhanced by a patient voice transmitted via a wireless microphone, dynamic physiology, and realistic clinical examination findings. A bedside monitor provided physiological parameters when requested by participants. A telephone handset was connected directly to the control room, and a member of the study team previously unknown to the participants played the role of a ward nurse, capable of a finite, predefined range of tasks. Because this study focused on the errors made by the junior doctors, the nurse helper neither provoked nor prevented errors from occurring, but did provide accurate and helpful advice whenever it was requested.
After a briefing that covered room layout, nurse helper capabilities, and mannequin features and limitations, the junior doctors were placed in groups of two or three. They were given information regarding the patient’s age, reason for admission to hospital, and current presenting symptom, and they were then invited to assess and treat the patient (mannequin) within the simulated setting. Observation of participants in teams rather than alone replicated the realities of clinical practice and encouraged verbalization of decisions and ideas. Each simulated scenario lasted between 20 and 25 minutes and was video-recorded (with audio). It was immediately followed by an audio-recorded debrief conducted by one of three trained senior clinicians (V.R.T. and two consultant anesthetic colleagues), which lasted between 30 and 40 minutes. Debriefing was aided by immediate playback of the scenario and encouraged articulation of the cognitive processes which had occurred, particularly in relation to the errors observed. Field notes were taken by the principal researcher (V.R.T.) during and immediately after both the simulated scenarios and the debrief discussions.
We conducted our analysis using the scenario video recordings, debrief audio recordings, and field notes. The video recordings of all 18 scenarios were reviewed by two clinician researchers (V.R.T. and S.E.S.). During video review, identification of an error prompted the researchers to pause the video and discuss the error in detail with each other, informed by referral to current resuscitation guidelines. All errors were attributed to the team of doctors rather than to a single participant, except when evidence existed for the same error having been made by more than one participant for different reasons. In such cases, the richness of the data was preserved by giving individual consideration to the actions of each participant, recorded as distinct errors. Observation of a single participant involved noting aspects of behavior, along with verbal and body language clues which helped to explain erroneous actions.
Immediately after review of each video recording, both researchers listened to the audio recording of the corresponding debrief in an attempt to glean additional information pertaining to the participant intentions. Audio recordings were chosen in preference to transcriptions because the presence of intonation or emphasis helped researchers to more accurately interpret the meaning of some participants’ comments or questions. After assimilation of the evidence, each error was reviewed in the context of the scenario and debrief to determine whether there was sufficient evidence to attribute the error to a single cause. In cases where a single cause was evident, the error was coded into the GEMS framework by template analysis.25 Errors which could not be coded into the GEMS framework were coded inductively.
Ethical approval was waived by the South East Scotland Research Ethics Service. We obtained written consent for audio and video data collection and publication of anonymized results from all participants.
All 38 junior doctors who were invited to be involved in the study participated in 18 simulated scenarios in pairs or threes. Participants included graduates from seven UK medical schools. In total, 243 errors were identified (range 8–20 errors per scenario). Sufficient evidence was available to attribute 190 of the errors to a single cause. For the 53 remaining errors, there was insufficient evidence from the scenario, debrief, and field notes to confidently attribute the error to one of a number of possible explanations.
It was possible to classify 164 of the errors according to GEMS without modification to the framework. An additional 26 errors were classified in new categories, which we propose below as an amplification of GEMS when used in a team-based context.
The existing GEMS framework
Slips relate to the execution phase of a task, whereas lapses result from failure of the storage phase and usually occurred when there was either a time lag or distraction between the formulation and execution of the plan. RBMs stemming from both the misapplication of “good” rules (those with proven utility in a particular context) and the application of “bad” rules were identified. Good rules were often misapplied when the clinical situations presented to the junior doctors shared some common features with the circumstances in which the chosen rules are pertinent. The clinical features which indicated that the rule being applied was inappropriate tended to be ignored by the junior doctors.
KBMs were, by definition, associated with situations that the junior doctors had not previously encountered. They related to many forms of knowledge, including clinical aspects, hospital systems, and medical equipment. Violations occurred in situations when the correct procedure or protocol was known to the juniors but compliance would have introduced a time delay or the necessary equipment was not readily available. Examples of each of the types of error that could be classified according to the original version of GEMS are shown in Table 2.
Proposed modifications to the GEMS framework
Some errors occurred solely because of a preceding error; we have thus termed them “compound errors.” This category includes errors stemming from the misunderstandings of others, as well as from a junior’s own misperception or misinterpretation of information. Two examples of compound errors are shown in Table 3.
At times, there was disagreement between the junior doctor participants as to the most appropriate course of action. The data revealed a second error type which has not been previously described in association with GEMS: submission error. Such an error occurred when a junior doctor was dissuaded from taking the most appropriate course of action by another participant advocating less appropriate measures. This type of error is clearly only applicable in situations where multiple individuals are working toward a common goal. Two examples of submission errors are shown in Table 3.
Discussion and Conclusions
Our findings demonstrate that Reason’s GEMS provides a valid framework for categorization of the errors made by junior doctors in simulated acute care contexts. We clearly identified examples of skill-based slips and lapses, RBMs, KBMs, and violations in the data from the video-recorded scenarios and audio-recorded debriefs. We have also proposed two new types of error: compound errors and submission errors.
In their work on junior doctors’ prescribing errors, Dornan and colleagues23 modified GEMS by the addition of a category called “communication error.” This additional category was used to describe prescription errors resulting from the receipt of erroneous information from patients or other health care professionals. Within this study, we attributed all errors to the team of doctors rather than to a single participant, except when evidence existed for the same error having been made by more than one participant for different reasons. Dornan and colleagues’ “communication errors” are therefore a subset of the wider group of compound errors observed in this study.
When a junior doctor commits an error due to incorrect information provided by another health care professional, Dornan and colleagues23 have noted the inevitable consequence of the junior becoming mistrusting of information given to him or her by other members of the team. We have demonstrated a second type of compound error stemming from the misperception or misinterpretation of information by oneself. The fallibility of human perception and memory systems is well documented in the cognitive psychology literature,26 but such concepts have been much slower to penetrate medical education research and curricula design. Elevated stress levels have been shown to impede performance in a multitude of cognitive processes required in acute care contexts, including those that involve divided attention, working memory, retrieval of information from memory, and decision making.6 Recent calls for training in error recovery,27 as complementary to more popular error-reduction strategies,28 may hold the key to developing junior doctors’ abilities to recognize error in both their colleagues and themselves. Rather than mistrusting their professional colleagues, developing an awareness of how affect and emotion can influence behavior may promote patient safety by prompting junior doctors to be less trusting of their own cognition in stressful, high-stakes situations.
Submission errors are restricted to situations in which teamwork is required. In this study, all participants had the same level of education and comparable clinical experience. We must assume, therefore, that participants’ willingness to deviate from their first-choice strategy reflected a lack of confidence, either in their clinical decision making or in their ability to convince others of the correct course of action. There were times, however, when junior doctors were diverted away from an inappropriate course of action and “saved” from poor decisions by the decisiveness of their colleagues. It would, therefore, be unwise to advocate obstinacy on the part of junior doctors; instead, distributed situation awareness and shared decision making should be encouraged. In contrast to the conventional model of situation awareness (“the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”),29 distributed approaches to situation awareness recognize the dynamic interactions between the junior doctors, other health care professionals, and the patient.30 The sharing of information, ideas, and projections was conspicuously absent from the scenarios in which an appropriate course of action was traded for a less appropriate one. Within the rigid hierarchy of hospital medicine, one might reasonably assume that the junior doctors in this study may be even less willing to highlight the perceived errors of their senior colleagues than they were to challenge their peers in the “safe” environment of simulation.
This study used the observation of high-fidelity simulated practice of junior doctors trained at various institutions to inform and amplify an existing error framework. However, it is probable that we did not identify all of the errors made in each scenario, and the identification of error was likely to be influenced by our own experiences and interests. Many of the errors observed could not be attributed to a single cause because of insufficient evidence from either the scenario or debrief recording. This may reflect a lack of debriefing time, participants’ reluctance to discuss particular errors, or the complexity of decision making in acute care contexts.
It is possible that, in the artificial environment of simulated scenarios, the junior doctors behaved in ways that did not reflect their behavior in everyday clinical practice, particularly in relation to violations. The risk of such discrepancy was minimized by the use of high-fidelity simulation and the absence of senior clinicians within the scenarios. Discussions between juniors during scenarios focused on their actions rather than omissions, and, as such, errors of omission were more difficult to identify and, subsequently, classify. Consequently, scenarios containing long periods of inactivity presented relatively few opportunities for error classification. As with all forms of interview, the collection and analysis of data will have been influenced by the social context of the discussion,31 particularly the power dynamics inherent within the hierarchy of clinical medicine. Our attempts to create a relaxed debriefing environment were unlikely to have negated the inhibitory effect of senior clinician presence. The junior doctors may have chosen to amend the explanations of their actions to be consistent with the perceived agenda of the facilitator.
Implications and further work
This study demonstrates that applying GEMS to the analysis of error may help to illuminate acute care error from a new perspective and suggests that the emphasis on diagnostic error within contemporary medical education discourse gives an incomplete picture when applied to acute care error. GEMS provides a pragmatic framework that incorporates, but is not restricted to, diagnostic error. We have adapted GEMS for use in acute care, and this amplified framework may be transferable to other situations involving close team working in small groups. Compound errors and submission errors almost certainly occur in other medical and nonmedical contexts, and future work could also focus on evaluating the extent to which the amplified framework is transferable to other fields. In terms of specific error types, it would be particularly interesting to explore the contributions of factors such as personality type and self-confidence to the occurrence of submission errors.
If the survival of critically ill patients is to be improved, the behavior of the junior doctors who constitute the first responders in such situations needs to be more fully understood. The multiplicity of influences on their behavior at this crucial time,32 commonly combined with diagnostic uncertainty and high-stakes outcomes, means that errors are somewhat inevitable. The amplified version of GEMS could be used in future studies to identify the knowledge and skills that are most vulnerable to specific error types, allowing tailored educational strategies to be developed.
Acknowledgments: The authors wish to thank Dr. Jeremy Morton, Dr. Halia O’Shea, Mr. Stephen Hartley, Dr. Edward Mellanby, and Mr. Chris Winter for their expertise in scenario design, implementation, and debriefing. Thanks also to the 38 junior doctors who participated in the study and to Professor Henry Walton for his insightful comments on draft versions of this report.
Funding/Support: This work was supported by grants from the Clinical Skills Managed Educational Network and the University of Edinburgh Principal’s Teaching Award Scheme.
Other disclosures: None.
Ethical approval: Ethical approval for this study was waived by the South East Scotland Research Ethics Service.
1. Institute for Healthcare Improvement. . Improving outcomes for high-risk and critically ill patients. http://www.ihi.org/IHI/Programs/Collaboratives/ImprovingOutcomesforHigh-RiskandCriticallyIllPatients.htm
. Accessed October 13, 2010 [no longer available].
2. General Medical Council. Tomorrow’s Doctors. 2009 London, UK General Medical Council
3. Illing J, Morrow G, Kergon C, et al. How Prepared Are Medical Graduates to Begin Practice? A Comparison of Three Diverse UK Medical Schools. 2008 London General Medical Council Education Committee
4. Tallentire VR, Smith SE, Wylde K, Cameron HS. Are medical graduates ready to face the challenges of Foundation training? Postgrad Med J. 2011;87:590–595
5. Jen MH, Bottle A, Majeed A, Bell D, Aylin P. Early in-hospital mortality following trainee doctors’ first day at work. PLoS One. 2009;4:e7103
6. LeBlanc VR. The effects of acute stress on performance: Implications for health professions education. Acad Med. 2009;84(10 suppl):S25–S33
7. Flin R, O’Connor P, Crichton M. Managing stress. Safety at the Sharp End: A Guide to Non-technical Skills. 2008 Farnham, UK Ashgate Publishing Ltd:157–190
8. Kohn LT, Corrigan JM, Donaldson MS To Err Is Human: Building a Safer Health System. 2000 Washington, DC National Academy Press
9. Battles JB, Shea CE. A system of analyzing medical errors to improve GME curricula and programs. Acad Med. 2001;76:125–133
10. Molloy G, O’Boyle C. The SHEL model: A useful tool for analyzing and teaching the contribution of human factors to medical error. Acad Med. 2005;80:152–155
11. Zhang J, Patel VL, Johnson TR, Shortliffe EH. A cognitive taxonomy of medical errors. J Biomed Inform. 2004;37:193–204
12. Eva KW, Link CL, Lutfey KE, McKinlay JB. Swapping horses midstream: Factors related to physicians’ changing their minds about a diagnosis. Acad Med. 2010;85:1112–1117
13. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008;121(5 suppl):S2–S23
14. Norman GR, Eva KW. Diagnostic error and clinical reasoning. Med Educ. 2010;44:94–100
15. Eva KW. Diagnostic error in medical education: Where wrongs can make rights. Adv Health Sci Educ Theory Pract. 2009;14(1 suppl):S71–S81
16. Croskerry P. Clinical cognition and diagnostic error: Applications of a dual process model of reasoning. Adv Health Sci Educ Theory Pract. 2009;14(1 suppl):S27–S35
17. Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009;84:1022–1028
18. Jolly B, Atkinson K. To err is human. Med Educ. 2010;44:15–16
19. Reason J. Human Error. 1990 Cambridge, UK Cambridge University Press
20. Rasmussen J, Jensen A. Mental procedures in real-life tasks: A case study of electronic troubleshooting. Ergonomics. 1974;17:293–307
21. Reason J The Human Contribution: Unsafe Acts, Accidents and Heroic Recoveries. 2008 Farnham, UK Ashgate Publishing Limited
22. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35–43
23. Dornan T, Ashcroft D, Heathfield H, et al. An In-depth Investigation Into Causes of Prescribing Errors by Foundation Trainees in Relation to Their Medical Education. EQUIP Study. 2009 London General Medical Council
24. Dean B, Schachter M, Vincent C, Barber N. Causes of prescribing errors in hospital inpatients: A prospective study. Lancet. 2002;359:1373–1378
25. King NSymon G, Cassell C. Template analysis. Qualitative Methods and Analysis in Organizational Research: A Practical Guide. 1998 London, UK Sage:118–134
26. Dror I. Perception is far from perfection: The role of the brain and mind in constructing realities. Behav Brain Sci. 2005;28
27. Dror I. A novel approach to minimize error in the medical domain: Cognitive neuroscientific insights into training. Med Teach. 2011;33:34–38
28. Brannick MT, Fabri PJ, Zayas-Castro J, Bryant RH. Evaluation of an error-reduction training program for surgical residents. Acad Med. 2009;84:1809–1814
29. Endsley M. Toward a theory of situation awareness in dynamic systems. Hum Factors. 1995;37:32–64
30. Fioratou E, Flin R, Glavin R, Patey R. Beyond monitoring: Distributed situation awareness in anaesthesia. Br J Anaesth. 2010;105:83–90
31. Reeves S, Lewin S, Zwarenstein M. Using qualitative interviews within medical education research: Why we must raise the “quality bar.” Med Educ. 2006;40:291–292
32. Tallentire VR, Smith SE, Skinner J, Cameron HS. Understanding the behaviour of newly qualified doctors in acute care contexts. Med Educ. 2011;45:995–1005