Situation Awareness in Anesthesia: Concept and Research
Schulz, Christian M. M.D.*; Endsley, Mica R. Ph.D.†; Kochs, Eberhard F. M.D.‡; Gelb, Adrian W. M.D.§; Wagner, Klaus J. M.D.‖
Accurate situation awareness (SA) of medical staff is integral for providing optimal performance during the treatment of patients. An understanding of SA and how it affects treatment of patients is therefore crucial for patient safety and an essential element for research on human factors in anesthesia. This review describes the concept of SA in the anesthesia environment, including the interaction with associated medical teams. Different approaches for its assessment in the work environment of anesthesia are provided. Factors contributing to expertise in SA are described and approaches for the training of SA in anesthesia are discussed, as are types of errors that occur during the development of SA. Finally, the authors briefly present strategies to improve SA during daily anesthesia practice through altered designs of monitor displays.
SITUATION awareness (SA) is defined as “the perception
of elements of the environment within a volume of time and space, the comprehension
of their meaning and the projection
of their status in the near future.”1
It describes the ability of an individual to maintain an adequate internal representation of the status of the environment in complex and dynamic domains where time constants are short and conditions may change within seconds and minutes. The concept was first used in the field of military and commercial aviation and described in detail for the first time in the late 1980s by Endsley.1
According to the definition, SA is subdivided into three hierarchical levels: perception (SA level I), comprehension (SA level II), and projection (SA level III).2
In 1995, the concept of SA was introduced into the field of anesthesia by Gaba et al.3
Later, SA was classified as an important nontechnical skill4
and was embedded in the Anaesthetists’ Non-Technical Skills (ANTS) taxonomy and behavior rating tool. This framework was designed especially for anesthetists and identified four main nontechnical skills: situation awareness, decision-making, teamwork, and task management.5
In this review, the concept of SA is presented, and its importance for accurate decision-making and performance is illustrated. Insights into different approaches for the assessment of SA are provided and skills that are related to high levels of SA are described. Starting from a cognitive theoretical background, we identify types of error that may occur during the development of SA. Furthermore, implemented SA training strategies from other domains are described and discussed with respect to their applicability in anesthesia. Finally, we briefly present strategies to improve the design of patient monitors with respect to SA during daily anesthesia practice.
What Is SA?
The term SA
has been used with respect to individuals, teams,1
To illustrate these different approaches, a critical incident is presented:
A 68-yr-old man presented with an acute abdominal crisis and clinical signs of sepsis. General anesthesia was induced by a fourth-year anesthesia resident for exploratory laparotomy. After uneventful induction of anesthesia, a catheter was placed in the radial artery for blood pressure monitoring. An internal jugular catheter had been placed 2 days earlier. Thirty-five minutes after the start of surgery, a gastrointestinal perforation was identified as the underlying cause for peritonitis. At this point, the patient had a systolic arterial blood pressure of 95 mmHg and a heart rate of 105 beats/min despite administration of low doses of noradrenaline and 1,000 ml of crystalloids. The surgeons mentioned a diffuse bleeding tendency, and the resident started to administer fresh frozen plasma. Two minutes after starting the fresh frozen plasma, he noticed that the blood pressure had decreased substantially and that the heart rate had increased further. He looked at the surgical field and asked the surgeons whether there was any acute and uncontrolled severe bleeding. The surgeons answered in the negative. In the belief that an acute bacteremia was compromising hemodynamic stability, the resident increased the doses of noradrenaline. However, the systolic blood pressure remained low at 60 mmHg, even at a dose of 2 µg kg-1 min-1 of noradrenaline. At this point, he called for the senior attending anesthetist. At the same time, he noticed that the electrocardiogram ST segment was becoming progressively more depressed. He increased the fraction of inspired oxygen to 1.0. The resident informed the surgeons about the severe problems in maintaining blood pressure and about relevant changes in the electrocardiogram. He also indicated that a myocardial infarction could be the cause of this constellation and advised the surgeons to be prepared to perform chest compressions. A nurse was requested to prepare adrenaline for both bolus injections and continuous administration. Some minutes later, the senior anesthetist entered the operating room and, after a short phase of orientation, advised the resident to administer adrenaline. At the moment of injecting adrenaline into the central venous line, the resident noticed severe urticaria in the skin in the vicinity of the central venous line. He immediately communicated this finding to the senior attending anesthetist, and the diagnosis of anaphylactic shock caused by fresh frozen plasma was made. After administration of histamine blockers and glucocorticoids, the remainder of the surgical procedure was uneventful and the patient recovered without any neurologic deficits.
SA Level I
SA level I (perception of the patient’s state) was formed in the aforementioned case by collecting relevant information from monitoring the patient’s vital parameters (low blood pressure, heart rate, oxygen saturation, electrocardiogram), the surgical field (no bleeding), and the appearance of the patient (urticaria). Additional information was acquired through communication with the team (bleeding tendency, intestinal perforation, no acute severe bleeding).
A patient’s condition is often static or changes very slowly during the maintenance phase of uneventful anesthesia. However, during induction and emergence from anesthesia or during a critical incident, they may change substantially in a very dynamic manner. The appropriate distribution of attention during such events determines the sensory input to the anesthetist and is therefore an important underlying process of SA level I.7–9
In terms of the critical incident described above, a failure of attention would have occurred if the anesthetist had not noted the urticaria (SA level I) during the administration of adrenaline via
the central venous catheter, even though this sign had been within his perceptual field. Such failures occur because of attention being directed at other information that appears more relevant.
SA Level II
SA level II encompasses the anesthetist’s comprehension and understanding of these variables. Accordingly, anesthetists integrate SA level I knowledge with their long-term memory about medical knowledge, medical guidelines, mental models of physiology and pharmacology, and alterations of the physiologic state by specific surgical procedures. With good level II SA, the anesthetists would recognize that anaphylactic shock is the cause of the low blood pressure and urticaria. Together with the senior anesthetist, the correct diagnosis was made based on the patient’s state and the underlying data. Thus, SA is more than perceiving data, it is integrating those data to understand what they mean and what is pertinent for the current situation.
SA Level III
SA level III is the highest level of SA and is the level at which anesthetists will project the expected future development of the patient’s status, which is crucial for early and adequate proactive management of resources to meet the goals of therapy. In the Case Example, administering fresh frozen plasma to address a bleeding tendency and advising the surgeons to be prepared for chest compressions are examples of decisions and communication that emerge from SA level III.
In conclusion, the anesthetist’s ability to actively direct their attention toward the main sources of information and to correctly understand and interpret the information they have at hand, enabling them to anticipate the future development of the case, plays a substantial role in treating critical incidents effectively.1
SA is considered indispensable for subsequent decision-making, teamwork, and task management3
and is therefore crucial for patient safety.
Anesthetists are only one part of an interdisciplinary team of individuals engaged in the common project of “patient treatment.” Therefore, the traditional SA concept where the object of interest is the cognitive process within the anesthetist’s mind has been extended. The aim is to determine how and to what degree SA is present across a team (team SA) and to study the mechanisms that are used to share SA within and between teams. Team SA is defined as “the degree to which every team member possesses the SA required for his or her responsibilities.”2
In this view, it is not sufficient that a team member has a piece of information; rather, it is necessary that each person be aware of circumstances that are relevant to their respective roles and responsibilities. However, not all the information needs to be shared with every involved individual. There would be cognitive overload if everyone’s SA was entirely the same. Rather, effective team performance requires that SA is shared only for those subsets of the information that are relevant to selected team members. Shared SA is therefore defined as “the degree to which team members have the same SA on shared SA requirements” (fig. 1
In a highly effective team approach, decisions should be based on information derived from all team members. This allows for an efficient plan of action; otherwise, a breakdown of performance would occur when team members are not able to anticipate which help is needed by the others.11
According to Endsley and Jones, team SA is affected by the following: (1) shared SA requirements, which can be specifically defined for the team; (2) shared SA processes, including group prioritization and contingency planning, a group norm of information sharing and working to develop a common understanding, and active cross-checking and questioning of information across the team; (3) shared SA mechanisms (such as shared mental models among the team)13
; and (4) shared SA displays (which can include auditory, visual, and other displays), verbal communication, nonverbal communications, and shared environment (fig. 2
These factors come into play in building team SA both for teams that are co-located (such as in an operating room) or for teams distributed in time or space such as with a shift change or in a recovery room, where team members may be coming and going (distributed team SA).
Referring to the critical incident described in the beginning of this section, the information about the developing urticaria was explicitly shared with the attending anesthetist (and not with the surgeons, although they may have perceived this information) with the aim of enhancing team SA. There was also cross-checking and questioning of information when asking whether there was acute severe bleeding. The patient monitor not only served for monitoring the patient but also was an important tool for sharing SA between the anesthetists. Finally, the team, consisting of two anesthetists, surgeons, and nurses, used shared mental models (differential diagnosis of shock, pathophysiology, and therapeutic goals) that made the development of team SA more efficient.
Recently, a successful transfer of team SA from theory into practice was undertaken by using the Surgical Safety Checklist for operating teams in a prospective multicenter study. In terms of team SA, fulfilling the checklist required the teams, among other things, to define a team goal and to actively share SA about the patient, the planned procedure, and potential critical events. Using this approach, Haynes et al
. demonstrated reduced perioperative complications, fewer surgical-site infections, fewer unplanned returns to the operating room, and reduced mortality.15
The model of individual and team SA is quite different from the concept of distributed SA (DSA) that was presented by Stanton et al.
Stanton et al.
argued that a concept of DSA that encompasses not an individual but a complex nonstatic system including human and nonhuman subsystems, and the interactions between them, may provide a better understanding of how certain output (performance) evolves from a team.6
Accordingly, in distributed teamwork (e.g.
, operating room), cognitive processes occur at a system level rather than an individual level.6
Recently, Fioratou et al.
reviewed the DSA approach for its applicability in anesthesia.10
The authors argued that anesthetist, surgeon, patient, and patient monitoring are subsystems that interact in a dynamic manner, permanently, and both implicitly and explicitly. For example, revealing the patient’s state to the anesthetist, the anesthesia machine display is a physical mediator between the patient and the anesthetist and can be classified as an indirect communication device between them.10
Fioratou et al
. suggested that the traditional SA approach includes only simple patient monitoring for gaining SA and concluded that the DSA approach captures better the mechanisms that led to ineffectively distributed information. However, this statement is controversial because existing theories of SA and team SA clearly state that people gain much of their SA from the objects and tools in their environment, along with communications from other teammates and directly from the environment itself.2#
In the course of the critical incident described earlier under What Is SA?, the attending anesthetist monitored the electrocardiogram and, at a certain time point, advised the resident to administer adrenaline. During administration of the drug, the resident noticed urticaria and informed the consultant, at which point the assumed diagnosis was changed and adequate therapy was provided. In DSA parlance: (1) the “knowledge” within this system is distributed (some knowledge is provided by the patient him- or herself [urticaria], other knowledge is technically mediated [electrocardiogram] or provided by individuals in the team; and (2) there was implicit communication rather than active sharing of mental models about differential diagnoses to be considered. For illustration of the DSA perspective, another example of the involved subsystems in the system operating room and related interactions is given in figure 3
In conclusion, DSA focuses at the system’s SA and analyzes the interactions between the subsystems (human and nonhuman) to understand how SA evolves. In contrast, the traditional SA approach focuses on individuals who together develop team SA and shared SA. The team SA model provides an explicit review of the various processes used by effective teams for gaining SA. Moreover, it describes additional ways in which teams can obtain shared SA of relevant information, such as by observing the same information in a shared environment or through effective shared mental models built up through training and common experience.
Assessment of SA
The assessment of SA from both simulation-based and clinical settings can be used to quantify training effects. It may thus provide an understanding of the factors contributing to SA. Below, we present different tools that have been used to assess SA in anesthesia or that may be suitable for anesthesia but have not been applied so far. There are both direct and indirect methods. Direct methods measure SA itself; indirect methods make use of process indices, behavior, and certain performance parameters that are considered to be indicators of SA.
Direct Measures of SA
Direct SA measures can be derived objectively or subjectively. A direct and objective SA measure that has been validated and applied across different domains, including the field of medicine, is the Situation Awareness Global Assessment Technique (SAGAT).17–22
SAGAT was designed to assess the different levels of SA using scenario-specific questionnaires when stopping at random points during a simulated scenario (table 1
Only one reported study in the field of medicine validated a direct assessment of SA. Hogan et al.
used SAGAT in an Advanced Trauma Life Support curriculum that investigated 16 volunteers with different levels of expertise.18
Each subject had to resolve three different standardized scenarios. At random points, the scenarios were frozen and the subjects answered SAGAT questions that assessed their knowledge of each aspect of the situation that is relevant for SA. SAGAT scores were found to be significantly dependent on the subject’s expertise and correlated significantly with traditional checklist performance scores.
The scenarios used in this study were designed for Advanced Trauma Life Support training and thus required the participants to use a highly standardized algorithm. In this setting, SAGAT provided good validity. A limitation with respect to describing the impact of SA on performance is that the correlation between SA and performance scores was not analyzed independently from the subject’s expertise. Unfortunately, the authors did not specify the SA queries designed for each SA level. Thus, it remains unclear to what extent each SA level contributed to the overall SAGAT score. The process of designing and scoring SAGAT queries, especially in the more advanced levels of comprehension
, may be much more challenging for anesthesia critical incident training, where the scenario development depends on the decision-making process of the trainees. Another aspect of interest is the intrusiveness caused by the obligatory freezes of the scenario. In the study by Hogan et al.
, only 12% of the participants were reported to be adversely affected by the freezes with respect to concentration and performance. Research in other domains confirms that the intrusiveness of the freeze procedure is negligible and does not affect objective performance measures in the simulation.20
Reader et al
. used a completely different approach to directly investigate team SA.26
Forty-four members of an intensive care unit were assessed for their accuracy in predicting patient outcome within the next 48 h (SA level III). For this purpose, the participants were asked to rate the likelihood of patient discharge, patient deterioration, need for ventilator support, and patient survival. These ratings were compared to the respective patient outcomes. This approach, which objectively scores SA, is similar to SAGAT. However, its application may be difficult in highly dynamic situations, where changes occur within minutes rather than days such as is the case during critical incidents in anesthesia.
In other domains outside of health care, subjective SA measures have been developed and were compared to SAGAT with respect to validity and reliability.27–29
The Situation Awareness Rating Technique is such a tool.30
Participants rate post hoc
10 dimensions (table 2
) on a seven-point Likert scale, with the endpoints 1 = low and 7 = high. The tool is nonintrusive and uses queries that are not required to be adapted to scenario content. Post hoc
ratings allow its application in clinical settings. These are major advantages for use in anesthesia. However, SA measured subjectively through the Situation Awareness Rating Technique was found to have no correlation with objectively measured SA using SAGAT.31
Therefore, significant concerns with respect to content validity and sensitivity of the Situation Awareness Rating Technique29
and a high correlation with workload measures have been raised.33
A major problem of retrospective self-ratings of SA is that the subjects may have acquired significant additional knowledge about scenario content on the comprehension
level of SA after having resolved a specific scenario. This is thought to be an important source of bias.8
In addition, people are often unaware of their incorrect assessment or of missed key information (i.e.
, they don’t know what they don’t know), thus limiting the validity of their ratings.8
In principle, SA can be assessed objectively and directly in anesthesia using SAGAT. However, valid SAGAT queries that investigate SA in dynamic decision-making–dependent anesthesia scenarios must be developed. Because SAGAT requires freezing the situation, it is only suitable in simulation environments. Subjective ratings are less intrusive but may suffer from insufficient validity. In real-world settings, other techniques may be preferable, such as concurrent probes,32
that provide SAGAT-like queries verbally one at a time during the simulation without freezing the scenario; or process measures, which are discussed in the next paragraphs. Vidulich, by reviewing a wide number of studies using SAGAT, found that it had good sensitivity and provided detailed diagnostic information beyond what was available from performance measures alone.35
Assessment of Behavior as an Indirect Indicator of SA
The ANTS scale was designed and validated for the global assessment of nontechnical skills and evaluates behavioral markers of SA.5
Trained observers use a standardized questionnaire to rate the participants.36–38
One of four subcategories evaluates behavior (gathering information, recognizing and understanding, anticipating) related to SA. This scale has been used to assess training effects39
and to compare the effectiveness of different debriefing methods.41–44
A major limitation of the ANTS scale is that interrater reliability varies widely across studies.36
This is most likely attributable to difficulties in handling the scale and differences in the quality of observer training. Two days of training are recommended for raters who are already familiar with the concept of nontechnical skills.38
Another scale applicable to the subjective evaluation of nontechnical skills is the Ottawa Global Rating Scale, which is a seven-point Likert scale with one item designated to assess SA.45
Compared with the ANTS scale, data about validity and interrater reliability are scarce.
Both the ANTS scale and the Ottawa Global Rating Scale use observer ratings and are applicable in clinical settings. They are intended to assess nontechnical skills in general and are not designed to specifically assess SA. So far, it remains unclear to what extent the behavioral markers in these scales provide an independent assessment of SA and whether the observer ratings may be influenced by ratings on the other aspects of the scales (decision-making, teamwork, and task management). Future research is needed to validate SA-related behavioral rating scales against direct measures of SA in medical settings.
In another analysis of behavior, apart from the above-mentioned rating scales, Hazlehurst et al.47
described how SA evolved from coordinated communication between surgeon and perfusionist during cardiac surgery. The authors identified six types of verbal exchange (i.e.
, direction, status, alert, goal-sharing, problem-solving, and explanation) that facilitated team performance by enhancing SA. This is an example of research that examines SA processes but not the level or quality of SA achieved.
Assessment of Performance as an Indirect Indicator of SA
According to Endsley’s theory of SA and the findings of Hogan et al.
good performance is generally based on good SA (unless one is very lucky). In anesthesia, a variety of measures have been applied to assess performance. Simulation technology provides highly realistic work environments that permit comparative evaluation of performance using standardized scenarios.48
In a variety of studies, including pediatric anesthesia,49–52
checklists have been applied to assess the quantity of key diagnostic and therapeutic tasks. Other measures of potential interest53
are the assessment of simulated patient outcome54
or the time between the onset of a problem and its detection or definitive treatment.56–59
Although such studies are useful for assessing the overall performance associated with a new display or training curricula, they do not directly provide information about the subject’s SA itself and the features of the displays.20
Assessment of Gaze Behavior as an SA Process Measure
SA may also be inferred from techniques such as eye tracking to understand the types of processes that people use to gather information within their environment. It can be used to determine whether attention is given to specific data that are relevant for developing accurate SA. The idea is that particular information-gathering techniques may be more effective than others, and that understanding what the person is viewing (or neglecting to view) can provide insights into their SA.
For example, Liu et al.
observed anesthetists’ gaze behavior to assess the impact of a head-mounted holographic display of monitoring data on the distribution of visual attention. Similar to their simulation study,58
the participants with the head-mounted display spent more time looking at the patient and the surgical field and less time looking at the monitoring equipment because the principal information was available without looking at the patient monitor.60
However, observation of video recordings of gaze direction may lack sufficient accuracy to assess SA because the observer cannot differentiate whether the participants looked at the patient or at the parameters provided by the head-mounted display that are projected and overlap the patient in space.
Recently, a head-mounted eye-tracking device has been used to objectively assess visual attention as an underlying process of SA during simulated critical incidents.9
Whether a critical incident occurred or not had a significant impact on how visual attention was distributed. This study showed that gaze behavior is adapted to the needs of developing and maintaining SA and that this adoption processes depend on experience.
Thus, the anesthetist’s distribution of visual attention determines what is in the anesthetist’s perceptual field and therefore contributes to the sensory input. Eye tracking provides an objective assessment of visual attention with high temporal and spatial resolution. How and to what extent expertise influences gaze behavior and whether there is a specific gaze behavior that is optimized to enhance SA in anesthesiology remains unclear and requires further investigation. So far, eye tracking and other SA measures have not been applied together in studies of healthcare providers. A combination of eye tracking and SAGAT could simultaneously determine what information had been seen
and to what degree this information had been perceived
by the individual. In other fields, eye tracking has been used to study inattentional blindness with respect to eye movements.61
Anesthetists (and physicians of all other specialties) establish SA in part through information available from displays, direct observation, and communication with the team. It is evident that the way information is gathered and processed is highly dependent on expertise.18
Obviously, some anesthetists are much more capable of gaining high SA levels, whereas others may make insufficient use out of the same information. A 10-fold difference in SA levels was found across 25 highly experienced pilots; that is, the SA of the pilot with the highest score was 10 times that of the lowest scoring pilot.63
This significant range in SA scores was found to be highly reliable, with test–retest ratios of 0.99, 0.98, 0.98, and 0.92 on four pilots who participated in comparable sets of 24 simulation trials. These differences were found to be attributable to differences in cognitive capabilities, including attention-sharing abilities, pattern-matching abilities, and spatial abilities. In addition, the knowledge bases and skills that are acquired with training and experience are very important for the effective development of SA. Experts are characterized by their ability to gain SA with less effort, faster, and more completely, and will achieve higher levels of comprehension and projection.64
The SA model suggests the following individual factors that determine whether good SA is achieved64
1. Capacity: Each individual has a limited capacity to attend to all the relevant information.
2. Working memory: Developing SA requires the working memory to store, integrate, and process the perceived information and to continuously update the current mental model of what is happening. As working memory’s capacity is exceeded, critical information may be forgotten or may not be properly integrated for development of SA at higher levels.
3. Goal-driven processing alternating with data-driven processing: In a top-down goal-driven process, the anesthetist’s goal (the ideal state of the patient) will direct which aspects of the environment are attended to. In a bottom-up data-driven process, the anesthetist’s attention is distributed across all relevant information and his or her attention will be captured by salient or key information that may indicate that the strategy to reach a goal has to be changed or to change the goal itself. Endsley considers an ongoing cycling between goal-driven and data-driven processing to be a key feature underlying SA. The importance of this is taught in Anesthesia Crisis Resource Management training: to reevaluate a situation at regular intervals.
4. Expectation: Expectations, whether justified or not, affect the visual search for information, the perception of that information and, as a consequence, the higher levels of SA.
5. Mental models:
These consist of cognitive mechanisms for interpreting and projecting events in complex domains. They are part of the long-term memory and serve to circumvent the limitations of the working memory.1
During the introductory Case Example described earlier, a useful mental model may have comprised the five or six differential diagnoses of the causes of shock with its typical signs. If the anesthetist possesses such a mental model, he or she might have actively searched for urticaria to verify or exclude this diagnosis but would not have to figure out the meaning of observed urticaria in the working memory.
6. Pattern matching: In a demanding situation that is similar to a prior episode, the process of developing good SA will be faster, because much information associated with this prior episode can be recalled. Cognitive workload is much less for anesthetists who have good pattern-matching abilities. In the Case Example, the resident saw the ST segment of the electrocardiogram becoming increasingly negative. Together with ventricular extrasystoles and a low blood pressure, he or she may have had in mind a prior situation with a patient suffering from cardiogenic shock during his or her work as an emergency physician. Based on this experience, the resident immediately concluded that a myocardial infarction and cardiogenic shock were the cause of the patient’s deterioration. Such pattern matching can be almost instantaneous and effortless.
7. Automaticity: Automaticity reduces the necessary cognitive load to perform a certain physical or cognitive task and thus frees up the mind, providing more resources for attention and working memory. For example, if the electrocardiogram suddenly becomes flat, a more experienced anesthetist may automatically look at the pulse oximeter to verify whether the electrocardiogram is an artifact or truly represents an asystolic patient. Alternately, performing repetitive physical tasks, such as suturing, automatically leaves attention free for thinking about other problems during the operation. These actions can occur without using working memory, leaving it free for developing SA.
8. Learned skills: Finally, there are numerous learned skills that are domain-specific and that support the development of accurate SA; for example, medical students learn how to interpret specific alterations of the morphology of an electrocardiogram and to derive a diagnosis.
To bring all of the above items together, we have adapted the work of Endsley2
and Gaba et al.3
to create a framework of SA in health care that links SA to performance (fig. 4
From a theoretical point of view,2
failures in anesthesia, intensive care, and emergency medicine may occur at each SA level, and SA can be incomplete, inaccurate, or both: information may not be detected or gathered correctly (SA level I), or the situation is not understood, although all of the relevant information is detected (SA level II); or the future is not correctly anticipated, although the situation is understood (SA level III). Consequently, affected SA elements that are involved in decision-making may lead to poor performance.
SA Level I
In SA level I, the anesthetist fails to perceive relevant information (incomplete SA). This type of error can result from a lack of detectability (e.g., because of a visual barrier or auditory masking), it can be a failure of the system design to make the information available, or the information is available but not attended to for various reasons. In the course of the illustrative critical incident, the anesthetist did not notice the urticaria because of a visual barrier. Another type of error at level I is inaccurate SA, in which a perceived value does not represent reality. For example, during carbon monoxide intoxication, the measured and thus perceived values of oxygen saturation may be falsely high as assessed by pulse oximetry. The data may all be present, but the attention of the anesthetist is focused on only a subset of it, neglecting important data. In other cases, a syringe may be misread and a wrong drug administered because the labeling is very similar to that of another drug.
SA Level II
SA level II errors occur when available cues are not integrated and understood correctly. Similar to level I, SA at level II can be either incomplete or inaccurate. Endsley discussed a variety of contributing reasons: missing mental models, selecting the wrong mental model, or a failure to recognize prototypical situations. Klein described errors in medical decision-making that evolved from interpreting a combination of successive signs into an existing diagnosis even though the symptoms clearly indicated a different diagnosis.65
This is an example of using a wrong mental model to interpret information. If there is no mental model available for a given situation because it is new to the anesthetist (missing mental model), SA level II has to be developed in the working memory and errors can occur because of working memory limitations, especially under high cognitive workload. The individual simply cannot figure
out what is happening fast enough during an event that has never been encountered before.
SA Level III
The prediction of a future state is incomplete or inaccurate even though the current situation is fully understood. An example is when the anesthetist fails to consider that a blood transfusion may be necessary based on the type of operation and therefore blood products are not ordered in a timely manner. In another case, the anesthetist may not consider the likelihood of a cardiac event even though the decreasing blood pressure and related electrocardiographic problems are clearly noted.
For novices, gaining adequate SA is very demanding and frequently incomplete, and may be faulty because of limited attention and working memory capacities or less developed mental models. How can we make them experts in SA abilities?
To date, it remains largely unclear how and to what extent relevant skills for the development of SA are acquired and what type of training should be implemented to achieve an optimal increase in SA abilities in healthcare providers. To some degree, some of the individual factors that facilitate the development of SA are taught during residency and other training forums such as Anesthesia Crisis Resource Management training.
In a first attempt to develop and evaluate a formal training of SA in anesthesia in final-year medical students, a recent study21
used SAGAT during simulated scenarios of severe sepsis in a pretest/posttest design. The effects of the simulator training were compared to a classroom-based approach to train SA skills. During the debriefing of the simulator training, the diagnosis and treatment of sepsis based on the respective scenario was taught; however, no nontechnical skills, such as SA, were considered. The classroom-based approach consisted of a theoretical lecture about SA, a video-based discussion about the team’s SA during a cardiac arrest situation, and different psychological exercises. With respect to SAGAT scores, the simulator training but not the classroom-based SA training was found to slightly improve participants’ SA. In the SAGAT queries, comprehension questions were not considered, and projection questions were used only during the second scenario freeze.
In the past decade, there have been numerous efforts to implement formal SA training with the aim of enhancing performance in aviation and other domains.66
These training approaches have attempted to improve cognitive skills and to build the cognitive structures that are necessary for high levels of SA. In the next paragraphs, we present training approaches that have been applied in other domains that may be considered for improving SA in anesthesia.
One approach was to train individuals and teams about SA and hazards that evolve from inaccurate SA.67
To realize this, a combination of classroom-based instruction and individual or team exercises was applied. Robinson,68
for example, described a 2-day program for training pilots in SA that combined training on SA with error management strategies.69
After receiving the training, the pilots were rated as having significantly better team skills and established significantly more SA at level III. Later, this program was implemented at two major European airlines. Ninety-nine percent of the pilots reported the training to be very useful, and the majority of them indicated that they were using the training during their daily work.71
Another approach focused on the underlying knowledge and skills for use of good SA by means of computer-based training programs that allow training of relevant cognitive mechanisms. McKenna and Crick improved SA level III in drivers by presenting video clips of driving scenes. During freezes, the trainees were encouraged to project what might happen next.72
Bolstad et al.
designed computer-based training modules to train specific basic skills in novice general aviation pilots. The modules addressed the training of checklist completion, communication, psychomotor skills, attention-sharing skills, preflight planning, and contingency planning.74
Another example is the work of Strater et al
who presented a personal computer-based tool to improve SA in infantry officers. Using a multimedia computer-based approach, trainees were exposed to many situations within a short period with the aim of generating memory for prototypical situations and recognizing its critical cues. Moreover, the tool requested the trainees to make complex operational decisions using a “What? So what? Now what?” approach. That is, the trainees learned to think about their SA needs in terms of: What
information do I have or do I still need? So what
does this information mean in terms of my goals? Now what
will happen next or what should I be prepared to deal with in the future situation?76
In a study of this training approach, Strater et al.
found that infantry leaders who received the training were significantly less likely to fall into traps in the testing scenario that would have had them attack a civilian refugee encampment.76
Virtual Environment Situation Awareness Review System
Another interesting approach applicable in anesthesia simulation environments is the Virtual Environment Situation Awareness Review System.66
It consists of a behavioral rating tool that assesses individual as well as team actions, a communications rating tool that evaluates team communications, and an SA query tool that allows direct assessment of individual and team SA.77
The rating results are presented in a postsession debriefing so that trainees understand the degree to which they actually were or were not able to form SA during specific events and why. This is thought to provide a more effective debriefing method with respect to SA training, as this creates a useful template for the debriefers and provides actionable feedback. Initial testing of this approach received positive subjective ratings from participants regarding the tool utility, but it has not yet been validated extensively.66
Important training principles66
include training in task management and prioritizing, self-checking behaviors, basic skills to find the relevant information, common types of SA errors, attention-sharing ability, critical behaviors and communication that are essential for team SA, development of mental models, and feedback on how good or poor the trainee’s SA is along with ideas for improvement, building better mental models to develop team SA more directly, and training teams in cross-checking their SA.
Interested readers are referred to the threat and error management systems approach that was developed with the aim of improving aviation safety. This approach provides a framework for understanding the role of human factors in complex dynamic environments and describes additional training principles that support the establishment of a safety culture.79
Applicability in Anesthesia
The findings from a single study in anesthesia,21
but also from other domains such as aviation, suggest that focused SA training is a promising approach for increasing healthcare providers’ ability to form individual SA and team SA and thus to promote better performance during patient care. All of the aforementioned approaches seem applicable in anesthesia. Apart from that, anesthesia can draw on more than two decades of experience in high-fidelity patient simulation and related debriefing techniques that are valuable tools for SA training.
These approaches go far beyond Anesthesia Crisis Resource Management training. Although Anesthesia Crisis Resource Management training includes a discussion of SA as an important factor, SA training focuses extensively on the development of key cognitive knowledge and skills that actually can increase levels of SA. A first step toward SA training in anesthesia may consist of providing theoretical knowledge about the concept of SA and thus sensitizing anesthetists to the factors that either compromise or convey the development of SA. For the implementation of formal SA training in a simulation-based setting, the debriefing should focus on the skills and behaviors that are associated with good SA. Computer-based training techniques as discussed above can be adapted for the challenges of SA in anesthesia. However, detailed and systematic research on common and critical problems that affect SA in health care is currently lacking. Therefore, the implementation of domain-specific and therefore goal-directed SA training in anesthesia still requires considerable research.
Displays of Patient Monitors
Observing the patient monitor, especially during dynamic phases of the operation, is an important but time-consuming activity and is a matter of routine during anesthesia.9
As stated earlier, the patient monitors provide much relevant information required for establishment of accurate SA. Therefore, another field of great interest and of growing scientific effort addresses the design of patient monitors.
To date, the patient monitors in routine use are based almost exclusively on the single-sensor single-indicator design. With such displays, anesthetists must monitor multiple variables and mentally integrate them to obtain the actual status of the patient’s situation. With limited attention and working memory, this can become overloading, particularly in highly dynamic situations, or can create vigilance problems in other cases.80
Thus, as more single-sensor single-indicator variables are added to the monitor screens, the expected increase in SA is doubtful.80
With the objective of overcoming these limitations, several empiric studies have attempted to enhance SA by providing additional or modified monitoring devices that assess performance but not SA itself. In 2008, Görges and Staggers reviewed studies that evaluated different physiologic monitoring displays.56
In 18 of 31 studies, the participant’s performance, assessed as time to detect or time to make a diagnosis or time to clinical decision, was found to be superior with a novel display using graphs or sounds. Accuracy in clinical diagnosis and decision-making was improved in 13 of 19 studies.56
In a more recent study, Charabati et al.57
compared four different graphic display designs for a monitoring system that presented integrated data with the aim of providing information about the depth of analgesia, hypnosis, and neuromuscular blockade. The primary outcome parameter was the time to detect and correctly verbalize the problem, and a secondary outcome parameter was workload as assessed by the National Aeronautics and Space Administration Task Load Index. With a combined numeric–graphic display, the detection times were significantly shorter and the workload was lower. Interestingly, this display resembled mostly the traditional single-sensor single-indicator design consisting of a split screen with curves on the left side and numeric values on the right side.
In another attempt to enhance anesthetists’ SA, Liu et al.
and Sanderson et al.
investigated auditory displays and head-mounted displays in simulator environments58
and in the real operating room.60
The auditory displays increased the number of potentially harmful events detected,58
whereas the head-mounted display did not result in the detection of more events or in a faster detection of those.58
Distracted anesthetists wearing a head-mounted display detected events even more slowly. According to the authors, this can probably be attributed to inattentional blindness.58
Head-mounted displays can inadvertently occlude information because of both visual interference and attentional shifts associated with these displays, thus reducing their benefit for many applications.81
Apart from these studies that used performance markers to investigate the impact of new monitoring devices, two studies have been conducted in anesthesia that evaluated the impact of new monitoring devices using SAGAT. With respect to anesthetists’ SA, Zhang et al.
compared a traditional single-sensor single-indicator display and an object display that showed functional cardiovascular physiology by the integration of hemodynamic variables.19
Four simulated test scenarios consisting of myocardial ischemia, arrhythmia, hypovolemia, and bronchospasm were designed. The authors did not find the object display to be superior with respect to SA and performance. It was assumed that the general unfamiliarity with the object display even after a training period posed a major problem. As a limitation, the SAGAT queries used by Zhang et al.
did not include projection
In another attempt to enhance SA, Ford et al
tested a vibrotactile belt that vibrated at four different points indicating changes in ventilation volumes and pressures. The hypothesis tested was that the belt would enhance the anesthetist’s SA by providing information in a way that could be processed simultaneously with the highly loaded visual sense and therefore result in better performance during the management of an anaphylactic reaction. The vibrotactile group was found to administer epinephrine significantly faster than the control group. Interestingly, no differences in SA were detected between the two groups. A limitation was that SA was measured using SAGAT in a post hoc
assessment based on viewing interrupted video recordings on a split screen that showed the participants acting in the simulator environment and the simulator’s vital signs. When SAGAT is used for a post-trial assessment, there is some risk of bias, particularly for assessment of the higher SA levels, because the subjects’ knowledge about the scenario content and development is higher when the scenario completed. Post hoc
assessment is also limited in that it can only reliably collect SA at the end of the trial.
Kiefer and Hoeft mentioned the methodologic shortcomings of studies that investigated new displays because of very short instruction times. Instead, the authors suggested several months of training and use before unbiased testing.80
Endsley and Jones recommended that the traditional display presentation (SA level I data) also should remain available along with such integrated displays, as one still needs to be able to understand individual parameters when digging deeper into situations.82
Good SA is crucial for decision-making, which leads to increased performance in patient treatment during anesthesia, intensive care, and emergency medicine. So far, limited attention has been paid to systematic research about SA in anesthesia. The ability to develop SA accurately, completely, and quickly increases with the anesthetist’s expertise. Important skills needed for this process are acquired with experience and training. A key point is that it is possible to more rapidly and consistently boost these skills and thus performance by focused training that is specifically designed to improve specific SA skills. For the development of formal SA training programs, it is a prerequisite to first identify anesthesia-specific factors that may affect individual and team SA in both research and clinical settings, so that such training programs can be tailored appropriately. More research on the role of SA in anesthesiology, both in simulations and in real work environments, that use direct and objective measures of SA and performance are needed to help support this process.
The design of tools for information presentation in health care can increase SA and constitutes another field of potential development. In this review, we focused briefly on the design of patient monitors and discussed studies that demonstrated increased performance because of novel displays. Facilitating the development of SA should be a primary target when new monitor displays are designed. Research from other domains demonstrates that displays can directly provide information on level II SA and level III SA, thus reducing both workload and errors.82
In anesthesia, low-level data could be embedded in physiology-based models, for example, to provide a more intelligent presentation of information (e.g.
, graphically and in only one region of interest, thus reducing the anesthetist’s workload). Another attempt may consist of providing information that directly represents SA at higher levels by integrating the single-sensor single-indicator data using physiologic models—that would be a monitor that “understands” the patient’s state to a certain degree and can support comprehension and projection.
In conclusion, SA has to be recognized as a critical component of decision-making that has a direct effect on performance in medical care. A variety of skills and behaviors that influence SA have been described that are based on research from a wide variety of domains. These findings are generally transferable and applicable to anesthesia, but domain-specific factors that are important for SA in this domain have not yet been investigated. Thus, there is an opportunity and a need for much more research in the perioperative setting in terms of individual SA and in terms of team SA. Because all of our decisions are based thereupon, the development and evaluation of new technology should take into account the effects on SA. Finally, various training approaches from other domains designed to improve SA abilities can be adapted to the needs in anesthesia to more rapidly improve this important competency in anesthetists.
# Schulz CM, Kochs EF: Distributed situation awareness: What is really new? e-Letter to the Br J Anaesth 2012, published 19 March. Available at: http://bja.oxfordjournals.org/content/105/1/83.full/reply#brjana_el_8549
. Accessed.[AQ: Please provide access date for the URL.] Cited Here...
1. Endsley MR. Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society 32nd Annual Meeting. October24–28, 1988 Santa Monica:pp 97–101
2. Endsley MR. Towards a theory of situation awareness in dynamic systems. Hum Factors. 1995;37:32–64
3. Gaba DM, Howard SK, Small SD. Situation awareness in anesthesiology. Hum Factors. 1995;37:20–31
4. Fletcher GC, McGeorge P, Flin RH, Glavin RJ, Maran NJ. The role of non-technical skills in anaesthesia: A review of current literature. Br J Anaesth. 2002;88:418–29
5. Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists’ Non-Technical Skills (ANTS): Evaluation of a behavioural marker system. Br J Anaesth. 2003;90:580–8
6. Stanton NA, Stewart R, Harris D, Houghton RJ, Baber C, McMaster R, Salmon P, Hoyle G, Walker G, Young MS, Linsell M, Dymott R, Green D. Distributed situation awareness in dynamic systems: Theoretical development and application of an ergonomics methodology. Ergonomics. 2006;49:1288–311
7. Endsley MR, Garland DJEndsley MR, Garland DJ, Mahwah NJ. Theoretical underpinnings of situation awareness: A critical review, Situation Awareness: Analysis and Measurement. 2000 Mahwah, NJ Lawrence Erlbaum:pp 3–32
8. Bolstad CA, Cuevas HM. Integrating situation awareness assessment into test and evaluation. ITEA J. 2010;31:240–6
9. Schulz CM, Schneider E, Fritz L, Vockeroth J, Hapfelmeier A, Brandt T, Kochs EF, Schneider G. Visual attention of anaesthetists during simulated critical incidents. Br J Anaesth. 2011;106:807–13
10. Fioratou E, Flin R, Glavin R, Patey R. Beyond monitoring: Distributed situation awareness in anaesthesia. Br J Anaesth. 2010;105:83–90
11. Endsley MR, Jones WMMcNeese M, Salas E, Endsley MR A model of inter- and intrateam situation awareness: Implications for design, training and measurement, New Trends in Cooperative Activities: Understanding System Dynamics in Complex Environments. 2001 Santa Monica Human Factors and Ergonomics Society:pp 46–67
12. Xiao Y, Mackenzie CF, Patey RTeam coordination and breakdowns in a real-life stressful environment. . Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, In. October 5–9, 1998 Chicago:pp 186–90
13. Westli HK, Johnsen BH, Eid J, Rasten I, Brattebø G. Teamwork skills, shared mental models, and performance in simulated trauma teams: An independent group design. Scand J Trauma Resusc Emerg Med. 2010;18:47
14. Moiser K, Chidester TQueinnec Y, Danielou F Situation assessment and situation awareness in a team setting, Designing for Everyone. 1991 London Taylor & Francis:pp 798–800
15. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AASafe Surgery Saves Lives Study Group. . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–9
16. Hutchins E Cognition in the Wild. 1996 Cambridge, MA MIT Press
17. Ford S, Daniels J, Lim J, Koval V, Dumont G, Schwarz SK, Ansermino JM. A novel vibrotactile display to improve the performance of anesthesiologists in a simulated critical incident. Anesth Analg. 2008;106:1182–8
18. Hogan MP, Pace DE, Hapgood J, Boone DC. Use of human patient simulation and the situation awareness global assessment technique in practical trauma skills assessment. J Trauma. 2006;61:1047–52
19. Zhang Y, Drews FA, Westenskow DR, Foresti S, Agutter J, Bermudez JC, Blike G, Loeb R. Effects of integrated graphical displays on situation awareness in anaesthesiology. Cogn Technol Work. 2002;4:82–90
20. Endsley MREndsley MR, Garland DJDirect measurement of situation awareness:. Validity and use of SAGAT, Situation Awareness: Analysis and Measurement. 2000 Mahwah, NJ Lawrence Erlbaum:pp 147–74
21. Hänsel M, Winkelmann AM, Hardt F, Gijselaers W, Hacker W, Stiehl M, Koch T, Müller MP. Impact of simulator training and crew resource management training on final-year medical students’ performance in sepsis resuscitation: A randomized trial. Minerva Anestesiol. 2012;78:901–9
22. Wright MC, Taekman JM, Endsley MR. Objective measures of situation awareness in a simulated medical environment. Qual Saf Health Care. 2004;13(suppl 1):i65–71
23. Endsley MR. Measurement of situation awareness in dynamic systems. Hum Factors. 1995;37:65–84
24. Hogg DN, Torralba B, Volden F A situation awareness methodology for the evaluation of process control systems: Studies of feasibility and the implication of use. 1993 Storefjell, Norway OECD Halden Reactor Project
25. Sarter NB, Woods DD. Situation awareness: A critical but ill-defined phenomenon. Int J Aviat Psychol. 1991;1:45–57
26. Reader TW, Flin R, Mearns K, Cuthbertson BH. Team situation awareness and the anticipation of patient progress during ICU rounds. BMJ Qual Saf. 2011;20:1035–42
27. Salmon P, Stanton N, Walker G, Green D. Situation awareness measurement: A review of applicability for C4i environments. Appl Ergon. 2006;37:225–38
28. Salmon PM, Stanton NA, Walker GH, Jenkins D, Ladva D, Rafferty L, Young M. Measuring situation awareness in complex systems: Comparison of measures study. Int J Ind Ergon. 2009;39:490–500
29. Endsley MR, Sollenberger R, Stein E. Situation awareness: A comparison of measures. In Proceedings of the Human Performance, Situation Awareness and Automation: User Centered Design for the New Millennium Conference, Savanna. 2000
30. Taylor R. Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design, Situational Awareness in Aerospace Operations: La perception de la situation au cours des opérations aériennes. Neuilly sur Seine, AGARD. 1990:pp 3/1–3/17
31. Endsley MR, Selcon SJ, Hardiman TD, Croft D. A comparative evaluation of SAGAT and SART for evaluations of situation awareness. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting. October 5–9, 1988 Chicago:pp 82–6
32. Jones DG, Endsley MR. Using real time probes for measuring situation awareness. Int J Aviat Psychol. 2004;14:343–67
33. Jones DG, Endsley MR Can real-time probes provide a valid measure of situation awareness? In Proceedings of the Human Performance, Situation Awareness and Automation. 2000 Savannah User Centered Design for the New Millennium Conference
34. Selcon SJ, Taylor R, Koritsas E Workload or situational awareness?: TLX vs SART for aerospace systems design evaluation. In Proceedings of the Human Factors Society 35th Annual Meeting. 1991 San Francisco, September 2–6
35. Vidulich MAEndsley MR, Garland DJ. Testing the sensitivity of situation awareness metrics in interface evaluations, Situation Awareness Analysis and Measurement. 2000 Mahwah, NJ Lawrence Erlbaum:pp 227–46
36. Morgan PJ, Kurrek MM, Bertram S, LeBlanc V, Przybyszewski T. Nontechnical skills assessment after simulation-based continuing medical education. Simul Healthc. 2011;6:255–9
37. Graham J, Hocking G, Giles E. Anaesthesia non-technical skills: Can anaesthetists be trained to reliably use this behavioural marker system in 1 day? Br J Anaesth. 2010;104:440–5
38. Flin R, Patey R, Glavin R, Maran N. Anaesthetists’ non-technical skills. Br J Anaesth. 2010;105:38–44
39. Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL, Mazer CD, Kurrek MM, Joo HS. Simulation-based training improves physicians’ performance in patient care in high-stakes clinical setting of cardiac surgery. ANESTHESIOLOGY. 2010;112:985–92
40. Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL, Karatzoglou BJ, Hamstra SJ. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. ANESTHESIOLOGY. 2005;103:241–8
41. Zausig YA, Grube C, Boeker-Blum T, Busch CJ, Bayer Y, Sinner B, Zink W, Schaper N, Graf BM. Inefficacy of simulator-based training on anaesthesiologists’ non-technical skills. Acta Anaesthesiol Scand. 2009;53:611–9
42. Welke TM, LeBlanc VR, Savoldelli GL, Joo HS, Chandra DB, Crabtree NA, Naik VN. Personalized oral debriefing versus standardized multimedia instruction after patient crisis simulation. Anesth Analg. 2009;109:183–9
43. Savoldelli GL, Naik VN, Park J, Joo HS, Chow R, Hamstra SJ. Value of debriefing during simulated crisis management: Oral versus video-assisted oral feedback. ANESTHESIOLOGY. 2006;105:279–85
44. Boet S, Bould MD, Bruppacher HR, Desjardins F, Chandra DB, Naik VN. Looking in the mirror: Self-debriefing versus instructor debriefing for simulated crises. Crit Care Med. 2011;39:1377–81
45. Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”). Simul Healthc. 2009;4:6–16
46. Kreith F The CRC Handbook of Mechanical Engineering. 1998 Boca Raton CRC Press
47. Hazlehurst B, McMullen CK, Gorman PN. Distributed cognition in the heart room: How situation awareness arises from coordinated communications during cardiac surgery. J Biomed Inform. 2007;40:539–51
48. Boulet JR, Murray DJ. Simulation-based assessment in anesthesiology: Requirements for practical implementation. ANESTHESIOLOGY. 2010;112:1041–52
49. Murray DJ, Boulet JR, Avidan M, Kras JF, Henrichs B, Woodhouse J, Evers AS. Performance of residents and anesthesiologists in a simulation-based skill assessment. ANESTHESIOLOGY. 2007;107:705–13
50. Fehr JJ, Boulet JR, Waldrop WB, Snider R, Brockel M, Murray DJ. Simulation-based assessment of pediatric anesthesia skills. ANESTHESIOLOGY. 2011;115:1308–15
51. Devitt JH, Kurrek MM, Cohen MM, Cleave-Hogg D. The validity of performance assessments using simulation. ANESTHESIOLOGY. 2001;95:36–42
52. Boulet JR, Murray D, Kras J, Woodhouse J, McAllister J, Ziv A. Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. ANESTHESIOLOGY. 2003;99:1270–80
53. Byrne AJ, Greaves JD. Assessment instruments used during anaesthetic simulation: Review of published studies. Br J Anaesth. 2001;86:445–50
54. Schulz CM, Mayer V, Kreuzer M, Kochs EF, Schneider G. A tool for immediate and automated assessment of resuscitation skills for a full-scale simulator. BMC Res Notes. 2011;4:550
55. Byrne AJ, Jones JG. Responses to simulated anaesthetic emergencies by anaesthetists with different durations of clinical experience. Br J Anaesth. 1997;78:553–6
56. Görges M, Staggers N. Evaluations of physiological monitoring displays: A systematic review. J Clin Monit Comput. 2008;22:45–66
57. Charabati S, Bracco D, Mathieu PA, Hemmerling TM. Comparison of four different display designs of a novel anaesthetic monitoring system, the ‘integrated monitor of anaesthesia (IMA)’. Br J Anaesth. 2009;103:670–7
58. Liu D, Jenkins SA, Sanderson PM, Watson MO, Leane T, Kruys A, Russell WJ. Monitoring with head-mounted displays: Performance and safety in a full-scale simulator and part-task trainer. Anesth Analg. 2009;109:1135–46
59. Sanderson PM, Watson MO, Russell WJ, Jenkins S, Liu D, Green N, Llewelyn K, Cole P, Shek V, Krupenia SS. Advanced auditory displays and head-mounted displays: Advantages and disadvantages for monitoring by the distracted anesthesiologist. Anesth Analg. 2008;106:1787–97
60. Liu D, Jenkins SA, Sanderson PM, Fabian P, Russell WJ. Monitoring with head-mounted displays in general anesthesia: A clinical evaluation in the operating room. Anesth Analg. 2010;110:1032–8
61. Koivisto M, Hyönä J, Revonsuo A. The effects of eye movements, spatial attention, and stimulus features on inattentional blindness. Vision Res. 2004;44:3211–21
62. Pappas JM, Fishel SR, Moss JD, Hicks JM, Leech TDAn eye-tracking approach to inattentional blindness. . In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, Orlando, September 26–30, 2005:pp 1658–62
63. Endsley MR, Bolstad CA. Individual differences in pilot situation awareness. Int J Aviat Psychol. 1994;3:241–64
64. Endsley MREricsson KA, Charness P, Feltovich P, Hoffman R Expertise and situation awareness, The Cambridge Handbook of Expertise and Expert Performance. 20061st edition. New York Cambridge University Press:pp 633–51
65. Klein G Sources of error in naturalistic decision-making tasks. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting. October 11–15, 1993 Seattle:pp 368–71
66. Endsley MR, Jones DGEndsley MR, Jones SA oriented training, Designing for Situation Awareness: An Approach to User Centered Design. 20112nd edition DG. London, Taylor & Francis,:pp 235–56
67. Prince C, Salas EEndsley MR, Garland DJTeam situation awareness, errors, and crew resource management:. Research integration for training guidance, Situation Awareness: Analysis and Measurement. 2000 Mahwah, NJ Lawrence Erlbaum:pp 325–48
68. Robinson D. The development of flight crew situation awareness in commercial transport aircraft. In Proceedings of the Human Performance, Situation Awareness and Automation: User-Centered Design for a New Millennium Conference, Marietta, 2000:pp 88–93
69. Helmreich RL, Merrit AC, Sherman PJ. Research project evaluates the effect of national culture on flight crew behaviour. ICAO J. 1996;51:14–6
70. Reason J Managing the Risks of Organizational Accidents. 19971st edition. London Ashgate Press
71. Hormann H, Blokziji C, Polo L ESSAI: A European training solution for enhancing situation awareness and threat management on modern aircraft flight decks. In Proceedings of the 16th Annual European Aviation Safety Seminar of the Flight Safety Foundation and European Regions Airline Association. 2004 Barcelona,
72. McKenna F, Crick JL. Hazard Perception in Drivers: A Methodology for Testing and Training. Reading, Transport Research Laboratory. 1994
73. McKenna F, Crick JL Developments in Hazard Perception. 1997 London Transport Research Laboratory
74. Bolstad CA, Endsley MR, Costello AGeneral aviation pilot training for situation awareness: An evaluation. . In Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society, September 29–October 4, 2002 Baltimore:pp 21–5
75. Bolstad CA, Endsley MR, Howell C, Costello A Situation awareness training for general aviation pilots: Final report. 2002 SATECH, Marietta
76. Strater LD, Reynolds J, Faulkner LA, Birch K, Hyatt J, Swetman S, Endsley MRPC-based tools to improve infantry situation awareness. . In Proceedings of the Human Factors and Ergonomics Society Annual Meetin. September 20–24, 2004 New Orleans:668–72E
77. Kaber DB, Riley JM, Lampton D, Endsley MR Measuring situation awareness in a virtual urban environment for dismounted infantry training. In Proceedings of the 11th International Conference on HCI. 2005 Mahwah, NJ Lawrence Erlbaum
78. Kaber DB, Riley JM, Sheik-Nainar MA, Hyatt J, Reynolds J Assessing infantry soldier situation awareness in virtual environment-based training of urban terrain operations. In Proceedings of the International Ergonomics Association. 2006 Maastricht
79. Helmreich R, Klinect J, Wilhelm JModels of threat, error and CRM in flight operations. . In Proceedings of the 10th International Symposium on Aviation Psychology, Columbus, OH, May 3–6, 1999:pp 677–82
80. Kiefer N, Hoeft A. Display of information in the operating room. Curr Opin Anaesthesiol. 2010;23:772–7
81. Panel on Human Factors in the Design of Tactical Display Systems for the Individual Soldier: Tactical Display for Soldiers: Human Factors Considerations. 1997 Washington, DC National Academies Press
82. Endsley MR, Jones DGEndsley MR, Jones DG Alarms, diagnosis, and SA, Designing for Situation Awareness: An Approach to User Centered Design, 2nd edition. 2012 London CRC Press:pp 147–68
83. Leedal JM, Smith AF. Methodological approaches to anaesthetists’ workload in the operating theatre. Br J Anaesth. 2005;94:702–9
84. Schulz CM, Schneider E, Fritz L, Vockeroth J, Hapfelmeier A, Wasmaier M, Kochs EF, Schneider G. Eye tracking for assessment of workload: A pilot study in an anaesthesia simulator environment. Br J Anaesth. 2011;106:44–50
85. Schulz CM, Skrzypczak M, Schneider E, Hapfelmeier A, Martin J, Kochs EF, Schneider G. Assessment of subjective workload in an anaesthesia simulator environment: Reliability and validity. Eur J Anaesthesiol. 2011;28:502–5
86. Ford S, Birmingham E, King A, Lim J, Ansermino JM. At-a-glance monitoring: Covert observations of anesthesiologists in the operating room. Anesth Analg. 2010;111:653–8
© 2013 American Society of Anesthesiologists, Inc.
Publication of an advertisement in Anesthesiology Online does not constitute endorsement by the American Society of Anesthesiologists, Inc. or Lippincott Williams & Wilkins, Inc. of the product or service being advertised.