The use of alarming systems in patient monitoring devices, such as ventilator/anesthesia workstations, is of paramount importance for patient safety. This accounts for both perioperative anesthesia and monitoring in the intensive care unit (ICU). Inadequate use or failure to respond to intraoperative alarms may result in patient hazard and undesirable outcomes.1,2 The majority of alarms are so-called threshold alarms, i.e., a violation of a predefined threshold leads to an acoustic and/or optical alarm. Therefore, it is crucial to set alarming thresholds correctly. Thresholds have to be tight enough to detect potential deteriorations in vital functions as early as possible. However, tight thresholds are naturally prone to a high number of false-positive alarms. Therefore, conversely, they have to be set wide enough to account for physiologic inter- and intrapatient variations. In addition, artifacts (e.g., patient movement or manipulation of sensors) may lead to false-positive alarms. Depending on the surgical procedure, differing patterns and frequencies of alarms have been described.3 For the situation in the operating room (OR), the rate of false alarms was described to even exceed the number of correct alarms, so that the actual function of the alarms was lost and they became a distraction.3 Furthermore, studies in different adult and pediatric ICUs found false alarm rates ranging from 72% to 99%.3–6 The dangerous consequence is the “crying wolf phenomenon,” i.e., that because of the density of total alarms and the high number of false alarms, correct and important alarms are ignored.4 Thus, an important goal, in particular for clinical situations with high-risk procedures, is to reduce false alarm rates to a minimum by an optimized setting of alarm thresholds. However, there are only few data on quantity and quality of alarming in a complex perioperative setting.
Therefore, this study was performed to characterize the patterns of alarms of a current patient monitor (Kappa XLT; Dräger, Lübeck, Germany) and an anesthesia workstation (Zeus, Dräger) during elective cardiac surgery with the use of extracorporeal circulation (ECC). The objective was to quantify and to characterize all occurring alarms including identification of their origin during the entire perioperative phase. Furthermore, we sought to quantify the number of false-positive alarms produced by the monitoring system.
The protocol of this observational study was approved and authorized by the local ethics committee, the IRB, and the privacy protection commissioner of the hospital. After providing informed consent, 25 consecutive patients scheduled for elective cardiac surgery (aortocoronary bypass grafting and valve surgery) were included. Perioperative care was given by an anesthesiologist who was informed of being videotaped but blinded to the aims of the study. All caregiving anesthesiologists were staff anesthesiologists or residents in their last year.
All patients were premedicated with 7.5 to 15 mg midazolam per os 1 hour before arrival in the OR. According to the institutional standards, induction of anesthesia was performed with sufentanil 0.7 μg/kg and etomidate 0.15 mg/kg. Tracheal intubation was facilitated by pancuronium 0.1 mg/kg. Patients' lungs were mechanically ventilated in volume-controlled mode (autoflow) with tidal volumes of 8 mL/kg, and a ventilatory frequency of 8 to 12 breaths/min (Zeus, Dräger). Anesthesia was maintained by isoflurane 0.5% to 1.0% and by sufentanil up to a dose of 0.7 μg/kg/h.
All patients were monitored with the same combination of patient monitor (Kappa XLT, Dräger) and anesthesia workstation (Zeus, Dräger). Monitoring included electrocardiogram and pulse oximetry. Furthermore, arterial blood pressure was measured invasively in 1 radial artery. Central venous pressure was continuously obtained with a central venous catheter in the right internal jugular vein. Bladder temperature was measured continuously. In addition, all ventilation data, i.e., respiratory rate, airway pressures, tidal volume, and inspiratory and expiratory concentration of carbon dioxide and isoflurane, were monitored by the anesthesia workstation. In the perioperative phase, all patients were repeatedly assessed by transesophageal echocardiography.
After initiation of monitoring and induction of anesthesia, patients were transferred to the OR and connected to the anesthesia workstation and the patient monitor. Both devices were directly connected to a laptop computer for digital data storage in dedicated full-disclosure files using special software (MedLink, Nortis, Lübeck, Germany and eData TapeRec, Erasmus MC, Rotterdam, The Netherlands). The data collection started at the moment of arrival in the OR. The following data were recorded with a sampling interval of 1 second: all numerical measurements of the patient monitor and the anesthesia workstation, all real-time waveforms of pressures and flow readings, and all alarm events that were generated by both devices. In parallel, video recordings of the anesthesia workplace from 2 different views were performed during the study (Fig. 1). Therefore, all reactions of the attending physician to upcoming alarms could be registered and annotated. All data from the patient monitor, the anesthesia workstation, and the video cameras were time stamped, allowing an exact synchronization of all stored data.
Fixed alarm settings were used for all patients (Table 1). The caregiving anesthesiologist was instructed not to change this setting. During ECC, only the mean arterial blood pressure alarm and the body temperature alarm were active; all other alarms were deactivated. Immediately after ECC, all alarms were reactivated according to the fixed alarm settings. Only the thresholds for the alarm “heart rate” were changed to 70 bpm (lower limit) and 110 bpm (upper limit), because of postoperative pacemaker use.
The stored information of each alarm consisted of the alarm grade, the parameter causing the alarm, and the alarm message. The alarm grade indicated whether the alarm was an advisory alarm (low priority), a serious alarm (medium priority), or a life-threatening alarm (high priority). Advisory alarms indicated technical problems, such as a disconnection of a sensor. Serious alarms were caused by a violation of thresholds. Life-threatening alarms were triggered only by arrhythmias such as ventricular tachycardia, ventricular fibrillation, or asystole. Furthermore, a so-called “static alarm” for the invasive arterial blood pressure monitoring was registered. A static blood pressure alarm is generated if the height of the amplitude of the arterial pulse pressure measured invasively is <3 mm Hg. This is normally used to recognize a potential damping of the signal or a disconnection of the arterial catheter. The principle of using a minimum amplitude to exclude technical problems with a pulsating signal is not limited to only this monitor. It is also used frequently in pulse oximetry or capnometry to detect patient disconnection.
After completion of data collection, all data were annotated by the same anesthesiologist. Every single alarm was analyzed on the basis of the numerical measurements, the digitalized waveforms, and the respective video sequence. Based on this, all recorded alarms were categorized as follows: “technically true/technically false,” “clinically relevant/not clinically relevant,” and “medical reaction: yes/no.” Alarms were categorized as technically true if the measurement was technically correct (without artifacts) and showed a real threshold violation. Alarms were annotated as clinically relevant if there was a need for medical intervention. Patient-related alarms were caused by variables of the patients' vital sign monitoring (without “static alarms” [Fig. 2]). The annotating anesthesiologist recorded whether or not a medical reaction was performed after an alarm occurred. In uncertain situations, the particular video sequences were reanalyzed by a panel of 3 attending physicians.
Every onset of an alarm was counted and every alarm was counted only once even though the sound continued for a longer time. If an alarm was silenced and appeared again after the 2 minutes, it was counted again.
Data were analyzed using the software R (http://www.r-project.org/) and Excel 2003 (Microsoft Corp., Redmond, WA). Normal distribution of data was tested with the Kolmogorov-Smirnov test. Normally distributed variables are expressed as mean ± SD, otherwise as median (25th–75th percentile).
Twenty-five patients (14 men, 11 women) were studied. The mean age was 67 ± 11.4 years. Twelve patients underwent arterio-coronary bypass surgery, 11 valvular surgery, and 2 patients had a combination of both. The euroSCORE at time of admission to the hospital was 4.7 ± 3.7 The duration of surgery was 4.95 ± 0.96 hours.
One hundred twenty-four hours of intraoperative monitoring were recorded. In total, 8975 alarms were recorded. There were 7556 hemodynamic alarms and 1419 ventilatory alarms. This accounted for 359 ± 158 alarms per procedure, or 1.2 alarms per minute. The overall reaction time (time from occurrence to confirmation) to the alarms amounted to 4 ± 43.67 seconds. In this observation, we found 6386 serious and life-threatening alarms, which were further analyzed. The remaining 2589 alarms belonged to the category advisory technical alarms (n = 836) or were “static blood pressure alarms” (n = 1753) during ECC (see below). Ninety-six percent of the serious and life-threatening alarms were caused by threshold violations (Table 2). Of those alarms, 4438 (70%) were valid, whereas 1948 (30%) were caused by artifacts and were not valid or relevant. Of the valid alarms, 1735 (39%) were classified as relevant, and 2703 (61%) were not relevant.
In all patients, ECC was used. For analysis, all procedures were separated into 5 phases: “pre-ECC,” “going on ECC” (last 15 minutes before start of ECC), “during ECC,” “weaning from ECC” (first 15 minutes after cessation of ECC), and “post-ECC.” During “pre-ECC” we found 2985 alarms (33%) with a density of 1.4 alarms/min. During “going on ECC,” we observed 930 (10%), or 2.5 alarms/min. In the phase “during ECC,” 3114 alarms (35%) with a density of 0.9 alarms/min were registered. During “weaning from ECC” 626 alarms (7%) with a density of 1.7 alarms/min were found. Finally, during “post-ECC,” 1320 alarms (15%) with a density of 1.5 alarms/min occurred. In addition, we found different patterns of alarms in the different phases of the procedure. Therefore, the phases “going on ECC” and “weaning from ECC” were characterized by overly proportional numbers of heart rate, arrhythmia, blood pressure, central venous pressure, and end-tidal carbon dioxide alarms. The tidal volume alarms were found mainly in “weaning from ECC.” Patterns are illustrated in Figures 3 and 4. Furthermore, the time, which elapsed from onset of the hemodynamic alarms until reaction, i.e., silencing or disappearance of the alarms (reaction time), was analyzed for each phase: reaction time during “pre-ECC” was 29.3 seconds (9 [3–23] seconds); “going on ECC” 20.9 seconds (5 [2–13] seconds); “during ECC” 12.0 seconds (4 [2–8] seconds); “weaning from ECC” 33.7 seconds (10 [4–25] seconds); and during “post-ECC” 45.6 seconds (9 [3.25–30.75] seconds).
In this observation, we found 1648 static blood pressure alarms, accounting for 18% of all analyzed alarms. All were generated during ECC.
One thousand nine hundred forty-eight (30%) of the analyzed alarms were technically false. Of those 1948 alarms, 781 (40%) were caused by artifacts or manipulations of the patient monitoring system or its sensors (n = 727), the anesthesia workstation (n = 29), or the patient (n = 25). The particular reasons for the artifacts were often not annotatable (72%). In 28%, the reason for the artifacts could be identified as blood draw or flushing of the arterial line (n = 387), connection or disconnection of sensors or ventilation (n = 96), or electrocautery (n = 58).
This study showed that in a standard perioperative setting in cardiac surgery, the patient monitor and the anesthesia workstation generated alarms with an overall density of 1.2 alarms/min. Nearly 80% of these alarms had no therapeutic consequence. Specific patterns of alarming throughout the perioperative phase could be characterized, which might help to optimize alarm settings in the future.
Alarms are an essential part of safety in clinical anesthesia. However, alarms can only fulfill their function if they indicate a dangerous patient condition correctly. Unnecessary alarms may lead to an impairment of communication and to a distraction from other tasks.8 Several studies were performed investigating patterns of alarms in different situations in ICUs.4–6,8–12 However, the situation of intraoperative care differs significantly from the situation in the ICU. First, patients in the OR are predominantly anesthetized and their lungs mechanically ventilated. They are all under permanent and direct surveillance of an anesthesiologist. Second, deteriorations in the patients' conditions often happen faster in the OR because of the direct consequences of surgical manipulation. Third, many surgical procedures are frequently associated with specific maneuvers, such as ECC in cardiac surgery. This results in different patterns of alarms in comparison to the ICU setting. Thus far, this has not been investigated systematically in a high-complex intraoperative setting, including complete analysis of all raw data forming the basis of the alarms.
The present study also differs with regard to data collection; in earlier studies, which were performed in the ICU, only the numerical results of cardiorespiratory measurements were obtained automatically in varying time intervals.5,8,12 In a recent study, Görges et al.10 manually recorded the time-stamped alarm information as well as the presence of health care team members, using a pocket personal computer. In 2 studies, nurses recorded and classified all alarms manually during their normal working time,4,8 whereas in other studies, specially trained observers who were present at the bedside, but not in charge of patient care, documented and annotated alarm situations.5,8,10 However, both study designs offer potential disadvantages: If the data collection is done by the caregivers themselves, there is the chance that in times with a high workload not all alarms can be assessed as precisely as needed. Conversely, if data collection is performed by extra investigators, the presence of those observers may result in a different behavior by the caregiving anesthesiologist.
Therefore, we decided for this investigation to record and to extract all available measurements, alarm settings, and alarms of the patient monitor and the anesthesia workstation in a strictly automated manner with precise time stamps at a high sample rate of 1 per second, and to perform annotations of all alarms on the basis of a recorded video of the complete anesthesia workplace. Siebig et al.11 used a similar setup in their investigation in an ICU setting. Furthermore, this offered the opportunity to examine every clinical situation as often as needed. This was particularly helpful when alarms occurred in quick succession. In addition, the video and monitoring data could be compared and evaluated more carefully without the pressure of the ongoing procedure.
The only disadvantage we see by annotating without a bedside observer is the missing opportunity to ask the attending anesthesiologist to verify certain situations, but even this may have resulted in an additional bias.
In contrast to other studies, we used fixed alarm settings.3,4,6,10,11 This was necessary for better comparability of the patients. A possible disadvantage may be a higher alarm rate in cases when the fixed settings are not optimal for the individual patient, but the fixed setting we used seemed to be adequate for most of the study population. From our point of view, predefined fixed alarm settings are useful and important. However, fixed alarm settings must be tailored for the patient population (general, cardiac surgery, pediatric, etc.), but also potentially tailored for the individual patient, and must be made as user friendly as possible.
We assessed the alarms with respect to validity, relevance, and medical reaction. Although comparable classifications were used in the above-mentioned studies in the intensive care environment, definitions of alarm classifications were slightly different among the investigations. We defined alarms as technically true if the underlying measurement was technically correct and a real threshold violation was present. Alarms were relevant if there was a need for medical reaction. In contrast, in the studies by Koski et al.8 and Lawless,4 alarms were defined as being relevant only if the patient's condition was actively checked or a treatment was administered. An alarm may sometimes trigger a potentially useful mental note without any noticeable reaction. This undoubtedly influenced our results; nevertheless, the quantity of these reminders can only be estimated. In our observation, we found a reaction time (from occurrence to confirmation) of 4 ± 43.67 seconds. However, it is possible that the attending anesthesiologist did react without pressing the “silence button” or that the alarm was silenced without any further reaction. Last, it is imaginable that an alarm, as described above, leads to a mental note without pressing the “silence button.”
Alarms occurring in situations in which a permanent and not self-correcting technical problem had to be solved were judged either as “false” or “artifact” alarms.4,8 In contrast, Chambrin et al.12 also regarded technical alarms as true-positive alarms if they were followed by an action. In agreement with Chambrin et al., we considered alarms caused by persistent technical problems as necessary, because correct measurement of physiologic variables must be ensured for patient safety. In current alarm systems, many of these alarms are already denoted as advisory alarms to indicate the technical problem. This concept of differentiating technical and physiologic alarms could be further refined by an intelligent alarm algorithm developed on the basis of our annotated dataset.
The annotations used in this study distinguish between technically true and technically false alarms. A technically false alarm is given if measurements do not correctly identify the patient's condition or if an advisory alarm is given without a technical problem. Tsien and Fackler5 defined these technical alarms as false-positive alarms, but others subsumed technically as well as clinically irrelevant alarms as false-positive alarms.4,8,12 Nevertheless, it must be the aim for future improved alarm algorithms to avoid all technically false alarms, because all of them represent false-positive alarms with regard to their clinical consequence.
Alarms are often caused by manipulations performed by the medical staff. To further characterize these alarms, we noted whether staff members were directly working with the patient, the monitoring system, or the anesthesia workstation at the occurrence of the alarm, because this often changed the significance of the alarm. Some advisory alarms or even technically true alarms may have been annotated as “not relevant” in the presence of and with active manipulation by staff members, but as “relevant” in the absence of and without a manipulation of staff members. As in our study, Lawless4 classified those alarms that were not clinically important and caused by staff manipulation as “induced alarms.”
In this observation, we found 1648 static blood pressure alarms, accounting for 18% of all analyzed alarms. All were generated during ECC. During ECC, a continuous nonpulsatile flow without pulse pressure is generated; therefore, a static alarm of the arterial blood pressure measurement is generated continuously as long as any threshold alarm for the blood pressure signal is enabled. This finding and the fact that the different phases of the procedure are characterized by different patterns of alarms (i.e., ventricular fibrillation during “going on ECC”) clearly underlines the need for phase-adapted alarm settings, as for ECC.
False-negative alarms, i.e., the situation is alarm relevant and no alarm occurred, were not investigated. Although the whole perioperative period was recorded on video, those situations of false-negative alarms cannot be identified reliably by such a recording. It is retrospectively not possible to reliably identify each medical action and its relation to an alarm that did not occur. However, the clinical experience concerning false-negative alarms is that they occur very rarely and then mostly because the alarm thresholds are set too wide. Studies that investigated the frequency and reliability of ICU alarms support these observations. In the study by Tsien and Fackler,5 not a single false-negative alarm situation was recorded during 298 hours of monitoring. Chambrin et al.12 recorded 24 false-negative cases during 1971 hours of care, but in this study, a false-negative observation was identified even when only alarm threshold settings were modified without a prior audible alarm.
It is difficult to compare the results of our study with those of other investigators, because most of the other studies were performed in ICU settings.4–6,8,10,11 Only Seagull and Sanderson3 surveyed an intraoperative setting, but it included only 5 patients who underwent cardiac surgery. The anesthesia workspace was not filmed and only handwritten documentation of alarms was performed. However, this study examined a combination of a single patient monitor and an anesthesia workstation; the alarming patterns of different devices may be tendentiously similar to our results.
Based on the results of our study, several targets can be identified for the achievement of an alarm reduction. First, the high rates of technical alarms need to be reduced by technical improvements with regard to the raw signals (for example, reduction of oxygen saturation alarms by improvement of artifact detection). Second, the predominant alarm type is threshold alarm, and alarm reductions based on robust signal extraction may be the most universally applicable solutions. However, methods based on artificial intelligence such as machine learning, neural networks, fuzzy logic, and Bayesian networks may also have a role in the future, although the requirements in testing datasets, general applicability, and legal issues are highly challenging.
In conclusion, we found that during 25 consecutive cardiac surgical procedures, approximately 80% of the 8975 alarms had no therapeutic consequences. The majority of static alarms were caused by ECC. The “not-valid” alarms were mainly caused by manipulations (i.e., blood sampling, electrocautery). The implementation of phase-specific settings (e.g., an ECC setting), reminders for their proper use, and optimization in artifact or technical alarm detection could improve patient surveillance and safety.13
1. Beatty PC, Beatty SF. Anaesthetists' intentions to violate safety guidelines. Anaesthesia 2004;59:528–40
2. Tinker JH, Dull DL, Caplan RA, Ward RJ, Cheney FW. Role of monitoring devices in prevention of anesthetic mishaps: a closed claims analysis. Anesthesiology 1998;71:541–6
3. Seagull F, Sanderson P. Anesthesia alarms in context: an observational study. Hum Factors 2001;43:66–78
4. Lawless ST. Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med 1994;22:981–5
5. Tsien CL, Fackler JC. Poor prognosis for existing monitors in the intensive care unit. Crit Care Med 1997;25:614–9
6. O'Carroll TM. Survey of alarms in an intensive care unit. Anaesthesia 1986;41:742–4
7. Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9–13
8. Koski EM, Makivirta A, Sukuvaara T, Kari A. Frequency and reliability of alarms in the monitoring of cardiac postoperative patients. Int J Clin Monit Comput 1990;7:129–33
9. Edworthy J, Hellier E. Fewer but better alarms will improve patient safety. Qual Saf Health Care 2005;14:212–5
10. Görges M, Markewitz BA, Westenskow DR. Improving alarm performance in the medical intensive care using delays and clinical context. Anesth Analg 2009;108:1546–52
11. Siebig S, Kuhls S, Imhoff M, Langgartner J, Reng M, Schölmerich J, Gather U, Wrede CE. Collection of annotated data in a clinical validation study for alarm algorithms in intensive care: a methodologic framework. J Crit Care 2009;25:128–35
12. Chambrin MC, Ravaux P, Calvelo-Aros D, Jaborska A, Chopin C, Boniface B. Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis. Intensive Care Med 1999;25:1360–6
© 2011 International Anesthesia Research Society
13. Eden A, Pizov R, Toderis L, Kantor G, Perel A. The impact of an electronic reminder on the use of alarms after separation from cardiopulmonary bypass. Anesth Analg 2009;108:1203–8