Secondary Logo

Journal Logo

Implications of Resolved Hypoxemia on the Utility of Desaturation Alerts Sent from an Anesthesia Decision Support System to Supervising Anesthesiologists

Epstein, Richard H. MD, CPHIMS*; Dexter, Franklin MD, PhD

doi: 10.1213/ANE.0b013e31825c7f0c
Economics, Education, and Policy: Research Reports
Free
SDC

BACKGROUND: Hypoxemia (oxygen saturation <90%) lasting 2 or more minutes occurs in 6.8% of adult patients undergoing noncardiac anesthesia in operating room settings. Alarm management functionality can be added to decision support systems (DSS) to send text alerts about vital signs outside specified thresholds, using data in anesthesia information management systems. We considered enhancing our DSS to send hypoxemia alerts to the text pagers of supervising anesthesiologists. As part of a voluntary application for an investigative device exemption from our IRB to implement such functionality, we evaluated the maximum potential utility of such an alert system.

METHODS: Pulse oximetry values (SpO2) were extracted from our anesthesia information management systems for all cases performed in our main operating rooms and ambulatory surgical center between September 1, 2011, and February 4, 2012 (n = 16,870). Hypoxemic episodes (SpO2 < 90%) were characterized as either (a) lasting one or more minutes or (b) lasting 2 or more minutes. A single simulated “alert” was modeled as having been sent at the timestamp of the first (a) or the second (b) hypoxemic value. The hypoxemic episode was considered resolved at 1, 3, or 5 minutes after the time of the alert if the SpO2 value was no longer below the 90% threshold. Two-sided 99% conservative confidence limits were calculated for the percentage of unresolved alerts at the 3 evaluation intervals and compared with 70%, the lower limit of an acceptable true alarm rate for clinical utility.

RESULTS: There was at least 1 hypoxemic episode lasting 1 minute or longer in 23% of cases, and at least 1 episode lasting 2 minutes or longer in 8% of cases. Only 7% (99% confidence interval [CI] 6% to 8%) of the 1-minute hypoxemic episodes were unresolved after 3 minutes, and only 8% (99% CI 6%to 9%) of 2-minute episodes after 5 minutes (both P < 10−6 in comparison with 70% minimum reliability rate).

CONCLUSIONS: Low utility should be expected for a DSS sending hypoxemia alerts to supervising anesthesiologists, because nearly all hypoxemic episodes will have been resolved before arrival of the anesthesiologist in the operating room. These results suggest that the principal research focus should be on developing more sophisticated alerts and processes within rooms for the anesthesia care provider to initiate treatment promptly, to interpret or correct artifacts, and to make it easier to call for assistance via a rapid communication system.

Published ahead of print June 13, 2012 Supplemental Digital Content is available in the text.

From the *Department of Anesthesiology, Jefferson Medical College, Philadelphia, Pennsylvania; and Division of Management Consulting, Department of Anesthesia, University of Iowa, Iowa City, Iowa.

Funding: Departmental funding.

The authors declare no conflict of interest.

Reprints will not be available from the authors.

Address correspondence to Richard H. Epstein, MD, Jefferson Medical College, 111 S. 11th Street, Suite 6215F, Gibbon Building, Philadelphia, PA 19107. Address e-mail to richard.epstein@jefferson.edu.

Accepted April 16, 2012

Published ahead of print June 13, 2012

Hypoxemia, defined as the occurrence of at least 2 consecutive minutes of pulse oximeter saturation (SpO2) <90%, has been reported by Ehrenfeld et al. to occur in 6.8% of adults undergoing noncardiac anesthetics in operating room (OR) locations.1 Because hypoxemia can cause deleterious patient outcomes,26 anesthesia care providers universally strive to avoid this condition during all phases of perioperative care.

Groups have described the use of anesthesia information management systems (AIMS) to provide text alerts to anesthesiologists supervising cases whenever vital signs outside specified threshold limits are recorded in the AIMS central database.7 Providing such alerts through a decision support system (DSS) may enhance situational awareness (“the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”8) and improve patient care.7

Our hospital maintains an anesthesia DSS that queries the AIMS database in real time and sends administrative messages and reminders about incomplete electronic anesthesia record documentation to 1-way alphanumeric pagers carried by all clinical anesthesia staff. The DSS alerts providers through screen popups if there are gaps in the recording of arterial blood pressure;9 absent hourly blood glucose determinations during insulin infusions;10 or missing expected repeat doses of prophylactic antibiotics.11 The DSS also continually recalculates the estimated time remaining in cases to assist the OR directors in running the OR.12

We considered studying the impact of broadcasting vital sign alerts using our DSS infrastructure to supervising anesthesiologists. In this context, the DSS would be functioning as an alarm manager, with paging devices acting as alarm communicators.a Because our mobile medical softwareb would create an extension to the physiologic patient monitors, we elected to prepare a request for an investigative device exemption, based on Food and Drug Administration draft guidance.c As part of the device risk assessment, we evaluated the maximum potential utility of the system using historical pulse oximetry data from our AIMS. The results provide insight to others considering the addition of alert functions to their anesthesia DSS for notification of supervising anesthesiologists.

Back to Top | Article Outline

METHODS

The Thomas Jefferson University IRB approved this retrospective study without a requirement for informed consent. SpO2 and timestamps (to the nearest second), recorded in the hospital's AIMS at approximately 1-minute intervalsd (Innovian, Dräger, Telford, PA), were extracted from all cases performed at the tertiary surgical suites and ambulatory surgery center between September 1, 2011, and February 4, 2012. Induction, tracheal intubation, and extubation times were also retrieved, when applicable.

Each SpO2 <90% in the database was considered to represent a hypoxemic value.1 Hypoxemic episodes were categorized and analyzed separately as either (a) 1 or more consecutive minutes of hypoxemia or (b) 2 or more consecutive minutes of hypoxemia. (The second definition corresponds to that used by Ehrenfeld et al.1) Hypoxemic values were considered consecutive if separated by ≤75 seconds, to account for minor differences from the nominal 1-minute recording interval. If a hypoxemic episode resolved, but then recurred within 180 seconds of the most recent hypoxemic value, the subsequent episode was included as part of the previous hypoxemic episode (Fig. 1). This was a conservative approach, because it deliberately underestimated the number of alerts that would have been transmitted had such correlated episodes been counted separately, or had an alert been sent every minute for the duration of the hypoxemic episode. As part of the analysis of the candidate alert system, a single alert was considered as having been sent to the supervising anesthesiologist for each separate hypoxemic episode (i.e., 1 or more minutes, or 2 or more minutes), as soon as the hypoxemic episode would have been identified (i.e., at the timestamp of the first value or at the timestamp of the second of 2 consecutive hypoxemic values). This was a conservative approach from the perspective of resolution of the episode, in that it assumes that the alert would have been transmitted instantaneously to the supervising anesthesiologist.

Figure 1

Figure 1

A hypoxemic episode was considered to have been resolved at 1, 3, or 5 minutes if hypoxemia was not present at 1, 3, or 5 minutes, respectively, after the timestamp associated with identification of the episode.e A hypoxemic episode was characterized as having occurred during the interval from induction to tracheal intubation if it began between the induction time and 3 minutes after the intubation time. A hypoxemic episode was characterized as having occurred during tracheal extubation if it began later than 3 minutes before the time of extubation. These time intervals deliberately were narrower than the intervals described by Ehrenfeld et al.1 (i.e., induction to start of surgery and end of surgery to leave the OR), because we wished to be conservative and therefore focus on the critical portions of cases surrounding intubation and extubation.

Two-sided 99% Clopper–Pearson confidence limits for the percentage of hypoxemic episodes unresolved (i.e., still below SpO2 = 90%) at various times from the onset of the episode were calculated (Excel 2010, Microsoft, Redmond, WA). The use of 2-sided confidence limits was deliberately conservative because only the lower 1-sided confidence limit was important. The confidence limits were 99%, rather than 95%, again to be conservative. The observed percentages of unresolved hypoxemic episodes were compared with 70%, the lower limit of an acceptable true alarm rate for clinical utility (see DISCUSSION).13 First, analyses were performed including all hypoxemic episodes, and then second, limited to 1 randomly selected episode from each case. The latter analysis determined our sample size of 5 months of data, providing 1336 such cases. We needed 1303 such cases to have 90% power to detect a 5% difference from the 70% limit.

Back to Top | Article Outline

RESULTS

When hypoxemic episodes were characterized using a single value below 90%, 23% (3906/16,879) of the cases had at least 1 hypoxemic episode (Table 1). Only 7% of the hypoxemic episodes were not resolved within 3 minutes from onset (Table 1). This was substantially less than the suggested13 minimum reliability rate of 70% for automated alerts to be useful (P < 10−6). The same applied 1 minute and 5 minutes after the start of the hypoxemic episode (P < 10−6). Of the hypoxemic episodes, 22% (99% confidence interval [CI], 21% to 23%) occurred during the interval before intubation, and 7% (99% CI, 6% to 7%) during the interval after extubation.

Table 1

Table 1

When 2 minutes of hypoxia were required to define a hypoxemic episode, 8% of cases had at least 1 hypoxemic episode (Table 2). Only 8% of such episodes were not resolved within 5 minutes from onset (P < 10−6).

Table 2

Table 2

When the hypoxemic threshold was decreased to 85% or to 80%, the effect was to reduce markedly the incidences of cases with at least 1 hypoxemic episode (2.8% for 85%, and 1.4% for 80%). In contrast, the time for hypoxemic episodes to be resolved changed little. For the <85% threshold, hypoxemic episodes unresolved at 1, 3, and 5 minutes were 30% (CI, 25% to 34%), 4% (CI, 2% to 6%), and 1% (CI, 1% to 2%), respectively (all Ps < 10−6). For the <80% threshold, hypoxemic episodes unresolved at 1, 3, and 5 minutes were 29% (CI, 22% to 34%), 3% (CI, 1% to 5%), and 1% (CI, 0% to 3%), respectively each (P < 10−6).

Back to Top | Article Outline

DISCUSSION

We demonstrated in this study that the expected utility of a threshold algorithm that would provide hypoxemia alerts to supervising anesthesiologists is far less than the minimum 70% threshold suggested for DSS alert systems.13 Most hypoxemic episodes were resolved by the time that the anesthesiologist would have arrived in the OR from another location. The pulse oximeter provides a continual audible representation of the SpO2, and most people are able to detect a change in pitch corresponding to 1% decrease in saturation.14 Thus, we consider it likely that intervention by the provider often began well before the SpO2 decreased below 90%, providing an explanation for the limited duration of most episodes of hypoxemia.

By analogy, hypoxia monitors with wireless interfaces have been deployed that send alerts to nurses on the floor, who then evaluate the patient, start treatment, as indicated, and summon help, if needed.15 These monitors do not directly page the rapid response team, because the false alarm rate is high. Given that the maximum potential utility of a DSS transmitting hypoxia alerts directly to the supervising anesthesiologist is so low, and because of concern about possible unintended adverse consequences (see below), we abandoned the project. Shortly after our decision, an epidemiological study was published drawing the same conclusions for monitors directly notifying the rapid response team.16

Our presented analyses were conservative (i.e., biased toward showing utility of the alarm management system). For example, we did not adjust for the fact that nearly one third of such alerts would have occurred during intubation and extubation, periods during which a supervising anesthesiologist would be expected to be present in the OR, regardless of alerts.17 Other hypoxemic episodes likely also occurred during times when the anesthesiologist was already in the OR (e.g., during repeated episodes of apnea as part of rigid bronchoscopy).

Although some (1% to 8%) of the hypoxemic episodes were still continuing 5 minutes after the start of the episode (Tables 1 and 2), there are several arguments against providing such information via a remote alert system. First, trust in decision support alerts18 is eroded so much by unreliable alerts that there is no net positive value when the true positive rate (observed ≤8%) is <70%.13 In the immediate context of a vital sign alert, if the condition is no longer present by the time that the recipient responds to the alert, this is equivalent to a false alarm, because notification of the anesthesiologist did not affect the outcome (i.e., resolution of the episode). This occurs because of the inherent latency in the delivery of such alerts, incorporating the data recording process,19 transmission of the text message to the recipient's pager, responding to the alert,20 and traveling to the OR from which the message originated. Anesthesiologists turn off monitor alarms because of false alarms,21 and may functionally respond similarly to such alerts delivered to them remotely.

Second, heterogeneity in the anesthesia providers' trust in the DSS18 will complicate assessment of an alarm management system. A first-year anesthesia resident with high trust in the DSS may assume that a manual page for help to the supervising anesthesiologist is not necessary if he believes that notification will take place automatically and help will arrive quickly. An experienced certified registered nurse anesthetist with a low level of trust in the DSS may page for assistance regardless of the presence of the alert system, rendering the automated message superfluous.

Third, notification is not necessarily harmless, because the anesthesiologist receiving the alert will often be providing clinical care to another patient. For example, an anesthesiologist who receives an alert while discussing an anesthetic plan with a resident may forget to complete the discussion after returning from the interruption, resulting in a failure to follow the intended course of action. More seriously, the anesthesiologist caring for a patient with a difficult airway may be distracted by a spurious alert from another OR. Simultaneous critical events occur virtually every day (99%) at facilities with anesthesiologists supervising 3 or more ORs.17

It is striking that none of these 3 issues applies to a system that would provide sophisticated alerts to anesthesia providers within the OR or to facilitate their responses, including, when necessary, calling for assistance.

One limitation of our study is that we studied only a single institution. However, the incidence of at least 2 minutes of hypoxemia <90% (7.0% of cases f) and rate of resolution we observed was nearly identical to that previously reported by Ehrenfeld et al. (6.8%)1 Thus, our findings are likely to be generalizable.

Second, we could not determine reliably from documentation in the anesthesia record which of the hypoxemic values were artifacts (e.g., from administration of vital dyes known to interfere with oximetry readings, peripheral vasoconstriction, ambient or incident light contamination, patient shivering). Because each hypoxemic episode, regardless of cause, would have tripped an alert, this limitation does not alter the findings of our study.

Third, the incidence of mobile alerts could be reduced by increasing the duration of hypoxia (e.g., to 3 minutes) before an alert is triggered. However, the proportion of superfluous text alerts would be unchanged, because severe hypoxemia for several minutes would likely prompt a call for help before the alert being transmitted. In this setting, the anesthesia provider (or circulating nurse in the OR) is functioning as a mediating “information system” that processes data and requests assistance, when appropriate. Although our hospital's DSS delivers text messages quickly, with a 99.5th percentile <13 seconds (unpublished data), other DSS message delivery systems can have latencies more than 2 minutes (Brian Rothman, personal communication). Thus, increasing the duration of hypoxemia before the alert would result in a clinically unacceptably long delay at some facilities. Abenstein et al. described a suitable rapid communication system for these emergencies that provides visual alerts throughout the surgical facility.22 Other hospitals use overhead audible paging for intraoperative emergencies (Mohamed Rehman, personal communication).

Fourth, we studied only hypoxemia, not hypotension, tachycardia, etc. Appropriate evidence-based thresholds for these other physiologic variables are ill-defined, and there can be complications attendant to therapy (e.g., hypotension resulting from too aggressive treatment of hypertension).

In conclusion, low utility of a DSS sending threshold hypoxemia alerts to supervising anesthesiologists should be expected. Results suggest that the principal research focus should be on developing more sophisticated alerts and processes within rooms for the anesthesia care provider to initiate treatment promptly,23 to interpret and correct artifacts,24 and to make it easier to call for assistance via a rapid communication system.

Back to Top | Article Outline

RECUSE NOTE

Franklin Dexter is the Statistical Editor and Section Editor for Economics, Education, and Policy for Anesthesia & Analgesia. This manuscript was handled by Dr. Steven L. Shafer, Editor-in-Chief, and Dr. Dexter was not involved in any way with the editorial process or decision.

Back to Top | Article Outline

DISCLOSURES

Name: Richard H. Epstein, MD, CPHIMS.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Richard H. Epstein has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Name: Franklin Dexter, MD, PhD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Franklin Dexter has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors thank John D. Lee, PhD, Emerson Electric Quality & Productivity Professor, Department of Industrial & Systems Engineering, University of Wisconsin—Madison, for referring us to the Wickens and Dixon article (reference 13). We also thank Frank E. Block, Jr., MD, for editorial assistance and insights into vital signs signal processing.

a Pattillo, M. IHE patient care device technical framework supplement: alarm communication management trial implementation. Available at: http://www.ihe.net/Technical_Framework/upload/IHE_PCD_Suppl_Alarm_Communication_Management_ACM_TI_Rev1–2_2011-07-01.pdf. Accessed March 22, 2012.
Cited Here...

b Food and Drug Administration. Draft Guidance for Industry and Food and Drug Administration Staff: Mobile Medical Applications. Available at: www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM263366.pdf. Accessed February 5, 2012.
Cited Here...

c Code of Federal Regulations Title 21, volume 8, part 812—Investigational Device exemptions. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?FR=812.2. Accessed February 5, 2012.
Cited Here...

d Oximetry values were determined by IntelliVue MP50/70/90 series monitors (Royal Philips Electronics, The Netherlands) with M3001A multimeasurement modules, which incorporate Nellcor® Oximax® technology (Covidien, Mansfield, MA). The oximetry values are averaged over a minimum of 10 seconds for the monitor display. Every minute, the Innovian determines the median of up to 15 oximetry values received from the Philips monitor during the previous 60 seconds (Damian Hartman, Dräger, personal communication). Results may differ for other oximetry monitoring systems and AIMS (e.g., if different averaging and filtering algorithms are used).
Cited Here...

e For example, if the Spo2 was 87% at 09:03:02, the onset of the episode was noted at 09:03:02 If the next consecutive value in the database at 09:04:05 were 91%, the episode would be considered to have been resolved at 1 minute.
Cited Here...

f 7.0% = incidence of at least one 1-minute hypoxemic episode (23.2%) × probability that the episode was unresolved 1 minute later (30.3%).
Cited Here...

Back to Top | Article Outline

REFERENCES

1. Ehrenfeld JM, Funk LM, Van Schalkwyk J, Merry AF, Sandberg WS, Gawande A. The incidence of hypoxemia during surgery: evidence from two Institutions. Can J Anaesth 2010; 57:888–97
2. Kazan R, Bracco D, Hemmerling TM. Reduced cerebral oxygen saturation measured by absolute cerebral oximetry during thoracic surgery correlates with postoperative complications. Br J Anaesth 2009;103:811–6
3. Gozal D, Daniel JM, Dohanich GP. Behavioral and anatomical correlates of chronic episodic hypoxia during sleep in the rat. J Neurosci 2001;21:2442–50
4. Bass JL, Corwin M, Gozal D, Moore C, Nishida H, Parker S, Schonwald A, Wilker RE, Stehle S, Kinane TB. The effect of chronic or intermittent hypoxia on cognition in childhood: a review of the evidence. Pediatrics 2004;114:805–16
5. Casati A, Fanelli G, Pietropaoli P, Proietti R, Tufano R, Danelli G, Fierro G, De Cosmo G, Servillo G. Continuous monitoring of cerebral oxygen saturation in elderly patients undergoing major abdominal surgery minimizes brain exposure to potential hypoxia. Anesth Analg 2005;101:740–7
6. Monk TG, Weldon BC, Garvan CW, Dede DE, van der Aa MT, Heilman KM, Gravenstein JS. Predictors of cognitive dysfunction after major noncardiac surgery. Anesthesiology 2008; 108:18–30
7. Rothman B, Sandberg WS, St Jacques P. Using information technology to improve quality in the OR. Anesthesiology Clin 2011;29:29–55
8. Endsley MR. Toward a theory of situation awareness in dynamic systems. Hum Factors 1995;37:32–64
9. Ehrenfeld JM, Epstein RH, Bader S, Kheterpal S, Sandberg WS. Automatic notifications mediated by anesthesia information management systems reduce the frequency of prolonged gaps in blood pressure documentation. Anesth Analg 2011;113: 356–63
10. Garg R, Jarry A, Pendergrass M. Delay in blood glucose monitoring during insulin infusion protocol is associated with increased risk of hypoglycemia in intensive care units. J Hosp Med 2009;4:E5–7
11. St. Jacques P, Sanders N, Patel N, Deshpande JK, Higgins M. Improving timely surgical antibiotic prophylaxis redosing administration using computerized record prompts. Surg Infect 2005;6:215–21
12. Dexter F, Epstein RH, Lee JD, Ledolter J. Automatic updating of times remaining in surgical cases using Bayesian analysis of historical case duration data and instant messaging updates from anesthesia providers. Anesth Analg 2009;108:929–40
13. Wickens WD, Dixon S. The benefits of imperfect diagnostic automation: a synthesis of the literature. Theoretical Issues Erg Sci 2007;8:201–12
14. Schulte GT, Block FE Jr. Can people hear the pitch change on a variable-pitch pulse oximeter? J Clin Monit 1992;8:198–200
15. Taenzer AH, Pyke JB, McGrath SP, Blike GT. Impact of pulse oximetry surveillance on rescue events and intensive care unit transfers: a before-and-after concurrence study. Anesthesiology 2010;112:282–7
16. Fagan K, Sabel A, Mehler PS, MacKenzie TD. Vital sign abnormalities, rapid response, and adverse outcomes in hospitalized patients. Amer J Med Qual 2012;[epub ahead of print]
17. Epstein RH, Dexter F. Influence of supervision ratios by anesthesiologists on first-case starts and critical portions of Anesthetics. Anesthesiology 2012;116:683–91
18. Madhavan P, Wiegmann DA. Similarities and differences between human–human and human–automation trust: an integrative review. Theoretical Issues Erg Sci 2007;8:277–301
19. Epstein RH, Dexter F, Ehrenfeld JM, Sandberg WS. Implications of event entry latency on anesthesia information management decision support systems. Anesth Analg 2009;108:941–7
20. Ledolter J, Dexter F, Epstein RH. Analysis of variance of communication latencies in anesthesia: comparing means of multiple lognormal distributions. Anesth Analg 2011;113: 888–96
21. Block FE Jr, Nuutinen L, Ballast B. Optimization of alarms: a study on alarm limits, alarm sounds, and false alarms, intended to reduce annoyance. J Clin Monit 1999;15:75–83
22. Abenstein JP, Allan JA, Ferguson JA, Deick SD, Rose SH, Narr BJ. Computer-based anesthesiology paging system. Anesth Analg 2003;97:196–204
23. Albert RW, Agutter JA, Syroid ND, Johnson KB, Loeb RG, Westenskow DR. A simulation-based evaluation of a graphic cardiovascular display. Anesth Analg 2007;105:1303–11
24. Takla G, Petre JH, Doyle DJ, Horibe M, Gopakumaran B. The problem of artifacts in patient monitor data during surgery: a clinical and methodological review. Anesth Analg 2006;103: 1196–204
© 2012 International Anesthesia Research Society