Secondary Logo

Journal Logo


Low Inadequate Oxygen Delivery Index is Associated with Decreased Cardiac Arrest Risk in High-Risk Pediatric ICU Patients

Dewan, Maya MD, MPH1–4; Cooper, David S. MD, MPH1,5; Tegtmeyer, Ken MD1,2

Author Information
doi: 10.1097/CCE.0000000000000600


Inhospital pediatric cardiac arrest has a high morbidity and mortality (1), making cardiac arrest prevention vital. Current systems to identify high-risk patients do not reliably identify patients at risk for arrest within the PICU or, when highly reliable, require a high number needed to evaluate (2–6). The number needed to evaluate is the number of patients needing to evaluate before a “true positive” is detected. We hypothesize that the use of time-series physiologic data could be used to “exclude” patients labeled as high-risk by clinicians who will not experience cardiac arrest and reduce the number needed to evaluate.

Bedside time-series physiologic data can be used for predictive modeling to assist with risk and trajectory. One such algorithm is the FDA-cleared T3 Visualization Platform Inadequate Oxygen Delivery Index (IDO2; Etiometry, Boston, MA). The IDO2 is designed to reflect the likelihood of a patient having a mixed venous oxygen saturation (SvO2) below a particular threshold. The threshold is 30%, 40%, or 50% and can be selected by the user based on clinical preference (7). The IDO2 varies between 0 and 100. An increasing IDO2 means there is an increasing risk of inadequate oxygen delivery, and attention should be brought to the patient. The minimum data required for calculation of the IDO2 is heart rate every 60 s and oxygen saturation via pulse oximetry and blood pressure every 10 minutes (6). If available, the calculation will also include arterial oxygen saturation, SvO2, hemoglobin, and atrial filling pressures. This algorithm is validated, and FDA cleared in postcardiac surgical ICU patients up to age 12. Our aim was to evaluate IDO2 as a predictor of no cardiac arrest in PICU patients less than 12 years old labeled as high-risk by clinical concern.


Design, Setting, and Patients

This is a prospective cohort study of all patients less than 12 years old labeled as high risk due to clinician concern in a single 35-bed PICU at an academic, freestanding, quaternary-care hospital from February 1, 2017, to May 20, 2020. The IDO2 algorithm was set to reflect the likelihood of SvO2 less than 50%, and necessary data were collected prospectively. IDO2 was calculated retrospectively and, therefore, not visible to clinicians in real time. Mean calculated IDO2 was collected for the 12 hours prior to the patient being labeled as high risk or a “watcher,” defined as the prewatcher period (Fig.1). Criteria for inclusion as a high-risk patient were clinician concern raised during bid safety huddles with patient labeled as a “watcher” as part of a baseline situation awareness system. Exclusion criteria included age greater than 12 years old and/or patients who received extracorporeal membrane oxygenation. This study was determined to be exempt by the Institutional Review Board at Cincinnati Children’s Hospital Medical Center (2018-0337).

Figure 1.:
The prewatcher period was defined as the 12 hr prior to watcher identification over which mean Inadequate Oxygen Delivery Index (IDO2) was calculated. The watcher period was the 12 hr following watcher identification in which we monitored for clinical deterioration.

A clinical deterioration event was defined as a cardiac arrest or code-bell activation with response by the unitwide code team requiring intervention. A cardiac arrest was defined as an acute event requiring chest compressions as a result of loss of spontaneous circulation or bradycardia. A code bell activation is an audible alarm with response by the unit wide code team that is categorized as airway emergency, life-threatening arrhythmia, emergent intubation, hypotensive crisis, neurologic decompensation, and other. Notification of clinical deterioration events occurred via a preexisting automated notification system that sends an e-mail and text with all code bell activations and cardiac arrest events occurring within the PICU. Automated notifications were reviewed daily by research staff and entered into the quality improvement clinical deterioration event database. Additionally, all patient code sheets were reviewed to ensure accurate event capture.

IDO2 Cohort Identification

Mean IDO2 for the prewatcher period was calculated retrospectively. To determine the IDO2 threshold that most effectively ruled out watcher patients while maintaining accurate prediction, prewatcher periods with a mean IDO2 less than a particular threshold were used to exclude watcher patients. Prewatcher periods with no preceding IDO2 were not removed to not overestimate the performance of the model. Threshold values ranging from 0 (no watcher periods removed) to 16 were tested. An event was considered predicted if it occurred within 12 hours of a patient being labeled a watcher. An individual patient could be labeled as a watcher for multiple 12-hour watcher periods. No adjustments were made for recurrent events within the same patient to mimic the applicability to the clinical environment where each patient is screened bid independent of prior events.

Data Analysis

We evaluated the likelihood ratios, positive and negative predictive values, and the number needed to evaluate of clinical concern alone and clinical concern plus low IDO2 to identify patients who experienced a clinical deterioration or cardiopulmonary resuscitation (CPR) event during the watcher period. The number needed to evaluate is analogous to the number needed to treat. This number signifies the number of 12-hour watcher periods during which no deterioration occurred for each 12-hour watcher period that experienced a clinical deterioration event or cardiac arrest event.


During the study period, 3,087 patients were monitored over 24,505 12-hour time periods. During this time, there were 224 clinical deterioration events of which 48 were cardiac arrest events. Non-CPR events included arrhythmia (10%, n = 23), unexpected airway emergencies (60%, n = 135), hypotensive crisis 1%, n = 3), urgent intubation (1%, n = 3), neurologic decompensation (3%, n = 7), and other (2%, n = 5).

In clinical watcher patients, 23 CPR events (48%) and 93 clinical deterioration events (42%) occurred during 4,017 watcher periods. In nonwatcher patients, 25 CPR events and 131 clinical deterioration events occurred during 20,488 watcher periods (Fig. 2). To determine the IDO2 index threshold, threshold values ranging from 0 (no watcher periods removed) to 16 were tested (Fig. 3). A mean IDO2 less than 5 during the prewatcher period was selected to remove watchers as it improved the positive predictive value for cardiac arrest events from 0.6% to 0.8% and clinical deterioration events from 2.3% to 2.6% reducing the number needed to evaluate from 167 to 125 and 43 to 38, respectively, with no change in the prediction of cardiac arrest (Supplemental Table,

Figure 2.:
Cohort identification. CPR = cardiopulmonary resuscitation.
Figure 3.:
Cardiopulmonary resuscitation (CPR) events that occur during watcher period demonstrating that you can decrease watchers by 26% without worsening prediction. Each point on the figure represents a 12-hr Inadequate Oxygen Delivery Index (IDO2) mean threshold labeled with the mean IDO2 ranging from 1 to 16. The chosen IDO2 mean of 5 is marked with a diamond.


In this prospective observational cohort study utilizing Etiometry IDO2, we demonstrated that the use of an IDO2 less than 5 during the 12-hour prewatcher period can safely rule out patients labeled as high risk by clinician concern who do not experience cardiac arrest improving the number needed to alert by 25% without changing the sensitivity of cardiac arrest prediction. Although the test characteristics remain far from ideal (8), the use of a streaming analytics platform to deidentify patients is a unique use case and deserves further evaluation. In addition, the positive predictive value reported here is in line with other pediatric early warning score systems (9), making this application of deimplementation a possible strategy for other similar systems.

Improvement in test characteristics through the removal of alerts is a type of partial reversal deimplementation. Deimplementation is the removal of unnecessary or harmful practices that may result in low-value care. Deimplementation must be thought of as an implicit component of implementation and improvement science (10). Partial reversal deimplementation changes the frequency, breadth, or scale of an intervention to provide the intervention to a smaller subgroup of patients. In systems with high number needed to alert, the application of evidence-based clinical criteria may drive improvements, but a focused deimplementation strategy to streamline workflows and improve prediction may also be necessary. Rather than focusing solely on the use of streaming analytics platforms, like T3 to drive identification, it is also vital to think of them as deidentification strategies. We propose that health systems using streaming analytics platforms not only use these scores to rule patients in but also study thresholds to allow for deidentification of high-risk patients faster and more accurately to drive system improvement (Fig. 4).

Figure 4.:
Proposed new workflow of addition of low Inadequate Oxygen Delivery Index (IDO2) to clinician concern to minimize false positives. Prior workflow demonstrated with dotted line (- - -) and new workflow with solid line.

This study has multiple limitations. First, the IDO2 was not clinically in use at the time of this study, and therefore, some patients were missing values to allow for a calculation of IDO2. As the mean IDO2 less than 5 was only used to rule out patients, it is likely that missing data only underestimate our test characteristics and increase the number needed to alert. Second, the baseline situation awareness system of clinician concern only had a very large number needed to alert. As other situation awareness system that use only clinician concern have not reported their number needed to alert, only outcomes (11,12), it is hard to know how the baseline system performs compared with others prior to the initiation of IDO2. However, given the rarity of pediatric cardiac arrest, it is highly likely that similar poor test characteristics exist in other nonautomated systems as has been seen in other pediatric early warning score prediction for deterioration (9). Patients may have not been identified as watchers due to a variety of factors including admission or deterioration between bid huddles, poor communication of team during huddle to list accurate patient risk status, or because the deterioration was unexpected and, therefore, not predicted.


The use of a streaming analytics platform employing physiologic monitor data can be applied to clinician-activated situation awareness systems to decrease the number needed to alert and improve system efficiency. The use of predictive analytic tools should be evaluated by centers to both assist in implementation and deimplementation of quality improvement processes.


We thank Etiometry for their partnership in providing Inadequate Oxygen Delivery Index data for this article.


1. Girotra S, Spertus JA, Li Y, et al.; American Heart Association Get With the Guidelines–Resuscitation Investigators: Survival trends in pediatric in-hospital cardiac arrests: An analysis from Get With the Guidelines-Resuscitation. Circ Cardiovasc Qual Outcomes 2013; 6:42–49
2. Rothman MJ, Tepas JJ 3rd, Nowalk AJ, et al.: Development and validation of a continuously age-adjusted measure of patient condition for hospitalized children using the electronic medical record. J Biomed Inform 2017; 66:180–193
3. Baronov D, McManus M, Butler E, et al.: Next generation patient monitor powered by in-silico physiology. Annu Int Conf IEEE Eng Med Biol Soc 2015; 2015:4447–4453
4. Pollack MM, Holubkov R, Berg RA, et al.; Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN): Predicting cardiac arrests in pediatric intensive care units. Resuscitation 2018; 133:25–32
5. Dewan M, Muthu N, Shelov E, et al.: Performance of a clinical decision support tool to identify PICU patients at high risk for clinical deterioration. Pediatr Crit Care Med 2020; 21:129–135
6. Dewan M, Soberano B, Sosa T, et al.: Assessment of a situation awareness quality improvement intervention to reduce cardiac arrests in the PICU. Pediatr Crit Care Med 2021 Aug 20. [online ahead of print]
7. Etiometry: User manual for T3 platform software (T3 data aggregation & visualization software module version 5.0 and T3 risk analytics engine software module version 6.0). 2020. Accessed May 1, 2021
8. Dewan M, Sanchez-Pinto LN: Crystal balls and magic eight balls: The art of developing and implementing automated algorithms in acute care pediatrics. Pediatr Crit Care Med 2019; 20:1197–1199
9. Trubey R, Huang C, Lugg-Widger FV, et al.: Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: A systematic review. BMJ Open 2019; 9:e022105
10. Wang V, Maciejewski ML, Helfrich CD, et al.: Working smarter not harder: Coupling implementation to de-implementation. Healthc (Amst) 2018; 6:104–107
11. Brady PW, Muething S, Kotagal U, et al.: Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics 2013; 131:e298–e308
12. Brady PW, Wheeler DS, Muething SE, et al.: Situation awareness: A new model for predicting and preventing patient deterioration. Hosp Pediatr 2014; 4:143–146

cardiac arrest; deimplementation; pediatrics

Supplemental Digital Content

Copyright © 2022 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine.