Journal Logo

Feature Articles

Late Rescue Collaborative: Reducing Non-ICU Arrests*

Dean, Nathan P. MD1; Ghebremariam, Emanuel MPH2; Szeles, Rosemary RN3; Levin, Amanda MD4; Colyer, Jessica MD5; Steinhorn, Robin H. MD6

Author Information
Pediatric Critical Care Medicine: June 2020 - Volume 21 - Issue 6 - p 513-519
doi: 10.1097/PCC.0000000000002224
  • Free
  • Infographics


Early recognition of patient deterioration in the acute care setting is critical for escalation and intervention. Unrecognized deteriorations or inadequate rescue plans may lead to non-ICU arrests, and non-ICU arrests are associated with greater morbidity and mortality (1). To address this problem, some hospitals use Pediatric Early Warning Scores (PEWS) to identify deteriorating patients and most hospitals use rapid response teams (RRTs) to rescue patients in the early stages of deterioration.

Current literature evaluating RRTs and PEWS is limited by the different outcomes used to demonstrate efficacy. Tucker et al (2), Agulnik et al (3), and Skaletzky et al (4) all demonstrated that an elevated PEWS correlated with an increase need for ICU transfer. Akre et al (5) retrospectively showed a link between elevated PEWS and RRT or code event. Dean et al (6) found a link between elevated PEWS and unplanned transfers and need for ICU-specific therapies. Parshuram et al (7) showed no difference in mortality between hospitals using bedside PEWS versus those without. They did find a reduction in a composite event composed of death before ICU admission, provision of cardiopulmonary resuscitation (CPR), tracheal intubation, administration of vasoactive, or initiation of extracorporeal membrane oxygenation. RRT literature typically evaluates the effect of implementation of an RRT on pre- and postimplementation mortality (8–10).

A second limitation of the current literature is the lack of long-term follow-up of the impact of recognition and resuscitation programs on rates of deterioration. In 2007, the Children’s Hospital Association (CHA) engaged 20 hospitals in a 12-month improvement project to eliminate non-ICU arrests and their associated mortality on inpatient units through various interventions. However, at the end of the 12-month intervention period, only a modest 3% reduction in non-ICU arrests was reported (11).

The Late Rescue Collaborative is a single-center, multidisciplinary intramural collaborative formed in 2014 to reduce critical deterioration events on acute care units that were occurring despite implementation of RRT and PEWS. We believed that creation of the Late Rescue Collaborative would reduce Non-ICU arrests. This reduction would occur by close tracking of deteriorations, open discussions of deterioration events, monitoring compliance with hospital protocols, identification of new unit and hospital-based protocols, and sharing lessons learned between representatives from different inpatient units that are eventually brought back to frontline providers. Non-ICU arrest was chosen as an outcome rather than unplanned ICU transfer or mortality, as unplanned transfers might be the best rescue for a deteriorating patient, and mortality of an acute care inpatient is a rare event. Our aim is to demonstrate a sustained reduction in non-ICU arrest through implementation of the Late Rescue Collaborative.



The Late Rescue Collaborative is a single-center intramural collaborative introduced in January 2014. It emerged through a cooperative effort between the hospital performance improvement department and the hospital code blue committee. RRTs and PEWS existed prior to the Late Rescue Collaborative formation. RRT implementation occurred in 2007 and PEWS in 2008 (12); however, there lacked systematic monitoring of compliance of either system. Review of multiple deterioration cases revealed a rarely activated RRT system where acute care staff instead relied on informal, unprotocolized PICU consults. Additionally, physicians were unaware of PEWS and it was rarely performed accurately (13). In 2011, our hospital code blue committee made attempts to strengthen the RRT/PEWS protocols and promote a culture of enhanced escalation. Rates of RRT activations rose but critical deterioration events still occurred. The escalation protocol was modified and individual divisions were educated at local sessions and during a three part grand rounds series sponsored by Aligning Education 4 Quality, a program created by the Association of American Medical Colleges to promote linkage of continuing medical education and quality improvement (14). After completion of the grand rounds series, planning meetings to create the Late Rescue Collaborative began.


The Late Rescue Collaborative models itself after the CHA’s multihospital collaborative: Eliminating Codes and Associated Mortality on Inpatient Units (11). It is led by a critical care physician, an acute care inpatient nursing director, and a performance improvement specialist. The collaborative represents a diversified body of physicians and nurses from each acute care inpatient unit, including all subspecialties, and inclusive of fellow and resident trainees. Monthly meetings occur to review data on unplanned transfers to the ICU, review specific cases of deterioration, and share lessons learned. Initially the collaborative monthly meetings consisted of a senior physician, a senior nursing lead, and nursing representatives from each unit. Physicians were invited to join, but participation was limited. Physician involvement increased after approval for American Board of Pediatric (ABP) Maintenance of Certification (MOC) part 4 credit, and 80 physicians have participated and received MOC credit during the study period (table 1). Many physicians claiming MOC credit continue with the collaborative even after meeting their ABP requirements.

Pediatric Subspecialties of Participants Claiming Part 4 Maintenance of Certification

The Late Rescue Collaborative implemented 16 interventions and improvement cycles to promote early recognition and rescue of deteriorating patients (Table 2). Four interventions led to the largest reductions in non-ICU arrests, including 1) “watcher” huddles and notes written by pediatric residents (intervention A); 2) automation of PEWS calculation by the bedside nurse (intervention B); 3) creation of unit-specific subgroups (intervention E); and 4) a quality improvement project to improve attending notification prior to RRTs (intervention F). This combination of improvements translated to improved processes and streamlined communication.

Interventions to Reduce Non-ICU Arrests

For intervention A, a nursing concern or PEWS equals to 4 or 3 in one category triggers a watcher huddle and requires the acute care resident/nurse practitioner/physician assistant and bedside nurse to meet at the bedside to evaluate the patient. Afterwards a structured plan including 1) specific actions to be taken; 2) expected outcome; 3) outcome deadline; and 4) escalation plan if outcome is not met is documented by the acute care provider. During July of 2015, modification of watcher note templates in the electronic medical record changed completion of these four plan fields from optional to required.

Intervention B occurred September of 2015. It involved automation of PEWS calculation in the electronic health record. Prior to this change, bedside nurses mentally calculated a PEWS score and directly entered it into the electronic health record. An audit of patient charts demonstrated that 29% of scores were inaccurately calculated as too low, and several deteriorations had missed mandatory RRT activations due to PEWS incorrectly scored too low. After the change, the bedside nurse enters each PEWS component into the electronic health record and the total score is automatically calculated. Charts sampled after the change showed 100% accuracy.

In March of 2016, intervention E began. The Late Rescue Collaborative created individual working groups, pairing unit-based physicians and nursing leaders. The leaders encouraged development of local quality improvement initiatives and delivered ongoing support. Lessons learned at unit-based meetings were then shared at the monthly whole group meetings.

Intervention F involved a project to improve attending notification prior to RRT activation. After conclusion of each RRT, the acute care team was asked whether or not the attending physician had been called prior to the RRT activation. The goal was to highlight this expectation and enhance a culture of earlier notification of the attending, so that interventions informed by an experienced attending might occur earlier.

Study of the Intervention

The aim of the interventions was to reduce non-ICU arrests. Non-ICU arrests are defined as 1) CPR; 2) defibrillation/cardioversion; or 3) acute respiratory compromise occurring on the acute care unit. Acute respiratory compromise is further defined as 1) intubation on the acute care floor or 2) application of positive-pressure ventilation via bag-mask or bag tracheostomy that is continued on transport to the ICU culminating in intubation or tracheostomy patients placed on mechanical ventilation. The collaborative reviewed all unplanned transfers/code blue activations for features consistent with non-ICU arrests. The Late Rescue steering committee discussed and classified all potential non-ICU arrests. Due to the rarity of these events, a second form of deterioration was tracked and termed “late rescue.” A late rescue was defined as an unplanned transfer requiring intubation or inotropes within 12 hours of admission to the ICU. Balancing measures included unplanned ICU transfers (including transfers to the PICU, neonatal ICU, and cardiac ICU) and RRT activations.


Control charts for unplanned ICU transfers, RRT activations, and non-ICU arrests describe events over the 4-year study period. Data were reported monthly and standardized per 1,000 non-ICU days. A run of eight or more consecutive points above or below the central line of the baseline period meets criteria for special cause variation and supports a statistically significant process shift.

Ethical Considerations

The Institutional Review Board (IRB) reviewed the project. The IRB acknowledged the project and determined this project qualified as a Quality Improvement initiative at Children’s National, and it does not constitute human subjects research. As such it was not under the oversight of the IRB and no consent to participate was required.


During the study period of October 2014 through December 2018, there were 218,185 non-ICU patient days recorded. Forty-one non-ICU arrests occurred, with four resulting in mortality. Non-ICU arrest rates fell from a baseline of 0.31 per 1,000 non-ICU patient days to a new centerline of 0.11 on December 2015 and sustained for 36 months (Fig. 1). Days between arrests increased from a baseline of 15.5 days to 61.5 days on November 2015 and sustained for 37 months (Fig. 2). Absolute mortality following non-ICU arrest dropped from one in 2014 and three in 2015 to zero for 2016, 2017, and 2018.

Figure 1.
Figure 1.:
Non-ICU arrest rates and timing of interventions u-chart. Letters represent interventions detailed in Table 2. CL = centerline, UCL = upper confidence limit.
Figure 2.
Figure 2.:
Days between non-ICU arrests t-chart. CL = centerline.

We monitored unplanned transfers and RRT activation rates as possible balancing measures resulting from early detection and rescue of deterioration. Although non-ICU arrests decreased during this study period, RRT and unplanned transfer rates remained stable (Figs. 3 and 4). RRT activation rates appeared to have a seasonal variation with peaks during the winter months.


Implementation of the Late Rescue Collaborative at our stand-alone pediatric hospital resulted in a significant reduction of non-ICU arrests that has been maintained for 3 years. This project uses a multidisciplinary team approach to create a systematic and cultural change around recognition and resuscitation of deteriorating patients.

When evaluating the 16 interventions, it is difficult to identify which intervention had the greatest effect on non-ICU arrests. Due to the rarity of non-ICU arrests, the effect of an intervention may not be seen at the same time of the intervention. Additionally, multiple interventions targeted changing the culture surrounding escalation, resulting in changes that take time to become pervasive. Finally, the overlap of interventions combined with the rarity of non-ICU arrests further complicates linking individual interventions to outcomes. Based on the review of deteriorations and changes in hospital culture surround escalation, the Late Rescue Collaborative Steering Committee identified the four highlighted interventions: A) watcher huddle with mandatory note fields; B) automation of the PEWS calculation; E) creation of unit-based work groups; and F) improving notification of attending physicians prior to RRT activation as having the greatest impact on outcome.

We believe that the success of this project is based upon utilizing the characteristics of high reliability organizations (15): preoccupation with failure, reluctance to simplify, sensitivity to operations, deference to expertise, and commitment to resilience. Prior to implementation of the collaborative, rates of unplanned transfers, rapid response activations, and non-ICU arrests were not routinely tracked. Now each acute care unit is aware when an event occurs and a system is in place to evaluate each event for vulnerabilities that could lead to recurrence. Teams have agreed that trying harder is not the solution, rather coordinated systems change and a culture of safety are essential. Our global approach leveraged improved utilization of the electronic health record to identify at-risk patients; focused on situational awareness to enhance the safety culture to express clinical concern, and learning from prior episodes of severe deterioration can lead to improved outcomes, and established clear pathways for escalation. The impact of interventions on daily work flow was taken into account, and each intervention was designed to augment existing policies to improve recognition and rescue from deterioration. Finally, meetings of the multidisciplinary team as a whole group and as individual units have become standard procedure at our institution.

Other studies have looked at reducing codes or other adverse events in the acute care setting through system-wide changes. Hayes et al (11) attempted to reduce codes outside the ICU through a CHA-sponsored multihospital collaborative but failed to show a significant decrease after 1 year of implementation. The study by Hayes et al (11) has the advantage of including 20 hospitals in a structured program but was limited to a 12-month time frame. Our study has the strength of being followed over a 4-year time frame and demonstrates that change takes time. Additionally, Hayes et al (11) allowed the definition of a code outside the ICU to be determined by each hospital, possibly confounding the outcome data. Our definition of a code was standardized according to the American Heart Association Get with the Guidelines-Resuscitation definitions (16). Brady et al (17) conducted a single-center study at Cincinnati Children’s Hospital to reduce critical deteriorations by improving situational awareness. They identified five risk factors for serious safety events and unplanned floor to ICU transfers and implemented daily huddles and robust action plans. Instead of monitoring non-ICU arrests, they followed “unrecognized situation awareness failure events” that were defined as unplanned transfers requiring intubation, inotropes, or greater than or equal to three fluid boluses in the first hour after the event. Over a 2-year period, they saw a 50% reduction in their outcome measure.

Our study is limited by its single-center design. It is possible that the cultural shifts we implemented to improve recognition and early resuscitation would not be effective at another institution. Additionally, although the Late Rescue Collaborative has become the driving force for escalation and deterioration, it is possible that other hospital initiatives were partly responsible for the reduced deterioration events. For example, a monthly resident morbidity and mortality (M&M) conference started around the same time as the Late Rescue Collaborative. Late Rescue leadership participated in and advised the resident M&M but does not directly control changes implemented as a result of this conference. Finally, our outcome measure is a possible limitation. Although we did use an outcome measure with strict criteria, it is possible that our outcome measure did not capture other unplanned transfers resulting in significant morbidity and warranting close scrutiny.


Through the creation of a multidisciplinary hospital committee to track and evaluate patient deterioration, we have improved focus on tools for early identification of deteriorating patients. We suggest that these tools promote rescue of patients in earlier stages of active deterioration and successfully reduces non-ICU arrest events.

Figure 3.
Figure 3.:
Rapid response team (RRT) activation rates u-chart. Letters represent interventions detailed in Table 2. CL = centerline, LCL = lower confidence limit, UCL = upper confidence limit.
Figure 4.
Figure 4.:
Unplanned ICU transfer rates u-chart. Letters represent interventions detailed in Table 2. CL = centerline, LCL = lower confidence limit, UCL = upper confidence limit.


1. Sandquist M, Tegtmeyer K. No more pediatric code blues on the floor: Evolution of pediatric rapid response teams and situational awareness plans. Transl Pediatr 2018; 7:291–298
2. Tucker KM, Brewer TL, Baker RB, et al. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs 2009; 14:79–85
3. Agulnik A, Forbes PW, Stenquist N, et al. Validation of a Pediatric Early Warning Score in hospitalized pediatric oncology and hematopoietic stem cell transplant patients. Pediatr Crit Care Med 2016; 17:e146–e153
4. Skaletzky SM, Raszynski A, Totapally BR. Validation of a modified pediatric early warning system score: A retrospective case-control study. Clin Pediatr (Phila) 2012; 51:431–435
5. Akre M, Finkelstein M, Erickson M, et al. Sensitivity of the Pediatric Early Warning Score to identify patient deterioration. Pediatrics 2010; 125:e763–e769
6. Dean NP, Fenix JB, Spaeder M, et al. Evaluation of a Pediatric Early Warning Score across different subspecialty patients. Pediatr Crit Care Med 2017; 18:655–660
7. Parshuram CS, Dryden-Palmer K, Farrell C, et al.; Canadian Critical Care Trials Group and the EPOCH Investigators: Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: The EPOCH Randomized Clinical Trial. JAMA 2018; 319:1002–1012
8. Chan PS, Jain R, Nallmothu BK, et al. Rapid Response Teams: A Systematic Review and Meta-analysis. Arch Intern Med 2010; 170:18–26
9. Kolovos NS, Gill J, Michelson PH, et al. Reduction in mortality following pediatric rapid response team implementation. Pediatr Crit Care Med 2018; 19:477–482
10. Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children’s hospital. JAMA 2007; 298:2267–2274
11. Hayes LW, Dobyns EL, DiGiovine B, et al. A multicenter collaborative approach to reducing pediatric codes outside the ICU. Pediatrics 2012; 129:e785–e791
12. Randhawa S, Roberts-Turner R, Woronick K, et al. Implementing and sustaining evidence-based nursing practice to reduce pediatric cardiopulmonary arrest. West J Nurs Res 2011; 33:443–456
13. Watson A, Skipper C, Steury R, et al. Inpatient nursing care and early warning scores: A workflow mismatch. J Nurs Care Qual 2014; 29:215–222
14. Davis NL, Davis DA, Johnson NM, et al. Aligning academic continuing medical education with quality improvement: A model for the 21st century. Acad Med 2013; 88:1437–1441
15. Shapiro MJ, Jay GD. High reliability organizational change for hospitals: Translating tenets for medical professionals. Qual Saf Health Care 2003; 12:238–239
16. American Heart Association. Get with the guidelines- resuscitation overview. Available at Accessed June 10, 2019
17. Brady PW, Muething S, Kotagal U, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics 2013; 131:e298–e308

deterioration; early recognition; non-intensive care unit arrests; pediatrics; Pediatric Early Warning Scores; rapid response teams

Copyright © 2020 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies