Introduction
Pediatric sepsis is a leading cause of death among children, accounting for at least 7,000 deaths per year in the United States.1 sepsis is defined as (1) two or more age-based systemic inflammatory response syndrome criteria, (2) confirmed or suspected invasive infection, and (3) cardiovascular dysfunction, acute respiratory distress syndrome, or dysfunction of two or more noncardiovascular organ systems.2-4 5-7 8 sepsis diagnosis and treatment recommendations still lack consensus for how and when common treatments such as antibiotics and fluids are administered, leaving room for considerable practice variations.9-11
In pediatric sepsis , administering both intravenous (IV) antibiotics and an IV fluid bolus (IVFB) within one hour is associated with decreased mortality, whereas separate delivery of these elements is not.12,13 14-16 team communication and situational awareness are an integral part of mitigating patient risk.17-19 sepsis and delivering timely interventions remain challenging given the multitude of clinical presentations, lack of treatment consensus, and need for team -based communication .9
A review of infection-related decompensation (IRD) safety events at our organization identified four common causes: delayed recognition, delayed treatment, suboptimal team communication /collaboration, and inadequate knowledge. In August 2017, we implemented a predictive electronic alert tool for children at high risk (Children at High Risk Alert Tool [CAHR-AT]) to address inadequate/delayed recognition.6,7,20 team communication /collaboration.
This project combined two separately proven elements, an electronic alert tool (CAHR-AT) and team -based communication , to trigger recommended therapies and increase the relative percentage of patients receiving these critical treatments within 3 hours of CAHR-AT activation by 20%.11
Methods
Study Design and Setting
A quality improvement project involving three time epochs from August 2016 through December 2020 was undertaken at a 206-bed freestanding, urban, academic children's hospital with an annual emergency department (ED) volume of approximately 53,000 patients, designated as a Level 1 Pediatric Trauma Center. The Institutional Review Board approved this quality improvement project as nonhuman subjects' research. Cerner Millennium (Cerner Corporation, Kansas City, MO) was our primary electronic health record (EHR). We used the Model for Improvement framework and Toyota Problem Solving methodology to identify root causes, opportunities for improvement, and to test and implement changes throughout this project.21,22 https://links.lww.com/JHQ/A160 team to improve timely critical therapies for patients at risk for IRD.
Selection of Patients
All patients with ED who activated the CAHR-AT were included in the analysis (see demographics Table, Supplemental Digital Content 2, https://links.lww.com/JHQ/A161
The CAHR-AT algorithm (August 2017) used two target standards to define IRD as either severe sepsis within 48 hours of ED arrival or early sepsis within 24 hours of ED arrival that resulted in “major” or “extreme” severity of illness (SOI, a diagnosis-based index of maximum physiologic decompensation during the hospital stay).4,6,20,23 sepsis ” for two reasons: (1) this definition facilitated earlier recognition of patient deterioration and (2) this definition identified a broader patient population because sepsis alone is typically an infrequent occurrence and listing “sepsis ” as a discharge diagnosis is uncommon (1.8% of CAHR-AT activations).
Iterative Improvements and Related Analyses
During the project, the CAHR-AT algorithm was updated multiple times to improve sensitivity and positive predictive value.20 team -based assessment/reassessment and communication , situational awareness, and standardized treatment bundles.
Because the CAHR-AT identified a broad range of at-risk patients, the team developed a new visual branding that better emphasized its capabilities, changing from “Pediatric Sepsis Screening Tool” to “Children at High Risk Alert Tool (CAHR-AT),” which shifted the focus from predicting only sepsis to predicting general IRD within 48 hours of ED arrival20 https://links.lww.com/JHQ/A161 20 20
A checkered flag indicated the need for a CAHR assessment. The workflow required a provider (Doctor of Medicine/Nurse Practitioner) and nurse (hereafter known as “dyad”) to jointly assess the patient at the bedside within 10 minutes of the CAHR-AT activation. After assessment, the dyad agreed on a care plan and stoplight color (red, yellow, or green). The color indicated their clinical assessment of the patient's status, resources needed, and frequency of dyad reassessment (Figure 1 ). The dyad completed their respective CAHR assessment forms in the EHR. The nurse acknowledged assessment completion and recorded the provider's name; the provider selected the agreed-on stoplight color and documented the nurse's name to generate accountability. On completion of both forms, the chosen stoplight color displayed on the ED tracking board for situational awareness (Figure 2 ), making the team aware of both assessment completion and the patient's current risk assessment.
Figure 1.: Stoplight activation algorithm used to determine stoplight color during dyad huddle. CAHR = Children at High Risk.
Figure 2.: CAHR-AT assessment flowchart showing the emergency department workflow related to the CAHR-AT. CAHR-AT = Children at High Risk Alert Tool; RN = registered nurse.
In addition to indicating the patient's status and resource utilization urgency, treatment bundles associated with each stoplight color were recommended to address delayed interventions and connect standardized treatment to team assessment workflow (see Figure, Supplemental Digital Content 3, https://links.lww.com/JHQ/A162 https://links.lww.com/JHQ/A162 team reassessment of CAHR+ patients at a designated frequency based on their assigned stoplight color (Figure 2 ), which the team originally documented on paper.
Following the PDSA, the project team implemented an electronic version of the huddle workflow (January 2019). On assigning a stoplight color from the initial dyad assessment, a nursing task appeared in the events column on the ED tracking board at frequencies of 30, 60, or 120 minutes, depending on the color chosen (Figure 2 ). During each subsequent huddle, the dyad reassessed the patient, communicated progress of care plan implementation, and reevaluated the care plan. If appropriate, the huddle was also used to change the stoplight color to reflect current patient status more accurately. The only other time a stoplight color changed was if the CAHR score increased because of new abnormal values for one or more of the 13 EHR factors evaluated, prompting another checkered flag and dyad assessment (Figure 2 ).
To standardize treatment, ED red and yellow stoplight order sets were developed based on the previously suggested stoplight bundles (see Figure, Supplemental Digital Content 3, https://links.lww.com/JHQ/A162
To obtain feedback during the project on the perceived impact of process improvements on their workflow and patient care, a subjective electronic anonymous survey was distributed to ED staff including providers, nurses, clinical technicians, and administrative support staff.
Method of Analysis
We analyzed initial huddle compliance in CAHR+ patients to assess process improvements related to team -based communication . To measure treatment-related outcomes, we analyzed the percentage of CAHR+ patients who received specific bundle elements (IV/IM antibiotics and IVFBs) by 3 hours post–CAHR-AT activation and the percentage of CAHR+ patients with ED collected bacterial cultures. Only bacterial cultures for blood, urine, cerebral spinal fluid, or fluid from a presumed sterile cavity were included. Although our internal goals were to administer these bundle elements within one hour, we analyzed the data based on more widely accepted published recommendations for antibiotic administration within 3 hours.10,24
Manual CAHR+ chart evaluations were completed to identify the percentage of patients who received either IV/IM antibiotics or an IVFB at an outside facility before arriving in the ED or when the EHR was offline. To ensure consistency in chart reviews, randomly selected charts were examined by all reviewers as a teaching aid to assess interrater reliability.
All data were analyzed using statistical process control charts created using Minitab Statistical Software v.19 (State College, PA). Data from the first and last time epochs were also analyzed using an exact chi-square test.
Results
The CAHR-AT currently activates on ∼2.0% of ED arrivals where the positive and negative predictive values for high SOI outcomes were 22.5% and 98.7%, respectively.20 https://links.lww.com/JHQ/A161
Team -Based Communication Process MeasuresCompared with the paper-based second time epoch, communication , situational awareness, and team -based assessment improved after electronic implementation of huddle workflow; initial huddles increased from 7.8% to 65.3% for CAHR+ patients with either a red or yellow stoplight activated (p < .0001; Figure 3 ). Clinical teams completed an initial huddle 70.8% of the time for red stoplight designation, 60.5% for yellow, and 26.1% for green. Figure 3 data exclude patients with the highest level of acuity because providing immediate clinical attention took precedence over huddle documentation (see Table, Supplemental Digital Content 2, https://links.lww.com/JHQ/A161 team should complete and document an initial huddle every time.
Figure 3.: Percent of CAHR red and yellow light patients with an initial dyad huddle completed. CAHR = Children at High Risk.
The electronic survey showed 69% of ED staff agreed that communication during huddles improved patient care, whereas 2% disagreed. In addition, 71% agreed and 0% disagreed that the new CAHR processes have or are beginning to improve communication within the care team . A total of 48 ED staff responded to the survey, approximately 50% of all ED nurses and providers.
Treatment Outcome Measures
The percentage of CAHR+ patients with ED collected bacterial cultures increased from 45.3% pre–CAHR-AT to 56.4% postelectronic huddle workflow (24.5% relative increase; p < .001; see Figure, Supplemental Digital Content 4, https://links.lww.com/JHQ/A163 p < .0001; Figure 4A ), while the percentage of CAHR+ patients who received an IVFB by 3 hours post-CAHR activation increased from 49.0% to 55.2% (12.7% relative increase; p = .0009; Figure 4B ). Apart from IVFB administration, we met our aim of improving the timeliness of critical treatments by 20% of the preimplementation baselines.
Figure 4.: A, Percent of patients with CAHR-AT activation who received IV/IM antibiotics by 3 hours postactivation. B, Percent of patients with CAHR-AT activation who received IVFB by 3 hours postactivation. CAHR-AT = Children at High Risk Alert Tool; IM = intramuscular; IV = intravenous; IVFB = IV fluid bolus.
Limitations
There were several limitations to this project. It is unknown how this alert tool and its associated workflow would perform at other institutions. Our method of evaluating this population as IRD may not translate to other health systems because of differences in electronic data availability and patient populations.4,20
Some patients receive antibiotics and IVFBs before arrival, and our current transport and in-hospital charting systems do not communicate with each other. Manual chart reviews were required to ascertain that ∼7% of CAHR+ patients received antibiotics at an outside hospital because of missing EHR documentation.
Currently, the CAHR-AT only accepts the provider's stoplight color designation. Occasionally, clinicians disagreed on the stoplight color designation, especially for noncritically ill patients. In these situations, nursing generally favored a higher risk designation compared with providers. This limitation may have leveled patients at a lower risk than nursing felt was appropriate. We will consider adding the stoplight selection capability for nursing. Evaluating CAHR-AT+ patients through admission might help determine the effect of ED treatment on patient outcomes.
Discussion
This project demonstrated that implementing an EHR-based tool to identify children at risk for IRD did not result in improved treatment-related outcomes until accompanied by standardized team huddles and reassessments, team -based decision making, and standardized treatment bundles.
Most other electronic alert tools attempt to aid in identifying active sepsis , making scoring more likely to align with the patient's status, and are not intended to be predictive.5,8,25,26 sepsis or other infection-related diagnoses more frequently. This shift likely resulted from implementation of consistent standardized huddles and the resulting collaborative decision making. To further gain stakeholder support, we used newsletters, 1:1 check-ins, and regular meetings.
Changing the alert tool name may have helped decrease diagnostic bias. With the focus initially on sepsis only, providers believed the tool was unreliable when the patient was not found to be septic. Rebranding encouraged critical thinking to consider a broader range of infection-related diagnoses.
Transitioning toward standardized team -based assessment and communication changed both physical workflow and the individualized, impromptu communication style common in the ED. Despite the CAHR-AT indicating a high-risk patient, staff inconsistently coordinated recurrent huddles, especially when perceived urgency was low, such as patients with a yellow or green stoplight. After electronic huddle implementation, it took approximately 6 months to see decreased variation in huddle coordination. Ongoing ED staff feedback about instances where they were not in compliance with the new workflow helped illustrate the value of team -based reassessments and standardized treatment bundles. This shifted the ED toward a culture of high reliability. Before electronic huddle implementation, interdisciplinary communication was disjointed and often required excessive time for effective nurse provider communication . Afterward, real-time joint decisions helped develop a sense of unity between nurse, provider, and patient/family. Over time, both providers and nurses took more initiative to complete the huddle in the required timeframe. Although it is common to include an initial huddle for institutions with sepsis alert tools, implementation of standardized reassessments for at-risk patients, as performed in this project, has not been reported to our knowledge.5,8,25 Figure 3 , the team learned that consistent huddle completion allowed for real-time adaptations in care as the illness progressed or the response to treatment became evident.
Standardized order sets optimize care delivery and outcomes and improve patient safety which, in turn, prevents therapeutic miscues and reduces time to intervention.15
Compared with IV/IM antibiotic administration, improved delivery of an IVFB by 3 hours postactivation was modest. Providers expressed hesitance ordering IVFBs without evidence of abnormal perfusion or hypotension, despite some literature recommending bundle administration for this high-risk population.13 Sepsis Guidelines (2020), “fluid bolus therapy in children with moderate hypotension was not beneficial or harmful compared with maintenance fluid only.”10 10,13,27
We maintained an expectation of antibiotic administration within one hour from completion of the dyad assessment for consistency in practice. However, we chose to analyze the data based on the three-hour published recommendations for antibiotic administration to improve the applicability of our findings to other organizations. A one-hour goal may have contributed to the significant increase in IV/IM antibiotic administration. The 33.7% relative improvement in IV/IM antibiotic administration from the preproject baseline is evidence of a shift in ED culture toward considering IRD in the differential diagnosis and proactively treating patients. This improvement magnitude aligns with other health systems, such as the 20% improvement in antibiotic administration times described by Lockwood et al28 sepsis response system. In retrospect, expecting antibiotics and IVFBs within one hour, without being hemodynamically compromised, may be unrealistic, especially considering the predictive accuracy of the CAHR-AT. In the future, we could increase our goal to 3 hours as other studies recommend while maintaining optimum outcomes.10,23,29
Conclusions
Although an electronic alert tool may improve early identification of children at high risk, improved timeliness of critical treatments for children at high risk for IRD only occurred when the alert tool was supplemented with standardized team -based assessment and communication , shared situational awareness, and implementation of standardized treatment protocols.
Implications
Cumulatively, team -based communication and standardized treatment increase the consideration of a broad range of infection-related diagnoses, increasing compliance and timeliness of treatment-based interventions. Together, these interventions improve the number of at-risk patients who receive initial huddles and timely treatment. Standardized reassessments encourage multidisciplinary discussions about the signs and treatment of IRD and aid in discouraging cognitive bias by considering different caregiver assessments.
Authors' Biographies
Elizabeth M. Martinez, RN, BSN, CHI, is a Business Process Engineer in Quality and Safety at Children's Hospital of the King's Daughters in Norfolk, VA, and is a licensed nurse (RN) and nationally certified health care interpreter. She manages quality improvement projects across the health system.
Robert J. Sepanski, MS, is a Senior Clinical Data Analyst at Children's Hospital of The King's Daughters (CHKD) in Norfolk, VA, and has graduate training in medical sciences, epidemiology, and biostatistics. He has worked in quality improvement and the design and implementation of adverse event screening tools at CHKD and at Le Bonheur Children's Hospital in Memphis, TN.
A. Dawn Jennings, RN, BSN, is the Nurse Operations Manager of the Emergency Department at Children's Hospital of the King's Daughters in Norfolk, VA. She completed her Bachelor of Nursing from Western Governors University.
James M. Schmidt, MD, MSPH, FAAP, FACEP , is the Associate Medical Director in the pediatric Emergency Department at Children's Hospital of the King's Daughters and Associate Professor of Pediatrics at Eastern Virginia Medical School in Norfolk, VA.
Thomas J. Cholis, MD, is a Pediatric Intensive Care Physician at Children's Hospital of The King's Daughters (CHKD) in Norfolk, VA, since 2005. He did his training at Pediatrics in Tulane/Ochsener and his Pediatric Critical Care fellowship at Children's National Medical Center. He is currently an Assistant Professor of Pediatrics at EVMS and an Assistant Medical Director of the CHKD Quality and Safety team .
Meaghan E. Dominy, MD, completed her pediatrics training at the University of South Carolina in Columbia, SC, and later a Pediatric Critical Care Fellowship at the University of Mississippi in Jackson, MS. She is currently a Pediatric Intensivist at Children's Hospital of the King's Daughters in Norfolk, VA.
Sanaz B. Devlin, MD, MS, MEdL, completed her general pediatrics training at Madigan Army Medical Center. She served as a general pediatrician in the Army for 7 years after her training. She is currently a pediatric hospitalist at Children's Hospital of The King's Daughters in Norfolk, VA. She is also the Pediatric Residency Program Director at Eastern Virginia Medical School. Dr. Devlin oversees the pediatric residents' quality improvement curriculum.
Lindsay Floyd Eilers, MD, completed her general pediatrics training at Eastern Virginia Medical School and Children's Hospital of The King's Daughters in Norfolk, VA. She completed a fellowship in pediatric cardiology at Baylor College of Medicine and Texas Children's Hospital and is currently an advanced fellow in Pediatric Interventional Cardiology at Baylor College of Medicine and Texas Children's Hospital in Houston, TX.
Arno L. Zaritsky, MD, was the Vice Chair of Pediatrics and Chief of Pediatric Critical Care Medicine at the University of Florida College of Medicine before spending his past 7 years in Norfolk, VA, before retiring as the Vice President for Clinical Services at Children's Hospital of The King's Daughters and Professor of Pediatrics at Eastern Virginia Medical School.
Sandip A. Godambe, MD, PhD, MBA, is the SVP Medical Affairs and Chief Medical Officer at Children's Hospital of Orange County (CHOC) in Orange, CA, and Professor of Pediatrics at the University of California Irvine in Irvine, CA. He oversees the clinical operations at CHOC's two hospital campuses, ambulatory network, and virtual health programs. He has primary oversight of activities involving medical staff, pharmacy, quality, safety, regulatory affairs, and risk and accreditation.
Acknowledgments
The authors thank Coleman McGehee; DeAnn Capanna; Catherina Stewart; David Douglas; Laura Sass, MD; Sara Dooley, RN; Stephanie Graefe, RN; Tracy Grimes, RN; Rebecca Roenker, RN; Rachel Horton, RN; Ngoc Loving, RN; Michael Poirier, MD; Adam Campbell, PhD; Christopher Mangum, CSSBB; Phillip Thomas, MD; Cecilia Sotomayor, RN; Malinda Sithisaiya; Angela Ortiz, RN, BSN, WTA; Louellen Klein, RN; Delores Deyarmond, RN; Anne McEvoy, MD.
References
1. Weiss SL, Balamuth F, Hensley J, et al. The epidemiology of hospital death following pediatric severe
sepsis : When, why, and how children with
sepsis die. Pediatr Crit Care Med. 2017;18(9):823-830.
2. Rhodes A, Evans LE, Alhazzani W, et al. Surviving
sepsis campaign: International guidelines for management of
sepsis and septic shock: 2016. Intensive Care Med. 2017;43(3):304-377.
3. Schlapbach LJ, Kissoon N. Defining pediatric
sepsis . JAMA Pediatr. 2018;172(4):312-314.
4. Goldstein B, Giroir B, Randolph A. International pediatric
sepsis consensus conference: Definitions for
sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005;6(1):2-8.
5. Balamuth F, Alpern ER, Abbadessa MK, et al. Improving recognition of pediatric severe
sepsis in the emergency department: Contributions of a vital sign-based electronic alert and bedside clinician identification. Ann Emerg Med. 2017;70(6):759-768.e2.
6. Sepanski RJ, Godambe SA, Mangum CD, Bovat CS, Zaritsky AL, Shah SH. Designing a pediatric severe
sepsis screening tool. Front Pediatr. 2014;2:56.
7. Sepanski RJ, Godambe SA, Zaritsky AL. Pediatric vital sign distribution derived from a multi-centered emergency department database. Front Pediatr. 2018;6:66.
8. Cruz AT. Between the devil and the deep blue sea: Use of real-time tools to identify children with severe
sepsis in the pediatric emergency department. Ann Emerg Med. 2017;70(6):769-770.
9. Dellinger RP, Schorr CA, Levy MM. A users' guide to the 2016 Surviving
Sepsis Guidelines. Intensive Care Med. 2017;43(3):299-303.
10. Weiss SL, Peters MJ, Alhazzani W, et al. Surviving
sepsis campaign international guidelines for the management of septic shock and
sepsis -associated organ dysfunction in children. Intensive Care Med. 2020;46(suppl 1):10-67.
11. Silverman AM. Septic shock: Recognizing and managing this life-threatening condition in pediatric patients. Pediatr Emerg Med Pract. 2015;12(4):1-25.
12. Dwyer J. An infection, unnoticed, turns unstoppable. N Y Times 2012, July 11.
13. Evans IVR, Phillips GS, Alpern ER, et al. Association between the New York
sepsis care mandate and in-hospital mortality for pediatric
sepsis . JAMA 2018;320(4):358-367.
14. Remick DG, Ayala A, Chaudry IH, et al. Premise for standardized
sepsis models. Shock 2019;51(1):4-9.
15. Buchert AR, Butler GA. Clinical pathways: Driving high-reliability and high-value care. Pediatr Clin North Am. 2016;63(2):317-328.
16. Leavitt M. Medscape's response to the Institute of Medicine Report: Crossing the quality chasm: A new health system for the 21st century. Med Gen Med. 2001;3(2):2.
17. Brady PW, Muething S, Kotagal U, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics 2013;131(1):298-308.
18. Brady PW, Wheeler DS, Muething SE, Kotagal UR. Situation awareness: A new model for predicting and preventing patient deterioration. Hosp Pediatr. 2014;4(3):143-146.
19. Soberano BT, Brady P, Yunger T, et al. The effects of care
team roles on situation awareness in the pediatric intensive care unit: A prospective cross-sectional study. J Hosp Med. 2020;15:594–597.
20. Sepanski RJ, Zaritsky AL, Godambe SA. Identifying children at high risk for infection-related decompensation using a predictive emergency department-based electronic assessment tool
. Diagnosis (Berl) 2021;8:458-468.
21. Langley G, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey-Bass Publishers; 2009.
22. Bonini J, Godambe SA, Mangum CD, Heer J, Black S. In: Shah R, Godambe S, eds. Building an Engaging Toyota Production System (TPS) Culture to Drive Winning Performance for Our Patients, Caregivers, Hospitals and Communities. New York, NY: Spring Nature Publishers; 2021.
23. Averill RF, McCullough EC, Goldfield N, et al. 3M APR DRG Classification System, Version 31.0: Methodology Overview. Wallingford, CT: 3M Health Information Systems; 2013.
https://www.hcup-us.ahrq.gov/db/nation/nis/grp031_aprdrg_meth_ovrview.pdf . Accessed June 21, 2022.
24. Weiss SL, Fitzgerald JC, Balamuth F, et al. Delayed antimicrobial therapy increases mortality and organ dysfunction duration in pediatric
sepsis . Crit Care Med. 2014;42(11):2409-2417.
25. Lloyd JK, Ahrens EA, Clark D, Dachenhaus T, Nuss KE. Automating a manual
sepsis screening tool in a pediatric emergency department. Appl Clin Inform. 2018;9(4):803-808.
26. Scott HF, Colborn KL, Sevick CJ, et al. Development and validation of a predictive model of the risk of pediatric septic shock using data known at the time of hospital arrival. J Pediatr. 2020;217:145-151.e6.
27. Kawasaki T. Update on pediatric
sepsis : A review. J Intensive Care 2017;5:47.
28. Lockwood JM, Scott HF, Wathen B, et al. An acute care
sepsis response system targeting improved antibiotic administration. Hosp Pediatr. 2021;11(9):944-955.
29. Han M, Fitzgerald JC, Balamuth F, et al. Association of delayed antimicrobial therapy with one-year mortality in pediatric
sepsis . Shock 2017;48(1):29-35.
