Secondary Logo

Journal Logo

Observational Study

The Modified Early Warning Score: A Useful Marker of Neurological Worsening but Unreliable Predictor of Sepsis in the Neurocritically Ill—A Retrospective Cohort Study

Hester, Jeannette MSN, RN, CCRN1; Youn, Teddy S. MD2; Trifilio, Erin PhD3; Robinson, Christopher P. DO, MS4,5; Babi, Marc-Alain MD4,5; Ameli, Pouya MD4,5; Roth, William MD4,5; Gatica, Sebastian MD5,6; Pizzi, Michael A. DO, PhD4,5; Gennaro, Aimee PharmD7; Crescioni, Charles RN8; Maciel, Carolina B. MD, MSCR4,5; Busl, Katharina M. MD, MS4,5

Author Information
Critical Care Explorations: May 2021 - Volume 3 - Issue 5 - p e0386
doi: 10.1097/CCE.0000000000000386


Early warning scores and systems are used in clinical practice by hospital rapid response programs as standardized means for early identification of patients at high risk of deterioration (1). Different aggregate weighted Early Warning Scores (EWSs) were developed to facilitate identification of at-risk patients in various clinical settings—most based on simple physiologic measurements and clinical observations, typically including heart rate (HR), respiratory rate (RR), blood pressure, level of consciousness, and other metrics depending on selected modification tool (2). They have shown to be more accurate than previous generations of prognostic criteria where a single variable set of any, equally weighted abnormal physiologic thresholds would trigger the risk alarm system and lead to rapid response team activation (3). The Modified Early Warning Score (MEWS) is one such scoring system, validated in adult medical-surgical patients, to identify risk of clinical deterioration and mortality when the MEWS is greater than or equal to 5 (4). It comprises five physiologic variables: systolic blood pressure (SBP), HR, RR, temperature, and mental status. Additional modifications of the MEWS with focus on Sepsis Recognition Score (MEWS-SRS) and addition of WBC count (Fig. 1) were found 96.5% sensitive and 96.7% specific in medical-surgical ICUs as a method for early identification and treatment initiation in the setting of infection or sepsis, and reduction of sepsis-related mortality (5,6).

Figure 1.
Figure 1.:
Modified Early Warning Score-Sepsis Recognition Score. *Do NOT score latest WBC, if low WBC is due to oncology therapy (e.g., chemo, bone marrow suppression). **Do NOT score mental status if patient has neurologic injury affecting mental status (e.g., traumatic brain injury, stroke) unless change from baseline. ICU only—If patient is receiving sedatives, score level of consciousness if patient’s Richmond Agitation and Sedation Scale (RASS) is less than or equal to goal RASS. HR = heart rate (beats/min), RR = respiratory rate (breaths/min), SBP = systolic blood pressure (mm Hg), Temp = temperature (degree Celsius).

Hospital-wide initiatives are often applied to all wards or ICUs to minimize variability and standardize bundle-care approaches (7). However, there are no specific data on the effectiveness of the MEWS, MEWS-SRS, or any other EWS, for the neurocritically ill population. We sought to determine the prognostic ability of the MEWS-SRS and MEWS to identify clinical deterioration, occurrence of infection or sepsis, and mortality in critically ill neurologic patients. Our hypothesis was that neurologic worsening is a contributive factor to trigger the warning score threshold, and our objective was to characterize performance of MEWS and MEWS-SRS in the neurocritically ill patient population.



We retrospectively reviewed prospectively collected data from a single, tertiary-care, academic center in North Central United States of consecutive adult patients admitted to the neuro-ICU or neurologic intermediate care unit (NIMC) from January 2013 to December 2016. The MEWS-SRS scoring system is used throughout our institution as standard EWS and recorded by the bedside nurse upon admission and every 4 hours, with closer interval follow-up when elevated greater than or equal to 5 (MEWS ≥ 5 as the defined trigger threshold). This is accomplished by the primary bedside nurse who will evaluate the clinical variables for timing and accuracy. The primary nurse scores the patient into several categories within the MEWS-SRS/MEWS tool and each category is assigned weighted scores from zero to three points (best to worst compared with normal findings). Key variables include most recent WBC count, specific vital signs of temperature, HR, RR, SBP, and mental status. Mental status measures include unresponsive coma (3 points), stupor, responds to noxious stimuli (2 points), lethargic, responds to voice or tap (1 point), alert calm, cooperative (0 points), mildly agitated, confused, anxious (1 point), very agitated, requires restraints (2 points), and extremely agitated and danger to self or others (3 points). The scoring takes into account the baseline neurologic assessment. The combined categories are tallied and validated by the primary nurse in the electronic medical record (EMR). For MEWS-SRS scores greater than or equal to 5, the nurse will initiate a call to initiate the critical care team to evaluate the patient immediately due to the elevated MEWS-SRS. The critical care time will then respond within 3 minutes and evaluate the patient and intervene clinically as indicated, including notifying the primary provider (e.g., the admitting neurosurgeon) of the clinical condition. Nursing may notify the critical care team if the patient’s clinical condition declines but does not reach a MEWS score greater than or equal to 5. The study was approved by the University of Florida Institutional Review Board (201700862) including a waiver of consent.

Clinical Variables

Baseline characteristics, MEWS-SRS scores, rates of sepsis, inhospital mortality, and discharge destination were abstracted for all patients. For patients with MEWS-SRS and MEWS scores less than 5, diagnosis of sepsis was retrieved from International Classification of Diseases (ICD), 9th Revision/10th Revision-based coding. For patients with MEWS-SRS score(s) greater than or equal to 5 at admission or during their neuro-ICU/NIMC stay, diagnoses (based on ICD codes, attending physician progress notes), physiologic variables, infections (based on microbiology results of blood cultures, urinalysis/urine culture, sputum cultures, bronchoalveolar lavage cultures, and radiographic imaging such as chest radiograph or CT scan), sepsis (following the Third International Consensus Definitions for Sepsis and Septic Shock [8]), treatment with antibiotics, and neurologic worsening were additionally abstracted manually from the EMR. Main scoring contributor was defined as the physiologic category that contributed most points to the MEWS-SRS score (for the individual elements of MEWS-SRS; Fig. 1). In cases of ties, the main scoring physiologic variable was defined as the variable that had the most extreme difference from prior value or was explicitly stated to be of concern in the note. Altered mentation included agitation, anxiety, and altered consciousness. Medical instability was defined as new/worsened cardiac arrhythmia, hemodynamic instability (requiring vasopressor support), or respiratory instability, requiring increasing respiratory care. Neurologic complications and worsening (“neuro-worsening”) were also abstracted and included cerebral edema, midline shift, hydrocephalus, intracranial pressure (ICP) elevation (ICP > 22 mm Hg for ≥ 5 min), intracranial hemorrhage (including hemorrhagic transformation), vasospasm or delayed ischemic neurologic deficit, or seizure/status epilepticus (nonconvulsive or convulsive). Subtypes of both neuro-worsening and medical instability were abstracted from EMR description at the time when the MEWS-SRS was greater than or equal to 5. A flow sheet for data collection is described in Figure 2. The original MEWS score (as described by Subbe et al [4]) was calculated from collected data with application of the above-listed data for MEWS original).

Figure 2.
Figure 2.:
Clinical algorithm and data acquisition for Modified Early Warning Score (MEWS)-Sepsis Recognition Score greater than or equal to 5 triggers.


Primary outcomes included discovery of a new infection, sepsis, or escalation of level of care (ELOC) for NIMC patients, and neuro-worsening.

Data Analysis

Descriptive statistics and parametric statistical comparisons, including t test mean comparisons and chi-square with post hoc tests, were performed with the R statistical package 3.5.3 and SPSS Version 25 (IBM SPSS Software). Each admission was treated as an independent case for analyses.


Cohort Characteristics

Of a total of 7,303 admissions, MEWS-SRS (and MEWS) data were available for 7,286 patients, and these were included in the analysis. Forty-three patients were admitted to the neuro-ICU/NIMC more than once during the study period. Data for MEWS-SRS are presented here; data for MEWS (original) are presented as Supplementary Material (

Demographics and discharge disposition are displayed in Table 1. Of the 7,286 patients, 1,120 (15.5%) had a MEWS-SRS score(s) greater than or equal to 5. Those in the MEWS-SRS greater than or equal to 5 group were significantly younger. There was no association between sex and MEWS-SRS/MEWS score greater than or equal to 5 (p = 0.303). There was a significant association between minority status (i.e., non-Caucasian) and MEWS-SRS score (p < 0.001). Elevated MEWS-SRS was significantly associated with discharge disposition (p < 0.001). Patients with a MEWS-SRS less than 5 were more frequently discharged home. Those in the MEWS-SRS greater than or equal to 5 group had longer hospital and ICU LOS and were more likely to be discharged to hospice (p < 0.001). Inhospitality mortality of the MEWS-SRS greater than or equal to 5 group was 22% (247/1,120), compared with 6% (379/6,166) for patients with MEWS-SRS less than 5 at all times, and 9% (630/7,286) in the cohort as a whole (χ2 = 305.377; p < 0.001). When applying the original MEWS score criteria, 836 (11.5%) would have met the threshold of 5 at some point during admission, with similar associations of elevated score for length of stay (LOS), mortality, medical history, and discharge disposition, while age was not significantly associated. The odds ratio for mortality if MEWS-SRS greater than or equal to 5 was 1.453 (95% CI, 1.346–1.530). Of patients with MEWS-SRS greater than or equal to 5, 14.3% (160) were diagnosed with sepsis, while 345 (5.6%) of the 6,166 patients who did not reach the threshold MEWS-SRS score of 5 during their hospitalization were diagnosed with sepsis. The positive predictive value of the MEWS-SRS greater than or equal to 5 for sepsis was 6.0% and sensitivity was 16.5%. See Supplementary Table 1 ( for results utilizing the original MEWS score.

Table 1. - Demographics, Characteristics, and Outcomes for Neuro-ICU Patients (n = 7,286)
Early Warning Score MEWS-SRS < 5, n = 6,166 (85%) MEWS-SRS ≥ 5, n = 1,120 (15%) p
Race/ethnicitya,b, n (%)
 White 4,831 (78) 800 (72)
 Black or African American 926 (15) 212 (19) < 0.001
 Other 409 (7) 108 (9)
Sex, n (%)
 Male 3,197 (52) 562 (50.2)
 Female 2,968 (48) 558 (49.8)
Age (yr)c, mean (sd) 60.2 (16.9) 58.9 (17.6) < 0.05
ICU LOS (d)a, mean (sd) 3.7 (5.3) 12.9 (12.1) < 0.001
Hospital LOS (d)a, mean (sd) 6.4 (15.1) 16.3 (15.5) < 0.001
Inhospital mortalitya, mean (sd) 379 (6.1) 248 (22.1) < 0.001
Medical history, n (%)
 Hypertension 3,820 (62.0) 705 (62.9)
 Diabetes 1,467 (23.8) 260 (23.2)
 Coronary artery disease 175 (2.8) 33 (2.9)
 Chronic obstructive pulmonary disease 716 (11.6) 144 (12.9)
 Cancera 1,638 (26.6) 211 (18.8) < 0.001
 Liver disease 143 (2.3) 27 (2.4)
Sepsisa 345 (5.6) 160 (14.3) < 0.001
Discharge dispositiona, n (%) < 0.001
 Home/home care/rehabilitationa 4,591 (74) 422 (38)
 Hospicea 186 (4) 85 (8)
 Nursing home/long-term acute care hospital/othera 1,012 (16) 365 (32)
LOS = length of stay, MEWS-SRS = Modified Early Warning Score-Sepsis Recognition Score.
aStatistical significance at p < 0.001 level.
bOf note: the p for race is reflecting a dichotomized evaluation (nonminority vs minority).
cStatistical significance at p < 0.05 level.
Descriptive statistics with χ2 and t tests by MEWS-SRS trigger (MEWS-SRS ≥ 5 vs MEWS-SRS < 5).

MEWS-SRS Greater Than or Equal to 5 Cohort

Admission neurologic diagnoses of the MEWS-SRS greater than or equal to 5 group (n = 1,120), clinical condition and clinical change surrounding the elevation of the MEWS score are shown in Table 2 (see Supplementary Table 2,, for results utilizing the original MEWS score). Main neurologic admission diagnoses included cerebral infarction, subarachnoid hemorrhage, and intracerebral hemorrhage. At the time of MEWS-SRS greater than or equal to 5, 64.5% (722) were mechanically ventilated and 39.4% (444) were treated with one or several antimicrobials.

Table 2. - Neurologic Diagnoses, Clinical Condition, and Change Surrounding Modified Early Warning Score-Sepsis Recognition Score Greater Than or Equal to 5 Trigger in Patients With Modified Early Warning Score-Sepsis Recognition Score Greater Than or Equal to 5 (n= 1,120)
Primary Diagnosis n (%)
Cerebral infarction 270 (24.1)
Subarachnoid hemorrhage 206 (18.4)
Intracerebral hemorrhage 128 (11.4)
Neoplasm 94 (8.4)
Subdural hematoma 68 (6.1)
CNS infections 66 (5.9)
Spine 66 (5.9)
Encephalopathy 60 (5.4)
Seizures/status epilepticus 59 (5.3)
Traumatic brain injury 49 (4.4)
Aneurysm/arteriovenous malformation (unruptured) 19 (1.7)
Other 34 (3.0)
Intubated 722 (64.5)
Treated with antibiotics 444 (39.6)
Presence of infection or sepsis
 Infection (total) 479 (42.8)
 New infection discovered 157 (14.0)
 Sepsis (total) 160 (14.3)
 New sepsis discovered 62 (5.5)
 New sepsis or infection 164 (14.6)
Underlying process/trigger
 Medical worsening 331 (29.6)
  Respiratory rate 281 (25.1)
  Blood pressure 157 (14.0)
  Heart rate 249 (22.2)
  Mental status 299 (26.7)
  Temperature 31 (2.8)
  WBC count 103 (9.2)
 Neuro-worsening 343 (30.6)
  Intracerebral hemorrhage 108 (9.6)
  Cerebral edema 88 (7.9)
  Seizures/status epilepticus 71 (6.3)
  Intracranial pressure elevation 47 (4.2)
  Hydrocephalus 45 (4.0)
  Vasospasm/delayed ischemic neurological deficit 43 (3.8)
  Midline shift 42 (3.8)

Infectious Complications and Clinical Condition Leading to MEWS-SRS Elevation

A new infection was discovered in 14.0% (157), and a new diagnosis of sepsis was made in 5.5% (62). Pneumonia (96; 39.5%) and urinary tract infection (69; 28.4%) were diagnosed most commonly with a majority of antibiotics being initiated for those two diagnoses.

The main contributors for the MEWS to elevate greater than or equal to 5 were altered mentation, tachypnea, and tachycardia (Table 3; and Supplementary Table 3, Tachypnea or tachycardia predominated as the main physiologic variables leading to either initiation or escalation of antibiotics or an escalation in the level of care.

Table 3. - Main Scoring Trigger in Patients With Modified Early Warning Score-Sepsis Recognition Score Greater Than or Equal to 5 (n = 1,120) and Relationship to Escalation in Antibiotic Treatment or Level of Care
Modified Early Warning Score Trigger Altered Mentation Including Agitation and Pain ↑↓Blood Pressure ↑↓Heart Rate ↑↓Respiratory Rate Temperature ↑↓WBC n
Total, n (%) 299 (27) 157 (14) 249 (22) 281 (25) 31 (3) 103 (9) 1,120
Antibiotics 50 34 55 61 9 34 243 (22)
ELOC 8 8 22 11 3 6 58 (5)
Both antibiotics + ELOC 2 2 11 3 2 5 25 (2)
ELOC = escalation in level of care.
In parentheses: % relative to total cohort of n = 1,120.

Neurologic Worsening

The most common clinical change leading up to a MEWS-SRS score greater than or equal to 5 was neuro-worsening (343, 30.6%), while medical worsening was documented in 29.6% (331). Among patients with worsening neurologic status, two or more neurologic changes were present in 92 patients (27%) at the time of their MEWS-SRS elevation. Medical instability was present in 136 patients (39.6%) with neuro-worsening.

Resource Utilization and Mortality

Table 4 presents the resources that were used following a MEWS-SRS alert: initiation or escalation of antibiotics and ELOC. Antibiotics were initiated or escalated in 243 patients (21.6%) following a MEWS-SRS activation. Of those who had antibiotics added and did not have a new infection or new sepsis post-MEWS-SRS greater than or equal to 5 (n = 124), 21 died in the hospital (16.9%). Mortality of patients with sepsis at any time (prior or at MEWS-SRS ≥ 5 time) was similar, 24% (39/160), compared with 22% (209/960) for patients who never had sepsis (χ2 = 0.539; p = 0.436). Among those in the MEWS-SRS greater than or equal to 5 group who were newly diagnosed with sepsis (n = 62), 11 (18%) died. This compares to a 31% mortality rate (37/119) in those without sepsis diagnosis (χ2 = 3.728; p = 0.053). Similarly, those diagnosed with infection following MEWS-SRS greater than or equal to 5 (n = 157) had a lower mortality rate than those without diagnosed infection (13% vs 19%; χ2 = 2.819; p = 0.093). In cases of antibiotic initiation or antibiotic escalation, 23.4% did not have an infection or sepsis and rather had neurologic deterioration as main etiology for MEWS-SRS elevation. This was also seen for ELOC, with one out of five (20.7%, 58) having neurologic deterioration without a new diagnosis of infection/sepsis. Data on assumed resource utilization (antibiotic initiation/escalation and escalation of care) in patients with and without neuro-worsening were calculated for the original MEWS score greater than or equal to 5 and are similar (Supplementary Table 4,

Table 4. - Resource Utilization After Modified Early Warning Score-Sepsis Recognition Score Trigger in Relation to Neuro-Worsening
Antibiotics Initiated/Escalated, n = 243 ELOC, n = 58 Both Antibiotics and ELOC, n = 25
S/I No S/I S/I No S/I S/I No S/I
Neuro-worsening 27 29 4 12 3 2
No neuro-worsening 92 95 17 25 13 7
ELOC = escalation in level of care, S/I = sepsis/infection. Newly diagnosed (NEW) sepsis or NEW infection post-Modified Early Warning Score.


In this study, we investigated the application of the MEWS-SRS in the neurocritically ill patient population and found that MEWS-SRS is associated with higher inhospital mortality, while not a reliable tool to indicate new infection or sepsis for this population, with a positive predictive value of only 6%. Instead, the MEWS-SRS is triggered evaluation preferentially in patients with neurologic deterioration. When considering the utilization of resources, antibiotics were initiated or escalated in one of five patients with MEWS-SRS greater than or equal to 5, but nearly half of them did not have any infection or sepsis. Further, the higher mortality in patients with MEWS greater than or equal to 5 was not driven by sepsis or infection. These main findings were concurrent when using the original MEWS score, which does not include WBC count as a physiologic variable; although, as expected, less patients reached the threshold, and associations were slightly weaker overall.

Our findings are aligned with prior data indicating higher risk for inhospital mortality by EWSs. A study of critically ill patients showed prognostic value of the MEWS in predicting mortality (9). A smaller, homogenous cohort of 274 stroke patients utilizing a slightly different version of an EWS also found that the score predictive of mortality (10). Our data expand the applicability of the association of the MEWS-SRS and MEWS EWSs with inhospital mortality in the neuro-ICU population.

We discovered that using MEWS-SRS score in the neuro-ICU population had poor sensitivity for a screening test (as discussed in [11]) detecting infections and sepsis, which may lead to improper resource allocation with initiation or escalation of antibiotics. This illustrates that a warning score developed for one patient population is not generalizable for a different patient population. Furthermore, while shown beneficial in certain settings, the overall value of early warning systems is questionable. A Cochrane review of automated monitoring systems compared with standard care for the early detection in critical ill patients (12) reviewed three studies with 680, 442, and 77 participants. In the largest study, the median time to initiation of antimicrobial therapy had a difference of 2.2 hours (5.6 hr in the intervention group vs 7.8 hr in the control group [12]). The other two studies, however, showed no improvement in mortality rate and in one even a worsening of mortality in the intervention group. Hence, the effect from automated systems for monitoring sepsis remains unclear, with only low-quality evidence available and continued need for individual review of the patient’s condition by experienced healthcare staff (12).

Other factors to consider include the application of warning scores and impact on healthcare teams. Increased numbers of positive triggers often produce additional work for the primary nurse, rapid response team, and responding physician (13)—an effort that must be weighed against the benefit in increased detection of adverse outcomes. The balancing act of clinical trigger surveillance and personnel response can have positive and negative implications. “Alarm-fatigue” with failure to respond to alerts quickly is a well-documented risk in critical care, especially if the false positive rate is high (14–17). This principle also applies to the use of aggregate EWSs like the MEWS-SRS, where “risk alarm fatigue” can equally affect staff resource allocation, distort patient safety focus, and lead to delays in clinical response (13). Additionally, while our study focused primarily on patients in the neuro-ICU/NIMC and not those in the general neurologic floor setting, it is important to consider the potential impact EWS activation might have on the demand for neurologic ICU/IMC bed availability—recent evidence supports significant increases in the number of ICU admissions as a result of implementing EWS systems in acute care hospital setting (18). In our data from the neuro-ICU population, the mortality of patients identified with acute sepsis at the time of the MEWS-SRS elevation was slightly lower than the mortality rate in the overall MEWS-SRS greater than or equal to 5 cohort. The underlying trigger for MEWS-SRS (and MEWS) greater than or equal to 5 score elevation in one of three patients was neurologic worsening in patients where neurologic worsening that had been already recognized at the time of the score reaching the trigger-to-evaluate threshold. As our data show, altered mentation deviating from the base disturbance in the setting of neurocritical disease was the major physiologic variable, but derangement of vital sign functions also play an important role as neurologic deterioration may lead to CNS-induced changes of vital signs (19), directly affecting the MEWS-SRS/MEWS calculation. Mortality in the setting of neurologic worsening is often driven by adjustment of goals of care, a factor that we did not examine in this study; however, we postulate that this might be the reason why mortality in the overall MEWS-SRS greater than or equal to 5 group was higher than in the subset that eventually was diagnosed with sepsis. Our findings further pose the question whether a score such as MEWS-SRS/MEWS might be, possibly with additional modifications, able to more precisely and earlier predict neurologic worsening in the neuro-ICU population. Current developments in neuro-ICU bioinformatics with prediction analysis of vital signs would support this conceptual framework (20) but are often based on waveform analysis, limiting availability, and practicality. Our results demonstrated neurologic worsening as the most common clinical change associated with a MEWS-SRS score greater than or equal to 5. Given this finding, a future revised and targeted EWS could potentially improve patient outcomes not only in neurocritical care patients with primary or secondary neurologic injury. A next step toward evaluating the potential utility of the MEWS-SRS or MEWS warning scores to identify early neurologic changes and the related triggering of interventions is prospective evaluation of this score in the neurocritical ill setting with specific attention to neurologic deterioration and related work-up and management.

Strengths of our study include the evaluation of the MEWS-SRS in a large cohort of a defined subspecialty population of neurocritically ill, detailed and reliable charting of EWS, and granular analysis of clinical circumstances at the time of the EWS elevation. Limitations include the retrospective design and confinement to a single center. While we observed disproportionally more MEWS greater than or equal to 5 in the Black/African American patient group, this finding likely is routed in cohort imbalance but will necessitate further details in data analysis to evaluate whether this may be due to worse manifestation of neurologic disease and existing racial disparities (21). Another limitation of our study specifically when considering potential to generalize our findings includes the interdependence between EWS and EMR systems to yield study results. If healthcare institutions are lacking EMR systems or have vastly different methods of documentation, our findings may not be as generalizable in non-EMR settings. In addition, we were not able to obtain detailed neurologic data for the entire admission population including patients with MEWS-SRS/MEWS score less than 5. Furthermore, MEWS-SRS scoring lack the ability to detect subtle changes in the neurologic status. Adding more granular detail to subsequent early warning scoring systems may contribute to allow potential use of the score for detection of neurologic changes. Last, for our comparison group with MEWS-SRS/MEWS less than 5, we relied on coding-based sepsis diagnosis, and it is well described that automated coding may overestimate or underestimate the number of sepsis cases (22). Wide variation in validity of recording sepsis data was demonstrated in a review showing sensitivity ranging from 5.9% to 82.3%, with specificity 78.3% to 100%, and negative predictive values of 62.1% to 99.7% (23). Nonetheless, it is likely that a subset of neurocritically ill with sepsis was not captured by MEWS-SRS.


The MEWS and MEWS-SRS scores are associated with higher mortality in the neuro-ICU population but are not reliable indicators for new infection or sepsis. They preferentially triggered critical care team notification and evaluation in known high acuity patients with neurologic worsening. Refined screening tools may be an avenue for earlier identification of neurologic worsening, and different strategies are needed for early warning of infection-related clinical deterioration specific to the neuro-ICU population.


We thank Michele Lossius, MD, Chief Quality Officer, UF Health Shands, for critical feedback and evaluation of our article. We thank Sarah Gul, Alexander Rivera-Rosado, and Kendra Murphy for their help with data collection. We also thank our research students Noelle Taylor, Annanya Agarwal, Anum Keen, Hanaa Salman, and Priyanka Devaguptapu for refining the layout of tables and figures.


1. Duncan KD, McMullan C, Mills BM. Early warning systems: The next level of rapid response. Nursing. 2012; 42:38–44
2. Smith GB, Prytherch DR, Schmidt PE, et al. Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation. 2008; 77:170–179
3. Churpek MM, Edelson DP. Moving beyond single-parameter early warning scores for rapid response system activation. Crit Care Med. 2016; 44:2283–2285
4. Subbe CP, Kruger M, Rutherford P, et al. Validation of a modified Early Warning Score in medical admissions. QJM. 2001; 94:521–526
5. Croft CA, Moore FA, Efron PA, et al. Computer versus paper system for recognition and management of sepsis in surgical intensive care. J Trauma Acute Care Surg. 2014; 76:311–317
6. Moore LJ, Jones SL, Kreiner LA, et al. Validation of a screening tool for the early identification of sepsis. J Trauma. 2009; 66:1539–1546
7. Gatewood MO, Wemple M, Greco S, et al. A quality improvement project to improve early sepsis care in the emergency department. BMJ Qual Saf. 2015; 24:787–795
8. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016; 315:801–810
9. Reini K, Fredrikson M, Oscarsson A. The prognostic value of the Modified Early Warning Score in critically ill patients: A prospective, observational study. Eur J Anaesthesiol. 2012; 29:152–157
10. Liljehult J, Christensen T. Early warning score predicts acute mortality in stroke patients. Acta Neurol Scand. 2016; 133:261–267
11. Trevethan R. Sensitivity, specificity, and predictive values: Foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017; 5:307
12. Warttig S, Alderson P, Evans DJ, et al. Automated monitoring compared to standard care for the early detection of sepsis in critically ill patients. Cochrane Database Syst Rev. 2018; 6:CD012404
13. Jarvis S, Kovacs C, Briggs J, Meredith P, et al. Aggregate National Early Warning Score (NEWS) values are more important than high scores for a single vital signs parameter for discriminating the risk of adverse outcomes. Resuscitation. 2015; 87:75–80
14. Oliveira AEC, Machado AB, Santos EDD, et al. Alarm fatigue and the implications for patient safety. Rev Bras Enferm. 2018; 71:3035–3040
15. Ruppel H, Funk M, Whittemore R. Measurement of physiological monitor alarm accuracy and clinical relevance in intensive care units. Am J Crit Care. 2018; 27:11–21
16. Winters BD, Cvach MM, Bonafide CP, et al.; Society for Critical Care Medicine Alarm and Alert Fatigue Task Force. Technological distractions (part 2): A summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit Care Med. 2018; 46:130–137
17. Hravnak M, Pellathy T, Chen L, et al. A call to alarms: Current state and future directions in the battle against alarm fatigue. J Electrocardiol. 2018; 51:S44–S48
18. Smith ME, Chiovaro JC, O’Neil M, et al. Early warning system scores for clinical deterioration in hospitalized patients: A systematic review. Ann Am Thorac Soc. 2014; 11:1454–1465
19. Ikeda M, Matsunaga T, Irabu N, et al. Using vital signs to diagnose impaired consciousness: Cross sectional observational study. BMJ. 2002; 325:800
20. Melinosky C, Yang S, Hu P, et al. Continuous vital sign analysis to predict secondary neurological decline after traumatic brain injury. Front Neurol. 2018; 9:761
21. Elkind MSV, Lisabeth L, Howard VJ, et al. Approaches to studying determinants of racial-ethnic disparities in stroke and its sequelae. Stroke. 2020; 51:3406–3416
22. Fleischmann-Struzek C, Thomas-Rüddel DO, Schettler A, et al. Comparing the validity of different ICD coding abstraction strategies for sepsis case identification in German claims data. PLoS One. 2018; 13:e0198847
23. Jolley RJ, Sawka KJ, Yergens DW, et al. Validity of administrative data in recording sepsis: A systematic review. Crit Care. 2015; 19:139

early warning scores; neurocritical care; neurologic deterioration; outcome prediction; rapid response scores; sepsis; severity scores

Supplemental Digital Content

Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine.