Share this article on:

Predicting Sepsis Risk Using the “Sniffer” Algorithm in the Electronic Medical Record

Olenick, Evelyn M. DNP, RN, NEA-BC; Zimbro, Kathie S. PhD, RN; D'Lima, Gabrielle M. PhD; Ver Schneider, Patricia; Jones, Danielle MPH

doi: 10.1097/NCQ.0000000000000198

The Sepsis “Sniffer” Algorithm (SSA) has merit as a digital sepsis alert but should be considered an adjunct to versus an alternative for the Nurse Screening Tool (NST), given lower specificity and positive predictive value. The SSA reduced the risk of incorrectly categorizing patients at low risk for sepsis, detected sepsis high risk in half the time, and reduced redundant NST screens by 70% and manual screening hours by 64% to 72%. Preserving nurse hours expended on manual sepsis alerts may translate into time directed toward other patient priorities.

Supplemental Digital Content is Available in the Text.

Sentara CarePlex Hospital, Hampton, Virginia (Dr Olenick); Phoebe Putney Memorial Hospital, Albany, Georgia (Dr Olenick); Quality Research Institute, Clinical & Business Intelligence, Sentara Healthcare, Virginia Beach, Virginia (Dr Zimbro); and Sentara Healthcare, Virginia Beach, Virginia (Dr D'Lima and Mss Ver Schneider and Jones).

Correspondence: Kathie S. Zimbro, PhD, RN, Quality Research Institute, Clinical & Business Intelligence, Sentara Healthcare, 800 Independence Blvd, Virginia Beach, VA 23455 (

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journal's Web site (

Accepted for publication: April 4, 2016

Published ahead of print: June 6, 2016

SEPSIS, a significant and growing health care challenge, is defined as the systemic inflammatory response syndrome (SIRS) to infection and if left untreated may rapidly become life-threatening.1 Sepsis affects more than 1 million individuals in the United States annually, with estimated treatment costs at more than $20 billion.2,3 Often, nurses use manual surveillance methods to screen patients admitted to the hospital for sepsis risk, which may not be an efficient use of nurses' time. Manual surveillance is labor intensive and may be performed too late or not at all. Empirical evidence to support the use of digital sepsis alerts in the electronic medical record (EMR) to identify sepsis risk early, minimize manual surveillance, and improve patient outcomes is lacking. The Sepsis “Sniffer” Algorithm (SSA), designed as a digital sepsis alert, was examined at a multihospital health care system. This study explored whether the SSA was a viable alternative to the Nurse Screening Tool (NST), a manual sepsis alert.

Back to Top | Article Outline


Sepsis remains problematic as evidenced by increased hospital admissions, worsening mortality rates, extended length of stay (LOS), and amplified treatment costs. In a national sample examined between 2000 and 2008, the rate of hospitalizations with any sepsis diagnosis increased 70% (from 22.1/10 000 to 37.7/10 000).2 Further examination for this same sample from 2008 revealed that “only 2% of all hospitalized patients were diagnosed with septicemia or sepsis, and yet combined, these patients accounted for 17% of all in-hospital deaths.”2 (p4) Mortality rates for sepsis remain high despite the availability of evidence-based treatment guidelines.2 Reported “mortality rates range between 25% and 30% for patients with severe sepsis” (eg, sepsis plus either organ dysfunction or organ hypoperfusion) and “40% to 70% for patients in septic shock” (eg, severe sepsis plus hypotension).4 (p44)

Embedding digital sepsis alerts in the EMR “may alter the processes of care by improving decisions, adding prognostic information that may alter treatment decisions, and potentially improve outcomes by increasing more appropriate care or decreasing harmful or futile care.”5 (p812) Digital sepsis alerts “may trigger on patients who are not septic”6 (p129) but may require immediate medical care and “provide solutions to timely recognition, prompting alerts delivered in real time to nurses.”7 (p472) When used outside the intensive care unit, digital sepsis alerts result in early interventions but “no improvement in patient outcomes or length of stay.”7 (p473) “As more complicated diagnostic algorithms for sepsis emerge, increased benefit may be realized” for digital alerts.8 (p2100) The need for automation and decision support in the EMR becomes critical as the number of bedside personnel is being reduced, and “patient care continues to evolve using [digital alerts], sensors, and information sources related to patient conditions.”9 (p2031)

Back to Top | Article Outline

Study aims

Primary aims for this study explored (1) differences in predictive accuracy and time to first detection of high sepsis risk between the NST and SSA, and (2) the impact of the SSA on NST manual workload. The secondary aim explored relationships between NST sepsis risk detection timeliness and patient outcomes.

Back to Top | Article Outline


Sample and setting

This study used a descriptive retrospective design to examine study variables for all patients 18 years or older, admitted to the hospital and discharged during calendar year 2013. The study site was an integrated health care delivery system, comprising 12 hospitals, located in the southeastern United States. This study explored data from 7 of the hospitals that used Epic (version 2012; Epic Systems Corporation, Verona, Wisconsin; 2012). The other 5 hospitals used a variety of EMRs.

Back to Top | Article Outline

Study variables

Study variables for the primary aims included sepsis risk detection method (NST and SSA), risk classification (high or low risk), time to first detection of sepsis high risk, and manual NST screens with associated surveillance hours. For the secondary aim, patients were divided into 2 groups (sepsis high risk detected within or greater than 4 hours) to explore the effect of time until detection on patient outcomes. Only patients with a coded diagnosis of sepsis were included in these analyses, and NST risk data were used to categorize patients as at sepsis high risk. Patient outcomes included LOS (number of days in the hospital), direct costs (patient treatment costs), and mortality (in-hospital deaths).

Back to Top | Article Outline

Sepsis alert methods

According to hospital protocol, nurses manually collected NST data for all patients within 4 hours of hospital admission and every 12 hours thereafter, with results manually entered into the EMR in near-real time or subsequent to clinical priority. This continuous 12-hour screening schedule was intended to identify sepsis high risk early and facilitate recognition of changes in patients' clinical condition that may warrant action to prevent deleterious outcomes. Similar to other hospitals nationally, the NST was derived from the Surviving Sepsis Campaign's evidence-based criteria for the recognition of sepsis and produced an ordinal score based on increasing patient acuity with evidence of infection or SIRS.10 The NST was also used to screen patients who experienced clinical changes and/or required assistance from the rapid response team. The NST ordinal scores included the following: 0 indicating no evidence of infection or SIRS; 1 indicating either evidence of infection or SIRS; 2 indicating both evidence of infection and SIRS; and 3 indicating evidence of infection, SIRS, and organ failure. For this study, patients scoring a 2 or 3 were categorized at high risk for sepsis; otherwise, patients were categorized at low risk.

The SSA, based on predefined clinical criteria, was designed to achieve the following goals: (a) establish criteria with strong face validity based on clinical importance so that providers can easily interpret why an alert triggered; (b) accurately identify patients at high risk for sepsis; (c) achieve a high negative predictive value (ie, patients identified at low risk are very unlikely to develop sepsis); (d) improve the timeliness of sepsis detection; and (e) minimize manual work associated with the NST. The SSA electronically monitored available patient demographic and biophysical data to assign a sepsis risk status (low risk, high risk).

Back to Top | Article Outline

SSA impact on NST manual workload

NST manual workload calculations were based on the following premises: (a) the current hospital protocol required an initial NST screen within 4 hours of admission and every 12 hours thereafter; (b) the NST was performed on all patients with a LOS of 1 hour or greater, but only patients 18 years or older were included in this study; (c) low sepsis risk was validated by the absence of a sepsis diagnosis code on the billing record (ie, true-negative cases of sepsis); (d) the SSA-triggered NST screen was expected to occur within 15 minutes of each SSA alert; and (e) once the SSA alert triggered, the alert mechanism was suppressed for 24 hours to eliminate unnecessary subsequent alerts. NST screen duration was estimated at 3 and 5 minutes, dependent on nurse experience and comfort level with the tool. Screen duration and the number of NST screens were used to calculate the total hours required for NST sepsis surveillance (number of NST screens × estimated screen duration [3 or 5 minutes]/divided by 60 minutes).

A conservative and aggressive approach was used to explore the impact of the SSA on NST manual workload. The conservative approach maintained the initial NST screen for all patients within 4 hours of admission. Subsequent NST screens every 12 hours thereafter were eliminated unless the SSA triggered a high sepsis alert. The aggressive approach eliminated the initial NST screen for all patients within 4 hours of admission, performing the NST screens based on SSA-triggered alerts alone. For either approach, both initial and subsequent NST screens would be conducted on the basis of clinical presentation and nurse judgment (eg, physician notification, implementation of sepsis treatment bundles) regardless of whether the SSA triggered a high sepsis alert.

Back to Top | Article Outline


Following institutional review board approval, data were extracted from the EMR and billing record repositories for all records meeting inclusion criteria. Potential prognostic factors (eg, vital signs, laboratory reports) recorded at the time of admission, during the hospitalization, and ordered at the time of discharge were included. Time and date stamps associated with discrete data points were retained. The validity and reliability of the data were important for providers to render high-quality care. Therefore, the health care system's record review committee reviewed a percentage of the medical records monthly to ensure high-quality data standards were maintained.

Back to Top | Article Outline

Data analysis

Chi-square tests of independence, Wilcoxon signed ranks test, and Mann-Whitney U tests were used to explore relationships between study variables. Receiver operating characteristic curve statistics was used to determine the NST's and SSA's discriminatory ability to correctly categorize patients. The gold standard for sepsis risk measurement was whether sepsis was coded on the billing record. Comparing whether an alert triggered according to the patient's diagnosis of sepsis was an indication of the criterion validity of the SSA. All statistical analyses were conducted using SPSS (version 22; IBM Corp, Armonk, New York; 2013). Statistical testing was 2-sided, with a significance level of α set to .05 a priori.

Back to Top | Article Outline


Overall, the predictive accuracy for the NST proved to be higher than the SSA (Table 1). Both the NST (98.03) and SSA (97.23) demonstrated a high negative predictive value in that patients identified at low risk did not have sepsis coded on the final bill. The false-negative rate (1.5%) was similar between the NST and SSA, minimizing the risk of incorrectly categorizing patients at low risk for sepsis. However, specificity (true-negative rate) was significantly higher for the NST (80.3) than for the SSA (59.5). The NST improved the true-negative rate by 25%, increasing the number of patients accurately categorized at low risk for sepsis, reducing manual workload when used in conjunction with the conservative SSA approach. The NST demonstrated a stronger relationship with sepsis diagnosis coding. Sensitivity, or the probability of actual positives, was similar between the NST (84.1) and SSA (84.2). The true-positive rate (7.8%) was also similar, but the number of patients incorrectly categorized (false-positive rate) at high risk for sepsis doubled with the conservative SSA approach. Performing NST screens based on SSA-triggered alerts alone significantly reduced manual surveillance associated with false-positive results.

Table 1

Table 1

The SSA had a positive overall effect on the number of manual NST screens required (Table 2). The baseline NST totaled 68 652 initial screens, with 704 232 subsequent screens during the study period. The conservative SSA approach maintained initial NST screens (68 652) on admission for all patients, but subsequent NST screens were performed only with a SSA-triggered sepsis alert. This conservative change in screening practice reduced the number of NST screens (70.3%; 495 124). The aggressive SSA approach eliminated the initial NST screens (65 652) on admission. Like the conservative approach, subsequent NST screens were performed only with an SSA-triggered sepsis alert. This aggressive change in screening practice reduced the number of NST screens (69.3%; 488 105). In the conservative approach, maintaining the initial NST along with the SSA doubled the manual workload associated with the increased false-positive rate. This increased workload was minimized in the aggressive SSA approach when the NST was performed only with SSA-triggered alerts. Overall, the conservative SSA approach preserved 24 751 to 41 253 hours (3- to 5-minute NST completion time), resulting in a 64% reduction in total manual NST screening hours (see Supplemental Digital Content, Table, available at: The aggressive SSA approach preserved 27 837 to 46 400 hours, resulting in a 72% reduction in manual NST screening hours.

Table 2

Table 2

Embedding the SSA in the EMR may be beneficial in that sepsis high risk was detected in half the time compared with the NST, 2.4 and 5.9 hours, respectively (Z W = 25.75, P < .001). On average, the LOS for patients with sepsis high risk detected within 4 hours of admission was 1 day less (Z M = −6.41, P < .001), with average direct cost $1145 less (Z M = −5.70, P <.001). Differences in mortality between study groups were not statistically significant (χ2 = 1.714, P = .190).

Back to Top | Article Outline


Leveraging digital alert technology, such as the SSA, may identify sepsis risk earlier and reduce manual surveillance efforts, leading to more efficient distribution of existing nurse resources and improved patient outcomes. Changes to alert criteria, notification methods, or the population under surveillance should be explored. The economic impact to hospitals may be substantial through savings in reduced LOS and direct cost as a result of early sepsis risk detection using the SSA. Additional savings may be realized by reducing the manual work associated with sepsis risk screening. The intent is not to replace nurses' critical thinking with the SSA but to enhance this skill by synthesizing a patient's response to illness and prioritizing patient care needs more efficiently and timely. Consequently, nurse resources for manual surveillance can be directed toward a much smaller high-risk population.

Nurse leaders play a fundamental role “[ensuring] that appropriate staffing and other resources are in place to achieve safe care and optimal patient outcomes.”11 (p710) Advances in information technology have become increasingly important in the provision of care as an aid to documentation, clinical decision making, and improved nursing efficiency.12,13 In clinical practice, if the SSA determines the sepsis status as high risk, the EMR should prompt the nurse to critically assess the patient's condition with the NST before initiating appropriate treatment protocols.

This study is congruent with the Institute of Medicine's focus on technology as a research priority for transforming nursing.13 Identifying and testing new and existing digital alert technologies, such as the SSA, intended to support nurses' decision making and care delivery will be even more important as health care organizations provide cost-effective care through efficient workforce management. Specifically, preserving nurse hours expended on manual sepsis surveillance may translate into time directed toward other patient priorities. For this very reason, the SSA warrants further investigation.

Back to Top | Article Outline


Study limitations may include lack of data integrity due to any of the following: missing data due to lack of physician orders for diagnostic testing and lack of specimen collection; untimely diagnostic specimen collection; inaccurate, untimely, or missing NST documentation; and billing records coded incorrectly. The lack of documentation of a sepsis diagnosis did not necessarily indicate the absence of sepsis but, rather, the absence of sepsis coded on the final billing records. The same principle applied for untimely or absent nursing documentation. This study did not control for disease severity to distinguish patients already in organ failure versus impending organ dysfunction or severe sepsis. Controlling for disease severity limits the effect of the confounding variable (eg, organ failure) to test the significance of timely identification of sepsis risk on mortality outcomes and should be included in future studies.

Back to Top | Article Outline


The SSA has merit as a digital sepsis alert but should be considered an adjunct to versus an alternative for the NST, given the SSA demonstrated lower specificity and positive predictive value. Performing NST screens based on SSA-triggered alerts alone significantly reduced manual surveillance associated with false-positive results noted with the SSA conservative approach. The SSA design included all of the NST components but lacked nurse-patient interaction, removing that aspect of a nurses' clinical judgment in identifying subtle changes in patient condition. With refinement, the digital SSA may help nurses accurately identify patients at high risk for sepsis earlier in the hospital stay and minimize manual labor associated with NST screening practices. Early recognition, as a by-product of the nurses' documentation and patients' prognostic findings, may decrease hospital LOS and direct care costs associated with sepsis.

Back to Top | Article Outline


1. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–1655.
2. Hall MJ, Williams SN, DeFrances CJ, Golosinskiy A. Inpatient care for septicemia or sepsis: a challenge for patients and hospitals (Data Brief 62). Published 2011. Accessed March 2014.
3. Torio CM, Andrews RM. National inpatient hospital costs: the most expensive conditions by payer, 2011. Agency for Healthcare Research and Quality Healthcare Cost and Utilization Project (Statistical Brief 160). Published August 2013. Accessed January 2016.
4. Gauer RL. Early recognition and management of sepsis in adults: the first six hours. Am Fam Physician. 2013;88(1):44–53.
5. Silverstein MD. Assessing automated extraction of prognostic information for intensive care patients. Mayo Clin Proc. 2012;87(9):811–813.
6. Buck KM. Developing an early sepsis alert program. J Nurs Care Qual. 2014;29(2):124–132.
7. Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011;39(3):469–473.
8. Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med. 2012;40(7):2096–2101.
9. Salinas J, Chung KK, Mann EA, et al. Computerized decision support system improves fluid resuscitation following severe burns: an original study. Crit Care Med. 2011;39(9):2031–2038.
10. Dellinger PR, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580–637.
11. Wong CA, Cummings CG, Ducharme L. The relationship between nursing leadership and patient outcomes: a systemic review update. J Nurs Adm. 2013;21(5):709–724.
12. Hebda T, Czar P. Handbook of Informatics for Nurses & Healthcare Professionals. 5th ed. Upper Saddle River, NJ: Pearson Education; 2013.
13. Institute of Medicine. The Future of Nursing: Leading Change, Advancing Health. Washington, DC: National Academies Press; 2011.

digital alert; digital sepsis alert; electronic sepsis alert; nurse workload; sepsis

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved