Secondary Logo

Sepsis Surveillance Using Adult Sepsis Events Simplified eSOFA Criteria Versus Sepsis-3 Sequential Organ Failure Assessment Criteria*

Rhee, Chanu, MD, MPH1,2; Zhang, Zilu, MS1; Kadri, Sameer S., MD, MS3; Murphy, David J., MD, PhD4; Martin, Greg S., MD, MSc4; Overton, Elizabeth, MS4; Seymour, Christopher W., MD, MSc5; Angus, Derek C., MD, MPH5; Dantes, Raymund, MD, MPH6,7; Epstein, Lauren, MD, MS6; Fram, David, BA8; Schaaf, Richard, SM8; Wang, Rui, PhD1; Klompas, Michael, MD, MPH1,2 for the CDC Prevention Epicenters Program

doi: 10.1097/CCM.0000000000003521
Late Breaker Articles
Free
SDC
Editor's Choice

Objectives: Sepsis-3 defines organ dysfunction as an increase in the Sequential Organ Failure Assessment score by greater than or equal to 2 points. However, some Sequential Organ Failure Assessment score components are not routinely recorded in all hospitals’ electronic health record systems, limiting its utility for wide-scale sepsis surveillance. The Centers for Disease Control and Prevention recently released the Adult Sepsis Event surveillance definition that includes simplified organ dysfunction criteria optimized for electronic health records (eSOFA). We compared eSOFA versus Sequential Organ Failure Assessment with regard to sepsis prevalence, overlap, and outcomes.

Design: Retrospective cohort study.

Setting: One hundred eleven U.S. hospitals in the Cerner HealthFacts dataset.

Patients: Adults hospitalized in 2013-2015.

Interventions: None.

Measurements and Main Results: We identified clinical indicators of presumed infection (blood cultures and antibiotics) concurrent with either: 1) an increase in Sequential Organ Failure Assessment score by 2 or more points (Sepsis-3) or 2) 1 or more eSOFA criteria: vasopressor initiation, mechanical ventilation initiation, lactate greater than or equal to 2.0 mmol/L, doubling in creatinine, doubling in bilirubin to greater than or equal to 2.0 mg/dL, or greater than or equal to 50% decrease in platelet count to less than 100 cells/μL (Centers for Disease Control and Prevention Adult Sepsis Event). We compared area under the receiver operating characteristic curves for discriminating in-hospital mortality, adjusting for baseline characteristics. Of 942,360 patients in the cohort, 57,242 (6.1%) had sepsis by Sequential Organ Failure Assessment versus 41,618 (4.4%) by eSOFA. Agreement between sepsis by Sequential Organ Failure Assessment and eSOFA was good (Cronbach’s alpha 0.81). Baseline characteristics and infectious diagnoses were similar, but mortality was higher with eSOFA (17.1%) versus Sequential Organ Failure Assessment (14.4%; p < 0.001) as was discrimination for mortality (area under the receiver operating characteristic curve, 0.774 vs 0.759; p < 0.001). Comparisons were consistent across subgroups of age, infectious diagnoses, and comorbidities.

Conclusions: The Adult Sepsis Event’s eSOFA organ dysfunction criteria identify a smaller, more severely ill sepsis cohort compared with the Sequential Organ Failure Assessment score, but with good overlap and similar clinical characteristics. Adult Sepsis Events may facilitate wide-scale automated sepsis surveillance that tracks closely with the more complex Sepsis-3 criteria.

1Department of Population Medicine, Harvard Medical School/Harvard Pilgrim Health Care Institute, Boston MA.

2Department of Medicine, Brigham and Women’s Hospital, Boston, MA.

3Department of Critical Care Medicine, Clinical Center, National Institutes of Health, Bethesda, MD.

4Division of Pulmonary, Allergy, Critical Care and Sleep Medicine, Department of Medicine, Emory University School of Medicine, Emory Critical Care Center, Atlanta, GA.

5The Clinical Research, Investigation and Systems Modeling of Acute illness (CRISMA) Center, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA.

6Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention, Atlanta, GA.

7Division of Hospital Medicine, Department of Medicine, Emory University School of Medicine, Atlanta, GA.

8Commonwealth Informatics, Waltham, MA.

*See also p. 467.

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Centers for Disease Control and Prevention, the Agency for Healthcare Research and Quality, or the National Institutes of Health.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal).

Presented, in part, at the 2018 Society of Critical Care Medicine Conference (Abstract #1411) (San Antonio, TX, February 26, 2018).

Dr. Rhee’s institution received funding from Agency for Healthcare Research and Quality (K08HS025008). Dr. Kadri received support from National Institutes of Health (NIH) Intramural funds. Drs. Rhee, Martin, Overton, Wang, and Klompas’ institutions received funding from Centers for Disease Control and Prevention (CDC) (U54CK000484). Drs. Seymour and Angus received support for article research from the NIH (R35GM119519). Ms. Zhang and Mr. Fram disclosed work for hire. Drs. Kadri and Epstein disclosed government work. Dr. Martin’s institution received funding from and Cheetah Medical; he received funding from Grifols; and he received support for article research from the CDC. Mr. Fram and Mr. Schaaf’s institution received funding from Harvard Pilgrim Healthcare Institute. Dr. Klompas received honoraria for lectures from Washington State Hospital Association, Dell Medical School, and Beth Israel Deaconess Hospital Plymouth. The remaining authors have disclosed that they do not have any potential conflicts of interest.

For information regarding this article, E-mail: crhee@bwh.harvard.edu

The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) defined sepsis as “life-threatening organ dysfunction due to a dysregulated host response to infection” and proposed operationalizing the detection of organ dysfunction as an increase in the Sequential Organ Failure Assessment (SOFA) score by 2 or more points from baseline (1 , 2). These criteria were chosen based on their content and construct validity as well as analyses demonstrating that an increase in SOFA by 2 or more points was highly predictive of hospital mortality in patients with suspected infection (3).

Sepsis-3 criteria were primarily designed to facilitate clinical care. However, there is no true gold standard for sepsis, and different sepsis definitions may be better suited for other purposes, such as clinical research, quality improvement, or surveillance (4 , 5). The SOFA score poses particular challenges to conducting surveillance using electronic clinical data, as many SOFA components are not consistently measured or recorded in electronic health records (EHRs) in a structured format (i.e., vital signs, vasopressor doses, blood gases, FIO2, and urine output). Even when EHRs do include vital signs, they are prone to inconsistent quality and documentation as well as transient perturbations that can complicate analysis (6 , 7). The Glasgow Coma Scale (GCS) is also problematic because it is not measured in most patients, unreliable in intubated patients, and has low interrater reliability (8 , 9).

The US Centers for Disease Control and Prevention (CDC) recently released the “Adult Sepsis Eventsurveillance definition based on the Sepsis-3 framework of suspected infection associated with organ dysfunction (10). The Adult Sepsis Event definition was designed for objective retrospective surveillance and simplifies the SOFA score so that it can be implemented using routine clinical data available in most EHR systems (11). There are relatively few data, however, about how sepsis patients detected using Adult Sepsis Event’s organ dysfunction criteria—which we call here “eSOFA”—compare to those identified using the full SOFA score.

Our aim was to compare the prevalence, clinical characteristics, and outcomes of sepsis patients identified using eSOFA (CDC Adult Sepsis Event) versus the SOFA score (Sepsis-3) using EHR data from diverse hospitals. We hypothesized that CDC’s simpler eSOFA criteria would identify similar types of patients as the full SOFA score with comparable predictive validity for mortality, which could support its use as a more practical method of sepsis surveillance.

Back to Top | Article Outline

METHODS

Study Design and Data Sources

We conducted a retrospective cohort study of adult patients (≥ 20 yr old) admitted as inpatients to a sample of U.S. acute care hospitals. The primary dataset comprised all admitted patients (including critical care and noncritical care units) in calendar years 2013–2015 to one of 111 hospitals participating in the Cerner HealthFacts dataset, a deidentified database populated with granular clinical data from diverse academic and community hospitals that use the Cerner EHR system (11–18). Findings were validated in an independent dataset that included all inpatient adult encounters from 2013 to 2015 at four academic and community hospitals in the Emory Healthcare system in Georgia. Emory hospitals use the Cerner EHR but are not included in the HealthFacts dataset. This study was approved by the Institutional Review Board at Harvard Pilgrim Healthcare Institute and Emory University with a waiver of informed consent.

Back to Top | Article Outline

Electronic Implementation of SOFA, eSOFA, Infection, and Sepsis Criteria

The SOFA score defines organ dysfunction across six organ systems and assigns 0–4 points for each organ system depending on the degree of dysfunction, whereas eSOFA replaces these with binary criteria for most of the same organ systems (Table 1) (2 , 10 , 11). eSOFA does not include GCS given its aforementioned limitations for surveillance. However, it includes a criterion for lactate greater than or equal to 2.0 mmol/L given its high clinical face validity and central role in identifying and risk-stratifying sepsis (19). Cerner HealthFacts contains all components necessary to compute SOFA except vasopressor doses and urine output. Thus, we used the number of vasopressors administered rather than doses to assign SOFA points for cardiovascular dysfunction and only used creatinine levels to assign points for renal dysfunction. We also used arterial oxygen saturation (SaO2)/FIO2 ratios for the respiratory SOFA component using previously validated conversion criteria if PaO2/FIO2 was missing (20). FIO2 values for nonintubated patients receiving supplemental oxygen were estimated assuming each 1 L/min of oxygen flow rate increased FIO2 by 4% over room air. Missing values for SOFA score components, including GCS, were handled by carrying over values from adjacent calendar days; if unavailable, 0 SOFA points were assigned to that category. Additional details on SOFA score implementation are described in the eMethods (Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

TABLE 1

TABLE 1

CDC’s Adult Sepsis Event definition includes standardized criteria for “presumed serious infection” and concurrent organ dysfunction (eSOFA). Sepsis-3 criteria, on the other hand, do not clearly stipulate how to identify patients with infection (3). We therefore used CDC’s presumed serious infection criteria for comparisons of eSOFA and SOFA. The CDC presumed serious infection criteria require a blood culture draw and new antibiotics started within ± 2 days of the blood culture and continued for at least 4 consecutive days; less than 4 days are permitted if the patient died, was discharged to hospice, or transferred to another hospital less than 4 days after antibiotics were started, and antibiotics were continued until the day or day prior to death, hospice discharge, or transfer (10 , 11). We defined a hospital encounter as having sepsis if there was presumed serious infection and organ dysfunction using either a rise in SOFA score by 2 or more points (Sepsis-3) or the presence of 1 or more eSOFA criteria (Adult Sepsis Event) within ± 2 days of the blood culture day (additional details in eMethods, Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

In the Emory dataset, maximum daily vasopressor doses were available, and daily SOFA scores were calculated independently according to internal data specifications. Only PaO2/FIO2 ratios, and not SaO2/FIO2 ratios, were used to calculate respiratory SOFA scores. Presumed infection, baseline SOFA scores, Sepsis-3, and Adult Sepsis Events were otherwise implemented using the same approach as in the Cerner dataset.

Back to Top | Article Outline

Comparison of Sepsis Patients Defined by SOFA and eSOFA

We examined the prevalence, characteristics, and outcomes of sepsis using either SOFA or eSOFA. We identified their likely infection syndromes using International Classification of Diseases, 9th Revision, Clinical Modification codes, adapting prior methodology (eTable 1, Supplemental Digital Content 1, http://links.lww.com/CCM/E164) (21 , 22). Crude mortality rates were compared using two-sample z-tests. The agreement between sepsis patients identified by SOFA and eSOFA was calculated using Cronbach’s alpha, a commonly used measure of reliability (3 , 23 , 24). The sensitivity and positive predictive value (PPV) of sepsis defined by eSOFA (Adult Sepsis Event) were compared with sepsis defined using the SOFA score (Sepsis-3) amongst patients with presumed infection. In order to examine the consistency of the relationship of eSOFA versus SOFA across important patient subgroups, these analyses were repeated in subgroups stratified by age, infectious syndromes, and comorbidities.

We then evaluated and compared discrimination for in-hospital mortality using eSOFA versus SOFA in patients with presumed infection, using the same methodology as the Sepsis-3 analyses (3). Specifically, we created a baseline model for the outcome of in-hospital death based on age, sex, race, and a composite comorbidity score (Elixhauser method [25]) among all patients with presumed infection. We divided encounters into deciles of baseline risk of in-hospital death. Within each decile, we compared mortality rates among patients with and without an increase in SOFA by 2 or more points, and among patients with and without 1 or more eSOFA criteria. We assessed model discrimination with area under the receiver operating characteristic (AUROC) curves for in-hospital death using SOFA and eSOFA alone when each set of criteria were added to the baseline risk model.

We performed two sensitivity analyses. First, we examined the performance of eSOFA criteria without lactate, since it is unclear whether lactate adds additional prognostic information above and beyond other organ dysfunction criteria (3). Second, we used a less stringent definition of presumed infection that allowed for any clinical culture—rather than blood cultures alone—to establish the infection window.

All analyses were conducted using SAS Version 9.3 (SAS Institute, Cary, NC). All tests of significance used two-sided p values at less than or equal to 0.05.

Back to Top | Article Outline

RESULTS

Sepsis Prevalence, Characteristics, Mortality, and Overlap of eSOFA and SOFA

The primary cohort from Cerner HealthFacts included 942,360 adult hospital encounters from 2013 to 2015. Of these, 104,903 (11.1%) met presumed infection criteria, 57,242 (6.1%) met Sepsis-3 criteria (presumed infection with concurrent increase in the SOFA score by ≥ 2 points), 41,618 (4.4%) met CDC Adult Sepsis Event criteria (presumed infection with ≥ 1 concurrent eSOFA criteria), and 34,174 (3.6%) met both Sepsis-3 and Adult Sepsis Event criteria (eFig. 1, Supplemental Digital Content 1, http://links.lww.com/CCM/E164). The frequency of missing variables for SOFA and eSOFA is shown in eTable 2 (Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

Demographics, comorbidities, and clinical characteristics of both sets of sepsis patients were broadly similar (Table 2). Pneumonia and urinary tract infections were the most common infectious diagnoses associated with both SOFA and eSOFA sepsis patients. Respiratory and neurologic dysfunction were the most common organ dysfunctions flagged by the SOFA score, whereas elevated lactate and doubling in baseline creatinine were the most common eSOFA-flagged organ dysfunctions (eFigs. 2 and 3, Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

TABLE 2

TABLE 2

In-hospital mortality rates in sepsis patients were higher with eSOFA versus SOFA criteria (17.1% vs 14.4%; p < 0.001). Mortality for patients with presumed infection without concurrent eSOFA was 3.4% (2,137 of 61,148 patients) versus 2.2% (1,047 of 46,614 patients) for patients without concurrent increases in SOFA by greater than or equal to 2 points (p < 0.001).

Patients with presumed infection who met SOFA but not eSOFA criteria (22.0% of the presumed infection cohort) tended to have mild hypoxemia that did not require mechanical ventilation or abnormal GCS scores. Conversely, patients who met eSOFA criteria but not SOFA (7.1% of the presumed infection cohort) most commonly had elevated lactate or doubling in serum creatinine (eFigs. 4 and 5, Supplemental Digital Content 1, http://links.lww.com/CCM/E164). The mortality in both these groups was relatively low (4.9% for presumed infection patients who were SOFA+/eSOFA− and 6.3% for those who were eSOFA+/SOFA−) (Fig. 1). In contrast, mortality was highest (19.8%) in the patients with presumed infection who met both SOFA and eSOFA criteria (32.6% of the presumed infection cohort) (Fig. 1). These patients tended to have elevated lactate levels and high rates of cardiovascular, renal, and respiratory dysfunction by both eSOFA and SOFA criteria (eFig. 6, Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

Figure 1

Figure 1

Agreement between SOFA/Sepsis-3 and eSOFA/Adult Sepsis Events was good (Cronbach’s alpha 0.81). Relative to Sepsis-3, the sensitivity of Adult Sepsis Event criteria was 59.7%, and PPV was 82.1%. The relative differences in prevalence and mortality, and the agreement between Sepsis-3 and Adult Sepsis Events, was generally consistent across patient subgroups stratified by age, infectious diagnoses, and comorbidity burden (Table 3).

TABLE 3

TABLE 3

Back to Top | Article Outline

Prognostic Accuracy for In-Hospital Death

Amongst patients with presumed infection, encounters with sepsis based on greater than or equal to 2 SOFA points versus greater than or equal to 1 eSOFA criteria had similar increases in odds of mortality across baseline risk deciles (Fig. 2). Overall discrimination for in-hospital mortality on top of the baseline risk model was slightly higher for eSOFA (AUROC, 0.774; 95% CI, 0.770–0.779) versus SOFA (AUROC, 0.759; 95% CI, 0.751–0.764; p < 0.001).

Figure 2

Figure 2

Back to Top | Article Outline

Sensitivity Analyses

Removing lactate from eSOFA criteria decreased Adult Sepsis Event prevalence from 4.4% to 3.5%, increased mortality from 17.1% to 19.0%, and made no difference in the AUROC for in-hospital mortality (0.773; 95% CI, 0.769–0.778). The sensitivity of eSOFA without lactate for SOFA/Sepsis-3 was lower than eSOFA with lactate (51.8% vs 59.7%), but PPV was higher (89.2% vs 82.1%). When expanding presumed infection criteria to include any clinical culture rather than blood cultures alone, findings were similar, with an AUROC for in-hospital mortality of 0.780 (95% CI, 0.776–0.784) for eSOFA (with the lactate criterion) versus 0.768 for SOFA (95% CI, 0.764–0.772; p < 0.001).

Back to Top | Article Outline

Findings in Independent Dataset

In the four-hospital Emory dataset, sepsis prevalence was slightly higher with eSOFA than SOFA (6.5% vs 6.1%) but other findings were similar, with eSOFA having slightly higher mortality (11.5% vs 10.8%) and better discrimination for in-hospital death amongst patients with presumed infection (AUROC, 0.755; 95% CI, 0.745–0.766) versus SOFA (AUROC, 0.717; 95% CI, 0.705–0.728; p < 0.001) (eTable 3, Supplemental Digital Content 1, http://links.lww.com/CCM/E164).

Back to Top | Article Outline

DISCUSSION

The SOFA score is a clinically rich and well-tested measure of organ dysfunction, but a simpler version that relies on more readily available data could facilitate automated and consistent sepsis surveillance in more hospitals with variable EHR systems. In this large cohort, we found that the eSOFA criteria in CDC’s Adult Sepsis Event definition have good overlap with the SOFA score used by Sepsis-3 and identify similar sepsis patients, with slightly lower prevalence and higher mortality. The relationship of eSOFA to SOFA was consistent across major age categories, infectious diagnoses, and comorbidity burden, indicating that tracking Adult Sepsis Events can provide reliable information on sepsis rates as identified by Sepsis-3 criteria across hospitals with different patient populations.

Two prior analyses in single healthcare systems also found good overlap between Sepsis-3 and an earlier version of CDC surveillance criteria (5 , 24). Our study expands on these findings using data from a larger set of hospitals and provides further insight into the relationship between these two definitions. We found good PPV (82%) but moderate sensitivity (60%) of Adult Sepsis Event detection relative to Sepsis-3, mainly due to the possibility of reaching 2 SOFA points from hypoxemia without mechanical ventilation and from abnormal GCS scores without any corresponding eSOFA criteria. Broadening Adult Sepsis Event surveillance to include hypoxemia beyond mechanical ventilation and mental status is conceptually attractive but operationally difficult given variability in measurement and data quality for arterial blood gases, SaO2 and FIO2 levels, and GCS (8 , 9).

Our findings also demonstrate several nuances of conducting sepsis surveillance using lactate levels. Including lactate in eSOFA increased eSOFA’s sensitivity for identifying patients with Sepsis-3 criteria, likely because the lactate criterion identifies some patients with mild hypotension, hypoxemia, or abnormal mental status that are sufficient to increase SOFA but insufficient to trigger eSOFA criteria. However, removing lactate also increased eSOFA’s PPV as many patients flagged by lactate alone had no organ dysfunction by SOFA. Furthermore, removing lactate from eSOFA resulted in a higher overall mortality and did not change the AUROC for discriminating for in-hospital death. This suggests elevated lactate levels alone without concurrent organ dysfunction have little impact on the risk of mortality.

Since there is no gold standard for sepsis (4 , 26), investigators have compared potential sepsis criteria based on their predictive validity for mortality in patients with suspected infection (3 , 27). In our study, eSOFA criteria had comparable or better discrimination for mortality compared with an increase in SOFA by greater than or equal to 2 points. These findings were consistent in the independent dataset and when expanding the definition of presumed infection to include all clinical cultures rather than blood cultures alone (28). Although eSOFA is not meant to facilitate early sepsis recognition or bedside management, this strong association with mortality gives further credibility to these criteria as a means of tracking sepsis and to the importance of conducting surveillance for Adult Sepsis Events.

Given the complexity of sepsis, no single set of criteria can adequately serve all the needs of various stakeholders (4 , 5). For example, quick SOFA and systemic inflammatory response syndrome criteria were designed as bedside prompts for early sepsis recognition, whereas the Sepsis-3 SOFA criteria are well-suited for clinically characterizing septic patients as well as prospective and clinical trials (1 , 5). In contrast, CDC’s Adult Sepsis Event eSOFA criteria are optimized for retrospective surveillance since they prioritize consistent monitoring of sepsis prevalence and outcomes using EHR data. EHR data overcome some of the biases inherent in conducting surveillance using administrative data, especially variable and changing diagnosis and coding practices overtime (26 , 29–31).

This study has several limitations. First, identifying sepsis using SOFA requires assumptions about patients’ baseline SOFA scores and the time period surrounding organ dysfunction and infection. However, we chose clinically reasonable time windows based on prior work validated by medical record reviews (11 , 29 , 32). The AUROC values we found for SOFA were also similar to those reported in previous studies, including the Sepsis-3 derivation work (3 , 27). Second, there may be idiosyncrasies in the Cerner dataset and our SOFA score implementation that limit the generalizability of our findings, particularly as we were unable to perfectly replicate the SOFA score due to the absence of vasopressor doses. However, the Emory cohort did contain these data and generated similar results for mortality and AUROC using SOFA versus eSOFA. Prevalence estimates of eSOFA versus SOFA differed slightly in the Emory cohort; this likely reflects differences in populations as well as the use of only PaO2/FIO2 values (and not SaO2/FIO2) in Emory to assign respiratory SOFA points. Third, our comparisons of infectious syndromes in Adult Sepsis Events versus Sepsis-3 cases may also be limited by diagnostic errors and coding inaccuracies (33). Last, our reference for eSOFA comparisons was the SOFA score, but the Sepsis-3 definition has not yet been universally embraced (34 , 35) and several other scores also accurately predict death in patients with possible infections (36–38).

In conclusion, we found that CDC’s Adult Sepsis Event eSOFA organ dysfunction criteria identify a smaller, more severely ill cohort of sepsis patients compared with those identified using the more complex SOFA score. There is substantial overlap, however, between patients identified by the two definitions. These findings support the use of Adult Sepsis Events as a practical tool that hospitals and public health agencies can consider using for consistent and automatable sepsis surveillance.

Back to Top | Article Outline

REFERENCES

1. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016; 315:801–810.
2. Vincent JL, Moreno R, Takala J, et al; On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med 1996; 22:707–710.
3. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: For the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016; 315:762–774.
4. Angus DC, Seymour CW, Coopersmith CM, et al. A framework for the development and interpretation of different sepsis definitions and clinical criteria. Crit Care Med 2016; 44:e113–e121.
5. Seymour CW, Coopersmith CM, Deutschman CS, et al. Application of a framework to assess the usefulness of alternative sepsis criteria. Crit Care Med 2016; 44:e122–e130.
6. Stevenson JE, Israelsson J, Nilsson GC, et al. Recording signs of deterioration in acute patients: The documentation of vital signs within electronic health records in patients who suffered in-hospital cardiac arrest. Health Informatics J 2016; 22:21–33.
7. Stevenson JE, Israelsson J, Petersson G, et al. Factors influencing the quality of vital sign data in electronic health records: A qualitative study. J Clin Nurs 2018; 27:1276–1286.
8. Gill MR, Reiley DG, Green SM. Interrater reliability of Glasgow Coma Scale scores in the emergency department. Ann Emerg Med 2004; 43:215–223.
9. Bledsoe BE, Casey MJ, Feldman J, et al. Glasgow Coma Scale scoring is often inaccurate. Prehosp Disaster Med 2015; 30:46–53.
10. Centers for Disease Control and Prevention: Hospital Toolkit for Adult Sepsis Surveillance. 2018. Available at: https://www.cdc.gov/sepsis/pdfs/Sepsis-Surveillance-Toolkit-Mar-2018_508.pdf. Accessed September 5, 2018.
11. Rhee C, Dantes R, Epstein L, et al. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA 2017; 318:1241–1249.
12. Choudhry SA, Li J, Davis D, et al. A public-private partnership develops and externally validates a 30-day hospital readmission risk prediction model. Online J Public Health Inform 2013; 5:219.
13. Goyal A, Spertus JA, Gosch K, et al. Serum potassium levels and mortality in acute myocardial infarction. JAMA 2012; 307:157–164.
14. Grodzinsky A, Goyal A, Gosch K, et al. Prevalence and prognosis of hyperkalemia in patients with acute myocardial infarction. Am J Med 2016; 129:858–865.
15. Lagu T, Pekow PS, Shieh MS, et al. Validation and comparison of seven mortality prediction models for hospitalized patients with acute decompensated heart failure. Circ Heart Fail 2016; 9.
16. Chan WW, Waltman Johnson K, Friedman HS, et al. Association between cardiac, renal, and hepatic biomarkers and outcomes in patients with acute heart failure. Hosp Pract (1995) 2016; 44:138–145.
17. Andes D, Azie N, Yang H, et al. Drug-drug interaction associated with mold-active triazoles among hospitalized patients. Antimicrob Agents Chemother 2016; 60:3398–3406.
18. Petrick JL, Nguyen T, Cook MB. Temporal trends of esophageal disorders by age in the Cerner Health Facts database. Ann Epidemiol 2016; 26:151–154.e4.
19. Rhodes A, Evans LE, Alhazzani W, et al. Surviving sepsis campaign: International guidelines for management of sepsis and septic shock: 2016. Crit Care Med 2017; 45:486–552.
20. Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med 2009; 37:1649–1654.
21. Christensen KL, Holman RC, Steiner CA, et al. Infectious disease hospitalizations in the United States. Clin Infect Dis 2009; 49:1025–1035.
22. Rhee C, Gohil S, Klompas M. Regulatory mandates for sepsis care–reasons for caution. N Engl J Med 2014; 370:1673–1676.
23. Eisinga R, Grotenhuis Mt, Pelzer B. The reliability of a two-item scale: Pearson, cronbach, or spearman-brown? Int J Public Health 2013; 58:637–642.
24. Johnson AEW, Aboab J, Raffa JD, et al. A comparative analysis of sepsis identification methods in an electronic database. Crit Care Med 2018; 46:494–499.
25. van Walraven C, Austin PC, Jennings A, et al. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care 2009; 47:626–633.
26. Rhee C, Kadri SS, Danner RL, et al. Diagnosing sepsis is subjective and highly variable: A survey of intensivists using case vignettes. Crit Care 2016; 20:89.
27. Raith EP, Udy AA, Bailey M, et al; Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes and Resource Evaluation (CORE): Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA 2017; 317:290–300.
28. Churpek MM, Snyder A, Sokol S, et al. Investigating the impact of different suspicion of infection criteria on the accuracy of quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores. Crit Care Med 2017; 45:1805–1812.
29. Rhee C, Kadri S, Huang SS, et al. Objective sepsis surveillance using electronic clinical data. Infect Control Hosp Epidemiol 2016; 37:163–171.
30. Rhee C, Murphy MV, Li L, et al; Centers for Disease Control and Prevention Epicenters Program: Improving documentation and coding for acute organ dysfunction biases estimates of changing sepsis severity and burden: A retrospective study. Crit Care 2015; 19:338.
31. Klompas M, Rhee C. We need better tools for sepsis surveillance. Crit Care Med 2016; 44:1441–1442.
32. Kadri SS, Rhee C, Strich JR, et al. Estimating ten-year trends in septic shock incidence and mortality in United States academic medical centers using clinical data. Chest 2017; 151:278–285.
33. Rothberg MB, Pekow PS, Priya A, et al. Variation in diagnostic coding of patients with pneumonia and its association with hospital risk-standardized mortality rates: A cross-sectional analysis. Ann Intern Med 2014; 160:380–388.
34. Simpson SQ. New sepsis criteria: A change we should not make. Chest 2016; 149:1117–1118.
35. Townsend SR, Rivers E, Tefera L. Definitions for sepsis and septic shock. JAMA 2016; 316:457–458.
36. Le Gall JR, Klar J, Lemeshow S, et al. The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. JAMA 1996; 276:802–810.
37. Churpek MM, Snyder A, Han X, et al. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med 2017; 195:906–911.
38. Macdonald SP, Arendts G, Fatovich DM, et al. Comparison of PIRO, SOFA, and MEDS scores for predicting mortality in emergency department patients with severe sepsis and septic shock. Acad Emerg Med 2014; 21:1257–1263.
Keywords:

Adult Sepsis Event; electronic health records; organ dysfunction; sepsis; Sequential Organ Failure Assessment score; surveillance

Supplemental Digital Content

Back to Top | Article Outline
Copyright © by 2019 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.