Share this article on:

Special Report: The Sick and the Not-Sick

Spiegel, Rory MD

doi: 10.1097/01.EEM.0000515673.53563.78
Special Report

Dr. Spiegelis a clinical instructor in emergency medicine at the University of Maryland Medical Center. Visit his blog athttp://emnerd.com, where this article first appeared, follow him on Twitter @emnerd_, and read his past articles athttp://bit.ly/EMN-MythsinEM.

qSOFA and SIRS can't compare with the clinical judgment of experienced emergency physicians. EM may never have a tool to detect all patients at risk of decompensation.

sepsis, qSOFA

sepsis, qSOFA

Figure

Figure

A number of proud South Carolinians woke up Jan. 1 to find themselves no longer living in their former state of residence, as did some residents of North Carolina. Without moving from the beds in which they slept, they were all magically transported into the neighboring state. The two states had disputed their common border for more than 20 years, and finally agreed to move the state line approximately 12 miles north of its previously designated coordinates. This seems like amusing trivia to most of us, but the consequences likely appeared more serious to the proud South Carolinians it directly affected. (State of North Carolina, Executive Order No. 118, Dec. 5, 2016; http://bit.ly/2mpwK5c.)

The same sort of shifting in geographic territories happened in the world of sepsis a year ago in February when Singer, et al., published the new Sepsis-3 definitions. (JAMA 2016;315[8]:801.) Not only was this a change in the current thinking of the underlying mechanisms causing this deadly presentation, but it was also one in the clinical tools suggested to identify high-risk patients.

The authors defined sepsis as a “life-threatening organ dysfunction caused by a dysregulated host response to infection.” This definition is likely a better representation of the physiologic underpinnings of the disease, but it does little to change the day-to-day logistical management of the patients who present to the ED.

The authors go on to suggest a change in how we identify sepsis and septic shock in the clinical arena. Sepsis is defined as an infection associated with an increase in the Sequential Organ Failure Assessment (SOFA) score of 2 points or more and septic shock as fluid-resistant shock requiring vasopressor supplementation to maintain a mean arterial pressure of 65 mm Hg or greater and serum lactate level greater than 2 mmol/L.

More importantly for the emergency department, the authors developed a novel clinical score to be used outside the ICU to identify patients with sepsis or septic shock. Calling it the quick SOFA (qSOFA), this score consisted of a respiratory rate greater than 22 bpm, altered mental status, and hypotension (systolic blood pressure less than 100 mm Hg). The original manuscript by Seymour, et al., deriving and validating this score noted its moderate predictive value outside the ICU. (JAMA 2016;315[8]:762.)

Using University of Pittsburgh School of Medicine records, these authors retrospectively identified patients who received antibiotics and had body fluid cultures drawn during their stay in the hospital. The authors required these two events to occur within 24 hours of each other. They then derived and validated the qSOFA using the traditional SIRS criteria and the more data-laden SOFA score as comparators.

Back to Top | Article Outline

Validating qSOFA

The qSOFA performed admirably when utilized in a population outside of the ICU, with an area under the curve (AUC) of 0.81. This was compared with 0.76 and 0.79 for SIRS and SOFA, respectively. The qSOFA's performance remained consistently acceptable when validated in multiple external data sets (0.71, 0.75, and 0.78, respectively). A great deal has been written about the utility of this score, essentially stating that it is difficult to determine how it will perform in the ED when used prospectively, given the retrospective origin of its derivation and validation.

A number of retrospective studies have attempted to validate qSOFA's performance and have found variable diagnostic characteristics, but like their predecessor, they suffer from the same retrospective, big data dredge methodology that leads to uncertainty regarding the scores stability. (Am J Respir Crit Care Med 2016 Sept 20 [Epub ahead of print]; Ann Emerg Med 2017 Jan 19 [Epub ahead of print].)

Published in JAMA almost a year after Seymour, et al., unveiled this controversial tool, Freund, et al., reported the first prospective analysis of the qSOFA score. (JAMA 2017;317[3]:301.) These authors examined patients with suspected infection presenting to 30 EDs across Europe. Patients were included in the final analysis only if two “experts” reviewing all the pertinent data from the patients' hospital course concluded the hospitalization had an infectious cause.

The three components of the qSOFA score were collected prospectively by the treating emergency physician. These values were not collected exclusively upon initial presentation, but utilized the patient's worst score during his stay in the ED. Unlike the previous retrospective evaluations, the presence of altered mental status was recorded separately from the patient's GCS. The components of the SIRS criteria and SOFA score were also collected during their ED stay.

The authors sought to compare a qSOFA score of 2 or greater, an increase in the SOFA score by two points, two or more SIRS criteria, and severe sepsis, which was defined as two or more SIRS criteria and a lactate greater than 2 mmol/L. They examined the diagnostic accuracy of each of these decision tools to identify patients who would die during their hospital course. They also examined admission to the ICU, length of stay in the ICU for more than 72 hours, and a composite of in-hospital mortality and 72-hour ICU admission.

A total of 879 of the 1088 patients screened were included in the final analysis. At a glance, the qSOFA performed admirably. The highest AUC for predicting in-hospital mortality was achieved by the qSOFA at 0.80. This was followed by SOFA (0.77), SIRS (0.65), and severe sepsis (0.65). The authors reported that the findings of the multiple secondary endpoints they measured closely resembled the results of their primary analysis. Freund, et al., in a sense validated the original derivation and validation cohort of Seymour, et al. But, of course, the reality is always far more complex.

Back to Top | Article Outline

Missing 30 Percent

qSOFA may have outperformed SIRS, SOFA and severe sepsis, but none of these scores performed that well. qSOFA, which had the best diagnostic test characteristics of the scores examined, only had a sensitivity of 70% and a specificity of 79%. I am not sure any of us would be comfortable missing 30 percent of the patients who died from sepsis.

None of these scores should be utilized in any typical diagnostic fashion. After all, our goal in the ED should not be to predict who will perish during a hospital course, but to identify critically ill patients who will benefit from the aggressive resuscitative measures we have come to associate with managing sepsis and septic shock. This cohort is far more nebulous, but examining each of these tools' prognostic abilities gives us a better idea of their respective performances.

The overall mortality in the Freund, et al., cohort was reported to be eight percent while the in-hospital mortality was only three percent for patients with a qSOFA score less than 2. Conversely, the in-hospital mortality was 24 percent in patients who had a qSOFA of 2 or greater. In contrast, the in-hospital mortality for patients with less than two SIRS criteria was fairly similar to those in the low-risk qSOFA group (2.2%), but the in-hospital mortality of patients with two or more SIRS criteria did not differ significantly from the overall mortality of the entire cohort (8% vs 10.6%).

This suggests that the SIRS criteria can identify a cohort at low risk of death during their hospital course, but it does so in such a small minority that it is essentially clinically useless. In fact, 653 (74%) of the 879 patients in the final analysis had two or more SIRS criteria. The qSOFA score, in contrast, only included 218 (25%) of the patients in its high-risk cohort. This analysis also likely overestimates the SIRS criteria's poor specificity. They only examined the patients who were admitted to the hospital for suspected infection, and excluded all patients with a benign infection that commonly present to the ED with two or more SIRS criteria and are discharged home.

But another issue is even more important. Let's look at the components of the qSOFA score: altered mental status, hypotension (defined as a systolic blood pressure less than 100 mm Hg), and an increased respiratory rate (greater than 22 bpm). A patient, to achieve a score of 2 or higher, has to be hypotensive and altered, hypotensive and in respiratory distress, or altered and in respiratory distress. None of these patients is clinically subtle.

qSOFA is not a marker of sepsis; it is a marker for the severity of illness. A recent retrospective analysis by Singer, et al., demonstrated the qSOFA score performed as well or poorly in patients with and without infectious causes of their presentation. (Ann Emerg Med 2017 Jan 19 [Epub ahead of print].) If we are to believe the results presented by Freund, et al., then for the most part, sick patients look sick. Some patients who appear well in the ED will decline during their hospital course, but the majority of patients presenting to the ED with sepsis do so in a clinically obvious fashion.

Back to Top | Article Outline

Hiding a Secret

So the next question becomes, is there a method to identify the patients who are secretly harboring a critical illness and will go unnoticed in the ED? The first solution is to utilize a tool like the SIRS criteria that has a far higher sensitivity than the qSOFA. It achieves this superior sensitivity, however, at the cost of including the majority of patients presenting to the ED with signs of infection. Adding more hay to the stack does not make it easier to identify the few needles that would otherwise go unnoticed.

Lactate has been suggested as a tool to identify this clinically occult subset of patients, but the original data set from Seymour, et al., and the recent validation set from Freund, et al., would suggest that the addition of lactate adds nothing to the risk stratification that the qSOFA score already provides. Seymour, et al., found that adding lactate at a threshold of 2, 3, or 4 mmol/L to the qSOFA score did not augment its diagnostic prowess. When Freund, et al., added lactate to the qSOFA score, it did not change its prognostic value either.

These findings actually should not come as a surprise. The original data on lactate supports its use in identifying cryptic shock, or tissue hypoxia, without outward signs of shock. Howell, et al., in their seminal 2007 paper, conducted a prospective observation trial that enrolled all patients who presented with a suspected infection to the ED. (Intensive Care Med 2007;33(11):1892.) The authors obtained serum lactate levels in all patients and examined its ability to predict in-hospital mortality. They found patients with increased serum lactate values died more frequently than patients with numerically normal values.

Serum lactate still maintained its predictive capability even when the authors controlled for hypotension. Seventy-three (5.7%) of the 1287 patients enrolled died during their hospital course. Patients who were not hypotensive or did not have an elevated lactate (> 4 mmol/L) had an in-hospital mortality of 2.5 percent, but mortality was 28.3 percent in patients who were hypotensive or had an elevated lactate.

The predictive values of qSOFA and lactate appear fairly similar when you compare the Howell, et al., and Freund, et al., cohorts side by side. Puskarich, et al., in their reanalysis of the Jones trial found fairly similar results to Howell's original assessment of lactate's predictive abilities. They found that a serum lactate was an equally good predictor of mortality as hypotension. The respective in-hospital mortality rates in these cohorts were 21 percent and 19 percent. (Resuscitation 2011;82[10]:1289.)

Clinically occult shock (a patient with a normal BP) should not be misinterpreted as a clinically occult patient (a well appearing patient). The cryptic shock patients were not hypotensive in the strictest sense, but they were by no means physiologically normal in Puskarich, et al. They were, on the contrary, older, more tachycardic, and had faster respiratory rates than their hypotensive counterparts. And though they were not hypotensive (<90 mm Hg), their blood pressures were not necessarily normal. The median blood pressure in the cryptic shock group was 108 mm Hg with an IQR of 92-126. (Resuscitation 2011;82[10]:1289.) To put it simply, these patients appeared clinically ill and likely would have been identified using the qSOFA criteria.

In fact, Howell, et al., found a lactate threshold of 4 mmol/L to be very specific (92%), but it only identified a small minority of the patients who went on to die during their hospital stay (sensitivity, 36%). The analyses by Seymour, et al., and Freund, et al., would suggest this subset of critically ill patients identified by an elevated serum lactate is included in the cohort identified with a qSOFA score of 2 or greater. Lactate still serves the purpose of identifying the spectrum of patients who are critically ill, but I doubt it will help us recognize the patients who appear well when they present to the ED but go on to decompensate during their hospital course.

The true complexity of sepsis is still being uncovered. What we do know is it presents itself in a multitude of ways, and its clinical course and the rapidity of its progression is determined by more variables than we are even know. To think we can identify a set of clinical characteristics that is easily utilized at the bedside and will perfectly differentiate the subset of critically ill patients from the more clinically benign is at best naive.

qSOFA and the SIRS criteria are reasonable from a triage perspective (depending on your accepted signal vs. noise ratio), but neither will stand up to the clinical judgment of an experienced emergency physician doing what he is trained to do: to differentiate the sick from the not-sick. It is also quite possible that we will never identify a tool that detects all the patients at risk of decompensation during their hospital stay because a subset of patients will have a clinical decline that cannot be predicted at initial presentation, at least not without adding so much diagnostic noise that it drowns out any clinically useful signal. Sepsis and septic shock are clinical diagnoses, far more nuanced than any criteria can encapsulate. Sepsis 3.0 may very well have changed the geographic borders in which we reside, but here in the trenches of the emergency department, it is still the same rocky ground below our feet.

Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.