The core aspect of early goal-directed therapy (EGDT) is defined by fluid resuscitation overlaid on antimicrobial strategies all administered in a timely fashion based on bedside physiologic parameters in patients with severe sepsis or septic shock (1). The first randomized trial that demonstrated a survival benefit in this specific patient population was published in 2001 (1). Even though the EGDT core interventions were based on physiologic principles, not on validated scientific evidence, the magnitude of survival benefit (16% absolute mortality reduction) with EGDT was unmatched by nearly any other therapy in the field of critical care medicine. The methodologies from the 2001 trial were adopted by the Surviving Sepsis Campaign (2). Over the following decade, a multitude of real-life observational studies was performed, many demonstrating significant survival benefits; however, no other large randomized clinical trials were performed (3). In the last 2 years, three more randomized trials have been completed (4–6). However, the lack of consistency among different trials has generated substantial scientific controversy (7).
The goal of our study is to address the following question: What are the reasons for the discordant survival outcomes observed in the most recent trials compared with the 2001 trial and observational studies?
PubMed, Embase, Cochrane Library, Evidence-based Medicine BMJ, and American College of Physicians Journal Club were searched from January 2001 through January 2016. The specific literature search strategy is included in eTable 3 (Supplemental Digital Content 4, http://links.lww.com/CCM/C296). No language restrictions were applied. If only data from the abstract form were found to meet our study inclusion criteria, but no article was published or available, then the abstracts’ results were excluded from the main analysis, but included in the sensitivity analysis. One author (A. C. K.) and one librarian (Dr. Cindy Schmidt) performed the literature search strategies separately. Any disagreement was resolved by a final consensus.
Inclusion criteria: All randomized and observational nonrandomized studies that evaluated EGDT in patients with severe sepsis and/or septic shock and reported mortality outcomes. Observational studies included prospective or retrospective prepost and quasi-experimental designs.
Exclusion criteria: 1) Mortality outcome was not provided; 2) mortality could not be collected separately for patients who received EGDT in conjunction with other sepsis bundles; 3) EGDT was used in both study arms (e.g., central venous saturation compared with lactate clearance); and 4) studies published before January 2001.
The following variables were collected from all studies: authors, publication year, study design, sample size, age; hospital location; ICU exposure; Acute Physiology and Chronic Health Evaluation (APACHE) II score; Sequential Organ Failure Assessment (SOFA) score; Simplified Acute Physiology Score II, Mortality in Emergency Department Sepsis score; mean arterial pressure; systolic blood pressure; lactate; vasopressor use; EGDT compliance; ScVO2 achievement; central venous pressure achievement; fluid administration; blood culture collection; antibiotic appropriateness; time-to-antibiotic administration (continuous and categorical) from the diagnosis of severe sepsis or septic shock; hospital mortality; 90-day mortality; 60-day mortality; 28-day mortality; ICU morality; and all-cause mortality (eTable 6, Supplemental Digital Content 7, http://links.lww.com/CCM/C299).
The authors prospectively defined the following steps: step 1: standard random-effects analysis (8) to identify the adjusted pooled mortality outcomes for all studies, and separately for randomized and nonrandomized studies; step 2: Bayesian hierarchical regression analysis (9–12) to evaluate the mortality outcomes, which involved a 3-level regression model for the pooled analysis of all studies to account for the variability within trials (first level) and between trials (second level), as well between study designs (third level); a uniform prior was used for this analysis. Based on the potential selection bias of nonrandomized studies and the difference in mortality outcomes between study designs, we also performed standard Bayesian analysis of the randomized trials only with various priors. As part of the standard Bayesian approach, we performed these analyses based on specific priors. Each prior provides the level of evidence before the execution of the next randomized or observational study. In other words, the accumulation of evidence will naturally provide more prior information for the context and analysis of each new study. The larger the amount of prior information (i.e., many studies, large sample size, and small variance), the more difficult it is for a new study to change the body of existing evidence; for example, severe skeptic prior. The smaller the amount of prior information (i.e., few studies, small sample size, and large variance), the easier it is for a new study to change the body of existing evidence; for example, the uniform prior. In sum, we performed this comprehensive Bayesian analysis to formally account for the uncertainty related to the existing evidence. A didactic and comprehensive review on this subject can be seen elsewhere (12). These 2-level Bayesian analyses were performed with the following priors: 1) uniform prior (no other evidence available before the randomized trials), 2) rivers prior (this trial was removed from the pool of randomized studies and used as the only prior evidence before all other randomized studies, 3) naïve prior (mean and variance from all observational studies), 4) equivalent prior (mean from observational and variance from randomized trials), 5) moderate skeptic prior (50% reduction on the mean survival effect from observational studies and variance from all observational studies), and 6) severe skeptic prior (50% reduction on the mean survival effect from observational studies and variance from all randomized studies); step 3: subgroup random-effects regression analyses of clinical baseline and interventional factors that were prospectively defined and deemed to affect mortality outcomes. In order to determine which variables could modify the statistically significant difference between randomized and observational studies, all subgroup analyses were performed with two independent variables: design type (observational vs randomized) and clinical variable of interest. This approach also prevented overfitting by not including too many variables in relation to the number of selected studies. The relative risk (RR) calculated by the random-effects model was the outcome measure used for all analyses, and hospital mortality was the primary outcome. If hospital mortality was not available, then the 60-day mortality was used, and if this was also not available, then the 28-day mortality was used. The Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines were followed according Enhancing the QUAlity and Transparency Of health Research recommendations (eTable 4, Supplemental Digital Content 5, http://links.lww.com/CCM/C297). Statistical software programs used for this article included R Program v. 3.1.3 (R Foundation, Vienna, Austria), Stata v. 14.0 (College Station, TX), and Biostat v. 3.2 (Englewood, NJ).
Thirty-one observational studies (n = 15,656) (for these references, see eTable 2, Supplemental Digital Content 3, http://links.lww.com/CCM/C295) and six randomized trials (n = 4,342) (1, 4–6, 13, 14) were selected for the main analyses (Fig. 1). Six studies from China were available only in abstract form (15–20); thus, they were included in the sensitivity analysis. Two studies compared lactate clearance versus ScVO2 and used EGDT in both arms; thus, they were excluded (21, 22). See eTable 1 (Supplemental Digital Content 2, http://links.lww.com/CCM/C294) for study design and quality.
Step 1: Standard Random-Effects Analysis
The pooled analysis of all trials showed that EGDT was associated with a 23% reduction in the risk of death (RR = 0.77 [95% CI, 0.71–0.83]; p < 0.0001; I2 = 60%; n = 19,998) (Fig. 2). The study design type showed a nonsignificant mortality reduction with the randomized studies (RR = 0.92 [95% CI, 0.78–1.07]; p = 0.268; I2 = 57%; n = 4,342) and a significant mortality reduction with the observational studies (RR = 0.73 [95% CI, 0.67–0.80]; p < 0.0001; I2 = 52%, n = 15,656) (Fig. 2). The meta-regression comparing the study designs showed that the observational studies’ findings were significantly different from the randomized studies (RR = 0.81 [95% CI, 0.68–0.95]; p = 0.01; I2 = 52%; n = 19,998) (Table 1). This analysis showed that the risk of death with EGDT compared with controls in the observational studies was 19% lower than that observed with EGDT compared with controls in the randomized studies.
Step 2: Bayesian Hierarchical Regression Analysis
The 3-level regression analysis generated the following posterior probabilities for the mortality outcome: randomized: RR = 0.87 (95% Credible Interval [95% CrI], 0.72–1.04) and observational: RR = 0.73 (95% CrI, 0.67–0.79). These results were similar to the ones observed in step 1. The 2-level Bayesian regression analyses modeled only randomized trials and the posterior probabilities for each prior evidence are the following: 1) uniform prior: RR = 0.91 (95% CrI, 0.75–1.12); 2) rivers prior: RR = 0.86 (95% CrI, 0.69–1.03); 3) naïve prior: RR = 0.76 (95% CrI, 0.70–0.82); equivalent prior: RR = 0.80 (95% CrI, 0.70–0.90); moderate skeptic prior: RR = 0.87 (95% CrI, 0.80–0.93); and severe skeptic prior: RR = 0.88 (95% CrI, 0.78–0.99). The efficiency and convergence findings from each regression model were verified and appropriate.
Step 3: Prospective Subgroup Regression Analysis
All univariate regression analyses are presented in Table 1, and the bivariable regression analyses are in Table 2. The factors that explained (and superseded) the statistically significant mortality differences between randomized and observational studies were time-to-first antibiotic (R2 = 87%), administration of antibiotics within 6 hours (R2 = 94%), 4 hours (R2 = 99%), 3 hours (R2 = 99%), and appropriate antibiotic use (R2 = 96%) (Table 2). Even though the univariate regression on appropriate antibiotic use was not significant, we decided to include it in the bivariable regression analysis due to this variable’s low reporting rates in most studies which suggested low statistical power, as well as due to its known clinical relevance. All bivariable regression models that included these three antibiotic therapy variables (i.e., time-to-first antibiotic, antibiotics administered within a specific time [6, 4, 3 hr], and appropriate antibiotic use) eliminated the significant mortality difference initially observed between randomized and observational study designs (Table 2). The mean time-to-first antibiotic administration in the observational studies (n = 13) was 1.84 hours for the EGDT arm and 2.44 hours for the control arm (mean difference, –0.60 hr [95% CI, –0.92 to –0.28]; adjusted mean difference, –0.20 hr [95% CI, –0.30 to –0.11]; p = 0.00004). The mean time-to-first antibiotic administration in the randomized studies (n = 2) was 1.33 hours for the EGDT arm and 1.21 hours for the control arm (mean difference, 0.12 hr [95% CI, –0.90–1.14]; adjusted mean difference, 0.02 hr [95% CI, –0.07 to 0.12]; p = 0.653). The meta-regression on survival and time-to-first antibiotic administration showed a significant survival reduction of 10% per every hour of antibiotic delay (p = 0.0006) (Fig. 3).
Six randomized studies performed in China and published in Chinese language journals were only available in abstract form (15–20). The inclusion of these studies did not change the mortality results for the randomized study design analysis (RR = 0.97 [95% CI, 0.81–1.15]; p = 0.697; I2 = 71%). The individual exclusion of either Protocolized Care for Early Septic Shock (ProCESS) or Protocolised Management in Sepsis (ProMISe) trials did not change our results; other sensitivity analyses are presented in eTable 5 (Supplemental Digital Content 6, http://links.lww.com/CCM/C298).
See eTable 5 (Supplemental Digital Content 6, http://links.lww.com/CCM/C298).
Our findings suggest that the differences in survival outcome were secondary to the specific designs of the original studies. The observed mortality difference between randomized and nonrandomized studies was not related to EGDT alone or to hemodynamic bundle compliance, but our results suggest that it was associated with faster and more appropriate administration of antibiotics. The antibiotic co-interventions that were associated with significant survival benefits observed in the EGDT arm of the nonrandomized trials were not replicated in the randomized trials because the same antibiotic co-interventions were similarly distributed between the EGDT and control arms in the randomized studies.
The strength of our findings is based on the established biological rationale in which the fast initiation of appropriate antibiotics is expected to improve the survival outcomes of patients with sepsis. This has been demonstrated by the large cohort study performed by Kumar et al (23) who showed that the survival of patients with septic shock was reduced by 7% for every hour of antibiotic initiation delay. Notably, our new findings suggest a very similar survival reduction of 10% per hour of antibiotic delay in the context of severe sepsis or septic shock (Fig. 3). Our categorical analyses based on the antibiotic administration within 6 hours (RR = 0.20), 4 hours (RR = 0.16), and 3 hours (RR = 0.09) from the diagnosis of severe sepsis or septic shock also showed a significant time-dependent survival improvement (Table 1). In addition, a more recent study by Ferrer et al (24) evaluated approximately 18,000 patients with severe sepsis and showed a significant increase in mortality with the number of hours of delay to first antibiotic administration.
Another strength of our findings is related to the innovative and robust statistical methodological approach used for all analyses. The combination of both Bayesian and conventional statistics was essential to understand and evaluate such a complex set of evidence. The use of different priors in our step 2 approach demonstrated that the survival outcome results of the randomized trials were highly susceptible to and easily superseded by the entirety of the available evidence. Furthermore, our comprehensive analysis identified the only clinical reason accountable for the survival benefits with EGDT: the faster and more appropriate use of antibiotic co-intervention in the EGDT arm compared with the non-EGDT arm of the nonrandomized studies.
An intriguing finding from our study was that the worst mortality associated with EGDT occurred in patients with higher disease severity (APACHE II, SOFA, and presence of shock) (Table 1). We conjecture that while a protocol-driven bundle approach may bring benefits for patients with sepsis in the emergency department or ICU settings, the same approach applied indiscriminately to all patients may bring harm to patients with sepsis who are too sick to receive the standard volume/hemodynamic resuscitation. This may be secondary to the fluid overload causing systemic and pulmonary edema, tissue hypoxia, end-organ damage, and increased overall mortality. This harmful effect from EGDT-induced fluid overload has been observed in four recent clinical studies in both adult and pediatric populations (14, 25–27). Two of these four studies were performed in resource limited areas of the world and the other two in the United States.
How do our study results compare with other previous meta-analyses? The studies by Angus et al (28) and Rusconi et al (29) analyzed only randomized trials and their results showed no survival benefits with EGDT, which were concordant with our randomized design analysis. The studies by Barochia et al (30) and Wira et al (31) included mostly observational studies and they showed survival benefits, also concordant with our observational analysis. Barochia et al (30) study showed that time-to-antibiotic were changed in the included studies. The study by Gu et al (32) included randomized trials and showed survival benefits; however, they included studies from a different era (over 20 yr old), studies of patients without sepsis, and studies that did not use central venous or mixed venous saturation, all of which make them not comparable with the most recent studies. The analysis by Sterling et al (33) included only 11 studies and close to 90% of their included population was derived from a single study; their population was clinically heterogeneous, multiple study designs were not accounted for in the analysis, and randomized trials were not included. Also distinct from our article, Sterling et al (33) did not evaluate antibiotic appropriateness and used administrative data, which lacked specific and relevant clinical information. What separates our new study from all others is the fact that we included 37 studies with approximately 20,000 patients and accounted for both randomized and observational study designs of comparable eras in order to ascertain the reasons for the different survival outcomes. Our new findings suggest that any study using the conventional approach of examining only one type of study design would not be capable of demonstrating mortality discordance. Our all-inclusive approach in conjunction with our use of both frequentist and Bayesian methodologies was the major strength of our meta-analytic approach and the basis for why we were able to not only detect mortality discordance but also discover the most likely reasons for the discordance between different study designs.
The consequences of our findings for both research and bedside practice of patients with severe sepsis and septic shock are several-fold: 1) Differences in baseline hemodynamics or compliance with the EGDT bundle are not likely the reasons for survival differences between randomized and nonrandomized trials; 2) independent of the effects of EGDT alone on survival, all sepsis resuscitation bundles and protocols should prioritize the early and appropriate use of antibiotics in order to improve survival; 3) because our results demonstrate that it is more likely than not we will continue seeing the same pattern of survival benefit secondary to antibiotic co-intervention, we believe it is improbable that additional randomized or nonrandomized trials on EGDT will be helpful in further defining utility of an already widely accepted practice; this does not preclude the performance of quality improvement initiatives to assess and improve local sepsis care, as well as of future clinical trials; and 4) patients with severe sepsis and high-disease severity may be harmed by EGDT.
Limitations of our study include the expected issues related to data aggregation and the lack of individual patient data. Another limitation is related to the heterogeneity found in several analyses. Nonetheless, the presence of heterogeneity was a critical factor in guiding the authors toward the possible reasons associated with the survival differences. Even though our regression models showed high R squared for the antibiotic analyses, other unknown factors could have influenced the discordant survival outcomes. Potential reasons for missing unknown factors may include indication bias, temporal trends, regression toward the mean, and concurrent changes in practice. Publication bias cannot be definitely ruled out, but our analyses by two different methods and a funnel plot showed no indication of this bias. Changes in clinical practice overtime, as well as flaws related to the design of each specific study (internal validity) (34) and meta-regression may have led to unmeasured bias in our study findings.
In conclusion, survival discordance was not associated with differences in EGDT bundle compliance or hemodynamic goal achievement. Our results suggest that it was associated with faster and more appropriate antibiotic co-intervention in the EGDT arm compared with controls in the observational studies but not in the randomized trials. EGDT was associated with increased mortality in patients with high-disease severity.
1. Rivers E, Nguyen B, Havstad S, et al: Early Goal-Directed Therapy Collaborative Group: Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001; 345:1368–1377
2. Dellinger RP, Levy MM, Carlet JM, et al. Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med. 2008; 36:296–327
3. Kalil AC, Sun J. Why are clinicians not embracing the results from pivotal clinical trials in severe sepsis? A Bayesian analysis. PLoS One. 2008; 3:e2291
4. Yealy DM, Kellum JA, Huang DT, et al: ProCESS Investigators: A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014; 370:1683–1693
5. Peake SL, Delaney A, Bailey M, et al. Goal-directed resuscitation for patients with early septic shock. N Engl J Med. 2014; 371:1496–1506
6. Mouncey PR, Osborn TM, Power GS, et al: ProMISe Trial Investigators: Trial of early, goal-directed resuscitation for septic shock. N Engl J Med. 2015; 372:1301–1311
7. Kalil AC. Wanted: Early goal-directed therapy for septic shock–dead or alive, but not critically ill! Intensive Care Med. 2010; 36:1–3
8. DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: An update. Contemp Clin Trials. 2007; 28:105–114
9. Prevost TC, Abrams KR, Jones DR. Hierarchical models in generalized synthesis of evidence: An example based on studies of breast cancer screening. Stat Med. 2000; 19:3359–3376
10. Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects meta-analysis of trials with binary outcomes: Methods for the absolute risk difference and relative risk scales. Stat Med. 2002; 21:1601–1623
11. Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res. 2001; 10:277–303
12. Kalil AC, Sun J. Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: A primer for clinicians. Crit Care Med. 2014; 42:2267–2277
13. Lin SM, Huang CD, Lin HC, et al. A modified goal-directed protocol improves clinical outcomes in intensive care unit patients with septic shock: A randomized controlled trial. Shock. 2006; 26:551–557
14. Andrews B, Muchemwa L, Kelly P, et al. Simplified severe sepsis protocol: A randomized controlled trial of modified early goal-directed therapy in Zambia. Crit Care Med. 2014; 42:2315–2324
15. He ZY, Gao Y, Wang XR, et al. [Clinical evaluation of execution of early goal directed therapy in septic shock]. Zhongguo Wei Zhong Bing Ji Jiu Yi Xue. 2007; 19:14–16
16. Chen ZQ, Jin YH, Chen H, et al. [Early goal-directed therapy lowers the incidence, severity and mortality of multiple organ dysfunction syndrome]. Nan Fang Yi Ke Da Xue Xue Bao. 2007; 27:1892–1895
17. Lu N, Zheng R, Lin H, et al. [Clinical studies of surviving sepsis bundles according to PiCCO on septic shock patients]. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2014; 26:23–27
18. Wang XZ, Lü CJ, Gao FQ, et al. [Efficacy of goal-directed therapy in the treatment of septic shock]. Zhongguo Wei Zhong Bing Ji Jiu Yi Xue. 2006; 18:661–664
19. Tian HH, Han SS, Lv CJ, et al. [The effect of early goal lactate clearance rate on the outcome of septic shock patients with severe pneumonia]. Zhongguo Wei Zhong Bing Ji Jiu Yi Xue. 2012; 24:42–45
20. Yan J. The effect of early goal-directed therapy on treatment of critical patients with severe sepsis/septic shock: A multi-center, prospective, randomized, controlled study. Zhongguo Wei Zhong Bing Ji Jiu Yi Xue. 2010; 22:331–334
21. Jones AE, Shapiro NI, Trzeciak S, et al. Lactate clearance vs central venous oxygen saturation as goals of early sepsis therapy: A randomized clinical trial. JAMA. 2010; 303:739–746
22. Yu B, Tian HY, Hu ZJ, et al. [Comparison of the effect of fluid resuscitation as guided either by lactate clearance rate or by central venous oxygen saturation in patients with sepsis]. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2013; 25:578–583
23. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006; 34:1589–1596
24. Ferrer R, Martin-Loeches I, Phillips G, et al. Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: Results from a guideline-based performance improvement program. Crit Care Med. 2014; 42:1749–1755
25. Kelm DJ, Perrin JT, Cartin-Ceba R, et al. Fluid overload in patients with severe sepsis and septic shock treated with early goal-directed therapy is associated with increased acute need for fluid-related medical interventions and hospital death. Shock. 2015; 43:68–73
26. Maitland K, Kiguli S, Opoka RO, et al: FEAST Trial Group: Mortality after fluid bolus in African children with severe infection. N Engl J Med. 2011; 364:2483–2495
27. Bhaskar P, Dhar AV, Thompson M, et al. Early fluid accumulation in children with shock and ICU mortality: A matched case-control study. Intensive Care Med. 2015; 41:1445–1453
28. Angus DC, Barnato AE, Bell D, et al. A systematic review and meta-analysis of early goal-directed therapy for septic shock: The ARISE, ProCESS and ProMISe Investigators. Intensive Care Med. 2015; 41:1549–1560
29. Rusconi AM, Bossi I, Lampard JG, et al. Early goal-directed therapy vs usual care in the treatment of severe sepsis and septic shock: A systematic review and meta-analysis. Intern Emerg Med. 2015; 10:731–743
30. Barochia AV, Cui X, Vitberg D, et al. Bundled care for septic shock: An analysis of clinical trials. Crit Care Med. 2010; 38:668–678
31. Wira CR, Dodge K, Sather J, et al. Meta-analysis of protocolized goal-directed hemodynamic optimization for the management of severe sepsis and septic shock in the emergency department. West J Emerg Med. 2014; 15:51–59
32. Gu WJ, Wang F, Bakker J, et al. The effect of goal-directed therapy on mortality in patients with sepsis—earlier is better: A meta-analysis of randomized controlled trials. Crit Care. 2014; 18:570
33. Sterling SA, Miller WR, Pryor J, et al. The impact of timing of antibiotics on outcomes in severe sepsis and septic shock: A systematic review and meta-analysis. Crit Care Med. 2015; 43:1907–1915
34. Ioannidis JP, Haidich AB, Pappa M, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001; 286:821–830
early goal-directed therapy; sepsis; septic shock
Supplemental Digital Content
Copyright © by 2017 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.