Sepsis is defined as life-threatening organ dysfunction due to dysregulated host response to infection (1 , 2). Recently, the World Health Organization (WHO) recognized sepsis as a global health priority, highlighting the importance of accurate international benchmarking to inform regional health policy (3). The key drivers for this WHO resolution include a high-extrapolated global sepsis burden of 15–20 million cases per annum (4 , 5) and the 26% average hospital mortality with sepsis-related critical illness (5). There are several global initiatives to address the high mortality from sepsis (6 , 7), but little consensus on how best to do international comparisons.
The crude hospital mortality from sepsis varies significantly between countries in prevalence studies (5 , 8–10). This variation in mortality has been linked to differences in critical care (ICU) provision (11 , 12), case definition (13), sepsis occurrence rate (9), economic region (8), and to differences in guideline compliance (10 , 14–16). In this scenario, Brazil and England have important differences, such as higher critical care bed availability per population in Brazil (13 per 100,000 inhabitants) compared with England (3.5–7.4 per 100,000 inhabitants) (17). These between countries differences highlight the need for standardized benchmarking to compare sepsis mortality because crude mortality comparisons would be not accurate (5 , 17).
At the patient level, both general patients characteristics (such as age, sex, presence of comorbidities) and sepsis-specific characteristics (such as site of infection and number and type of organ dysfunction) are important determinants of mortality in septic patients (17–19). However, lack of readily available calibrated risk prediction models and agreed risk-adjustment methods that are easy to implement across countries precludes risk-adjusted mortality comparisons between countries. Added to this is the heterogeneity observed among patients with sepsis, which presents a further challenge in designing risk-adjusted mortality comparisons (17 , 20). Together with incorporating new biomarkers and deriving phenotypes (20), another approach uses main determinants of the syndrome sepsis, such as site of infection and associated organ dysfunction, to account for sepsis heterogeneity (17).
It is feasible to do comparisons between countries using national level databases (21), and it is also feasible to collect general and sepsis-specific patient characteristics accurately across countries (8 , 9). In this context, we hypothesized that a sequential adjustment for general and for sepsis-specific patient characteristics would illustrate the limitations of comparing crude mortality rates. To test this hypothesis, we harmonized two ICU databases from Brazil and England, with the first ICU episode for adult medical admissions with sepsis in 2013. We report crude, stratified, and adjusted hospital mortality comparisons between the two countries, incrementally adjusting for general, and then for sepsis-specific, patient characteristics.
Data Sources and Funding
We used two national ICU databases to identify sepsis cohorts from Brazil and England admitted in 2013. The ORganizational CHaractEeriSTics in cRitical cAre (ORCHESTRA) study was a multicenter retrospective cohort study of critical care organization in 11 Brazilian states coordinated by the Department of Critical Care at the D’Or Institute for Research and Education. The data was collected by dedicated and trained staff through the Epimed Monitor System (Epimed Solutions, Rio de Janeiro, Brazil) (22). During selection of ICUs for ORCHESTRA, six ICUs with greater than 10% of missing data on core variables (age, ICU admission diagnosis, ICU and hospital length of stay [LOS], and hospital mortality) and two ICUs with less than 12 months of patient data were excluded, resulting in 78 ICUs at 51 hospitals (46 private and five public) with available data. For this study, we excluded 16 specialized ICUs (e.g., cardiothoracic, neurologic) resulting in 62 general ICUs in Brazil sepsis cohort. The Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme Database (CMPD) is the national clinical audit for adult general ICUs in England. This database includes data from 164 ICUs in England. The data are collected by dedicated and trained staff to a defined dataset specification including the hierarchical ICNARC Coding Method (23). All 164 ICUs in England had less than or equal to 10% of missing data on core variables and contributed greater than or equal to 12 months of patient data. Both these high-quality ICU databases collect sociodemographic, comorbidity, physiologic, and outcomes data for consecutive ICU admissions to precise rules and definitions. Both databases undergo extensive data validation; the details of which have been reported previously (22 , 23). Local Ethics Committee at the Instituto D’Or de Pesquisa (Parecer: 334.835) and the Brazilian National Ethics Committee (CAAE: 19687113.8.1001.5249) approved the ORCHESTRA study. Support for the collection and use of the ICNARC CPMD data has been obtained under Section 251 of the National Health Service Act 2006 (approval number: PIAG 2–10(f)/2005).
Harmonization of Datasets
We used a flexible harmonization method (24). First, we defined, a priori, the target variables necessary to test our hypothesis. The data collection forms and data dictionaries from ORCHESTRA and the ICNARC CMPD datasets were then assessed and consensus achieved on the target variables that could be reliably determined and mapped in both datasets. Both datasets were separately prepared, quality checked, and anonymized using the same algorithm to generate the harmonized variables and a merged dataset that included country as a categorical variable. Harmonization steps were conducted before any statistical analysis.
Patients and Variables in the Harmonized Dataset
We included the first ICU admission for adult (≥ 16 yr old) medical (nonsurgical) patients with sepsis to adult general ICUs between 1 January and 31 December 2013. We excluded patients transferred in from other hospitals and patients admitted to specialized ICUs (e.g., cardiothoracic, neurologic/neurosurgical, and stand-alone high dependency units).
General patient characteristics included age, sex, comorbidities, functional status, source of admission, and time from hospital to ICU admission. Comorbidities were based on the Acute Physiology and Chronic Health Evaluation (APACHE) II definitions in the ICNARC CMPD and the Charlson comorbidity index in the ORCHESTRA study. We classified patients’ functional status in two categories: independent and partially or fully dependent (eTable1, Supplemental Digital Content 1, http://links.lww.com/CCM/E9). Sepsis-specific patient characteristics included site of infection, number and type of organ dysfunction (17). Sepsis was defined as infection and greater than or equal to 2 points in any individual domain of the Sequential Organ Failure Assessment (SOFA) score using data from the first 24 hours of ICU admission (eTable2, Supplemental Digital Content 1, http://links.lww.com/CCM/E9). Site of infection was determined at ICU admission and coded using hierarchical codes in both datasets (eTable3, Supplemental Digital Content 1, http://links.lww.com/CCM/E9).
The primary exposure was country (Brazil or England), and the primary outcome was hospital mortality. The analysis plan was finalized prior to any analysis. We used three sequential random effects, multilevel logistic regression models to evaluate the impact of country on hospital mortality. First, we fitted a baseline model with the country (Brazil/England) as a fixed effect and ICU as random effect (model-1). Second, to model-1, we added all general patient characteristics (age, sex, comorbidities, functional status, source of ICU admission, and time from hospital admission to ICU admission) (model-2). Third, to model-2, we added sepsis-specific patient characteristics (site of infection and organ dysfunctions representing acute illness severity) (model-3). We ran the same three sequential models in two sensitivity analyses: 1) in the subpopulation of patients without chronic comorbidities, to evaluate the potential impact of SOFA definition in patients with chronic organ dysfunction and 2) handling the immunosuppression category as originally entered in the database, to evaluate the potential different prognostic impact of each immunosuppressed category. Because of the multicollinearity between type and number of organ dysfunctions, we assessed the impact of organ dysfunction in two different ways: 1) in model-3A, we adjusted only for the types of organ dysfunctions and their first-order interactions and 2) in model-3B, we adjusted only for the number of organ dysfunctions. Age and time from hospital admission to ICU admission were fitted using restricted cubic splines to deal with nonlinear associations in all models. The potential for effect modification of the effect of country on outcome was explored by adding interaction terms between country and the following prespecified covariates in model-3: source of admission (emergency department vs other), time from hospital admission to ICU admission (< 1, 1–2, 3–7, ≥8 d) and site of infection, type of organ dysfunction (model-3A), or number of organ dysfunctions (model-3B). Finally, we explored the interaction between country effect and baseline predicted risk of hospital mortality using the linear predictor of fixed effects from model-3.
We used Wald tests for all hypothesis testing. As the amount of missing data at individual patient level for all noncore variables was expected to be low (< 5%), the primary analysis was complete case analysis. Continuous variables are presented as mean (SD) for those that were not skewed and median (interquartile range) for those skewed. Categorical variables are presented as counts and percentages. All analyses were performed using Stata/SE Version 13.0 (StataCorp LP, College Station, TX).
Over the study period, amongst the 34,150 and 58,316 adult medical ICU admissions, 10,167 patients (29.8%) and 19,491 patients (33.4%) had infection as the main reason for admission to ICUs in Brazil and England, respectively. From these cohorts, we included 4,505 of 10,167 patients (44.3%) with sepsis from Brazil and 17,921 of 19,491 (92.0%) from England who had organ failure and met the sepsis criteria (eFig.1, Supplemental Digital Content 1, http://links.lww.com/CCM/E9). There were 22,061 sepsis patients (98.4%) for complete case analysis.
Sepsis admissions to ICUs in Brazil were older, with greater prevalence of comorbidities and impaired functional status. The most common source of admission to ICU in Brazil was the emergency department (ED) (n = 3,250; 72.1%) and the general ward (n = 11,023; 61.5%) in England. Sepsis patients admitted to ICUs in Brazil spent a longer pre-ICU admission period either in the ED or on the ward compared with sepsis admissions to ICUs in England (Table 1).
The most common infection source amongst ICU admissions with sepsis was respiratory for both countries. Genitourinary as an infection source was 3.5 times more common in Brazil, and gastrointestinal was twice as common in England. The percentage of sepsis admissions with single organ dysfunction was higher in Brazil (54.9% vs 32.8%), whereas those with two, three, four, and five organ dysfunctions were higher in England. Cardiovascular was the most common organ dysfunction in Brazil (41.2%), and respiratory was the most common organ dysfunction in England (85.8%) (Table 1; and eTable4, Supplemental Digital Content 1, http://links.lww.com/CCM/E9).
The crude hospital mortality for all medical ICU patients and for infected ICU patients was lower for Brazil compared with England (18.6% vs 30.6% for medical; 27.1% vs 37.2% for infected patients). In contrast, the crude hospital mortality for ICU sepsis admissions was comparable between Brazil (41.4%) and England (39.3%). However, when stratified by source of admission, site of infection, type, number and combinations of organ dysfunction, sepsis admissions to ICUs in Brazil had higher crude mortality (Table 2 and Fig. 1). Overall, ICU and hospital LOS were similar between countries; however, nonsurvivors had greater ICU and hospital LOS in Brazil compared with England (eTable4, Supplemental Digital Content 1, http://links.lww.com/CCM/E9).
Impact of Sequential Risk Factor Adjustment With Multilevel Logistic Regression Models
There was no statistically significant difference in the crude mortality between the two countries (model-1: crude odds ratio [OR], 1.12 [0.98–1.30] for Brazil vs England; p = 0.11). After adjusting for general patient characteristics, there was a change in the point-estimate of the OR, which highlighted that the general patient characteristics confounded the sepsis-mortality relationship (model-2: adjusted OR, 0.88 [0.75–1.02] for Brazil vs England; p = 0.09). When we adjusted for general and for sepsis-specific patient characteristics, the point estimate reversed, with sepsis admissions in Brazil having significantly greater odds of hospital death compared with England (model-3A: adjusted OR, 1.22 [1.05–1.43]; p = 0.01 and model-3B: adjusted OR, 1.40 [1.22–1.62]; p < 0.01) (Table 3). The two sensitivity analyses showed similar phenomena in the reversal of the point-estimate of the association when adjusting for general and sepsis-specific characteristics (eTable5, Supplemental Digital Content 1, http://links.lww.com/CCM/E9).
Effect Modifiers of the Country Effect on Mortality
The country effect also varied with time from hospitalization to ICU admission, site of infection, type and number of organ dysfunctions strata, implying effect modification of these variables in the sepsis-mortality relationship (Fig. 2; and eFig.2 and eTables 6 and 7, Supplemental Digital Content 1, http://links.lww.com/CCM/E9). Brazil had worse outcomes for those with very low- and very-high predicted risk of hospital death, which implies interaction between country effect and baseline predicted risk of death (Fig. 2; and eFig.2 and eTables8 and 9, Supplemental Digital Content 1, http://links.lww.com/CCM/E9).
We report the first study testing the impact of general and sepsis-specific patient characteristics, whereas accounting for country effect, in the sepsis-mortality relationship, using harmonized raw patient level clinical data from two high-quality ICU databases. Brazil had a greater prevalence of patients with sepsis-specific characteristics associated with lower risk of death such as genitourinary infection and single organ dysfunction and a greater prevalence of patients with general patient characteristics associated with higher risk of death such as older age, comorbidities, and impaired functional status. Although the overall crude hospital mortality was similar between Brazil and England, there were major differences in the stratified hospital mortality. Brazil had a significantly higher adjusted risk of death from sepsis when general, and sepsis-specific patient characteristics were accounted for.
The framework for international comparisons of sepsis cohorts presented in this study addresses the challenges highlighted with crude comparisons (17) and provides novel data toward 2017 WHO agenda (3 , 25)—“on the need for better evaluation tools for global sepsis epidemiology.” As illustrated in our two national cohorts, sepsis-specific patient characteristics such as site of infection, the number and types of organ dysfunctions are likely to vary between healthcare systems/countries. Therefore, it is expected that general prognostic models such as APACHE II, where sepsis might be a single category weighted by a single coefficient, are not enough to capture the impact of the variation of sepsis-specific patient characteristics on mortality. Our model accounted for those important variables, used commonly collected variables and used two accessible modeling strategies to enhance the adjustments: 1) for sepsis-specific patient characteristics, by including a first-order interaction between organ dysfunctions and 2) for unmeasured variability at the health unit level, by including random intercepts for each unit (17). Nevertheless, it is possible that a more complex model, allowing for other acute physiologic derangements, could improve the adjustments between cohorts.
The strong impact of sepsis-specific characteristics on the country-associated hospital mortality also must be highlighted. Indeed, sepsis from neurologic infections and those patients with increasing number of organ dysfunctions seem to drive the change on the direction of the OR after adjusting for general and sepsis-specific characteristics. This should be further evaluated, but it might represent differences in health practices within each country, such as clinical suspicion and organ support indications. Furthermore, it might be informative for international benchmarking, highlighting room for improvement in specific sepsis diagnosis and care areas.
Our study has strengths. We harmonized data variables between two representative national ICU databases (22 , 23), using similar definitions (24), over the same period and explicitly tested for country effect after adjusting for key confounders (26). We developed our statistical analysis plans blinded to country level outcome data. The harmonization of data variables and application of hierarchical multilevel regression analysis, allowing for random effects, is methodologically robust for testing our research question (27). We used previously reported sepsis-specific patient characteristics and current knowledge from sepsis epidemiology for adjustment and comparison between cohorts (17). Importantly, the proposed adjustment based on target variables for general and for sepsis-specific patient characteristics are feasible and reproducible, both fundamental features for international comparisons, and for informed benchmarking.
Our study has several limitations. Although neither ICU database was setup for exploring sepsis epidemiology, they are appropriate to inform how adjustment for both general and sepsis-specific patient characteristics could improve upon the crude mortality comparisons often reported. There is no “gold-standard” for international comparisons of sepsis epidemiology (17). However, we illustrate that outcome variability is partially related to differences in type of sepsis patients cared between Brazil and England. Thus, such comparisons are feasible, and our hypothesis-driven risk-adjustment proposal provides a valid starting point to generate between country estimates for external benchmarking. We used our previously published feasible risk adjustment methods (17), including variables commonly measured in the ICUs that are well-known risk factors for sepsis mortality, and illustrate the impact of sequential adjustment for general and for sepsis-specific patient characteristics. Nevertheless, we made minor adaptations in the comorbidities and SOFA definitions to harmonize both datasets. We believe these adaptations might have introduced some degree of non-differential misclassification, according to the flexible harmonization approach (24). We could not measure access to the health system, the ICU capacity strain during the period, do-not-resuscitate practices, refused admissions to critical care and discharge practices. The finding that lower- and higher-risk patients have worse outcomes in the Brazilian cohort might reflect differences in these factors. Restricted by available data in the ICU databases, we could not generalize our findings to sepsis patients treated outside the ICU; however, we applied a standardized sepsis definition in patients admitted to general ICUs, strengthening our internal validity. We applied a modified version of the SOFA score, missing the liver domain. However, the expected occurrence rate of liver dysfunction is low in adult general ICUs and commonly occurs simultaneously with other organ dysfunctions, such that we anticipate that we may have missed only very small differences between the cohorts. Finally, we compared two countries for this case study and explored the country variable as a fixed effect. If more countries were to be evaluated, it may be advisable to consider a random-effects approach.
Our study informs future research. First, our study highlights an approach for international comparisons of sepsis mortality that requires replication. Our approach is very similar to the cancer literature, where international differences in stage-specific cancer survival between countries could be explained by differences in general patient characteristics such as comorbidities and cancer-specific patient characteristics such as cancer stage (28 , 29). Second, the process of care improvement strategies for sepsis is likely to vary by country (14) which requires formal testing. For example, the impact of pre-ICU time: we observed that patients in Brazil were commonly referred direct to ICU, whereas higher proportions of patients in England were transferred to general wards prior to ICU admission. Additionally, the country effect on sepsis hospital mortality varied with time from hospitalization to ICU admission. This could also imply differences in the magnitude of strain (30 , 31) operating on the general wards and in the ICUs between countries. The treatment these patients received pre-ICU admission is also of paramount importance, because of the importance of early treatment for sepsis patients (32). We observed that, despite similar proportions of patients with infection at ICU admission, the proportion of admissions with sepsis was lower in Brazil compared with England, potentially explained by the lower admission threshold of private ICUs contributing to the ORCHESTRA database. Third, international harmonization of ICU clinical trials and large global adaptive platform trials are being considered (33). Our study highlights the need to have stratification by country to account for the differences in crude mortality and to explore biological differences in treatment response in sepsis populations between countries.
Our study highlights that crude mortality comparison of sepsis patients admitted to critical care units is of limited value in understanding reasons for differences in outcome between countries. Both general and sepsis-specific patient characteristics influence mortality of sepsis patients admitted to the ICU and are distributed differently between cohorts from the two countries. We provide a template for studies attempting international benchmarking of ICU sepsis mortality.
1. Singer M, Deutschman CS, Seymour CW, et al. The third international
consensus definitions for sepsis
and septic shock (sepsis
-3). JAMA 2016; 315:801–810
2. Shankar-Hari M, Phillips GS, Levy ML, et al; Sepsis
Definitions Task Force: Developing a new definition and assessing new clinical criteria for septic shock: For the third international
consensus definitions for sepsis
and septic shock (sepsis
-3). JAMA 2016; 315:775–787
3. Reinhart K, Daniels R, Kissoon N, et al. Recognizing sepsis
as a global health priority - a WHO resolution. N Engl J Med 2017; 377:414–417
4. Adhikari NK, Fowler RA, Bhagwanjee S, et al. Critical care and the global burden of critical illness in adults. Lancet 2010; 376:1339–1346
5. Fleischmann C, Scherag A, Adhikari NK, et al; International
Forum of Acute Care Trialists: Assessment of global incidence and mortality of hospital-treated sepsis
. Current estimates and limitations. Am J Respir Crit Care Med 2016; 193:259–272
6. Rhodes A, Evans LE, Alhazzani W, et al. Surviving sepsis
guidelines for management of sepsis
and septic shock: 2016. Intensive Care
Med 2017; 43:304–377
8. Vincent JL, Marshall JC, Namendys-Silva SA, et al; ICON Investigators: Assessment of the worldwide burden of critical illness: The Intensive Care
Over Nations (ICON) audit. Lancet Respir Med 2014; 2:380–386
9. Vincent JL, Sakr Y, Sprung CL, et al; Sepsis
Occurrence in Acutely Ill Patients Investigators: Sepsis
in European intensive care
units: Results of the SOAP study. Crit Care Med 2006; 34:344–353
10. Phua J, Koh Y, Du B, et al; MOSAICS Study Group: Management of severe sepsis
in patients admitted to Asian intensive care
units: Prospective cohort study. BMJ 2011; 342:d3245
11. Wunsch H, Angus DC, Harrison DA, et al. Variation in critical care services across North America and Western Europe. Crit Care Med 2008; 36:2787–2793, e19
12. Wunsch H. Is there a Starling curve for intensive care
? Chest 2012; 141:1393–1399
13. Gaieski DF, Edwards JM, Kallan MJ, et al. Benchmarking the incidence and mortality of severe sepsis
in the United States. Crit Care Med 2013; 41:1167–1174
14. Ferrer R, Artigas A, Levy MM, et al; Edusepsis Study Group: Improvement in process of care and outcome after a multicenter severe sepsis
educational program in Spain. JAMA 2008; 299:2294–2303
15. Levy MM, Dellinger RP, Townsend SR, et al. The surviving sepsis
campaign: Results of an international
guideline-based performance improvement program targeting severe sepsis
. Intensive Care
Med 2010; 36:222–231
16. Rhodes A, Phillips G, Beale R, et al. The surviving sepsis
campaign bundles and outcome: Results from the International
Multicentre Prevalence Study on Sepsis
(the IMPreSS study). Intensive Care
Med 2015; 41:1620–1628
17. Shankar-Hari M, Harrison DA, Rowan KM. Differences in impact of definitional elements on mortality precludes international
comparisons of sepsis epidemiology
-a cohort study illustrating the need for standardized reporting. Crit Care Med 2016; 44:2223–2230
18. Mayr FB, Yende S, Angus DC. Epidemiology
of severe sepsis
. Virulence 2014; 5:4–11
19. Mayr FB, Yende S, Linde-Zwirble WT, et al. Infection rate and acute organ dysfunction risk as explanations for racial differences in severe sepsis
. JAMA 2010; 303:2495–2503
20. Sweeney TE, Azad TD, Donato M, et al. Unsupervised analysis of transcriptomics in bacterial sepsis
across multiple datasets reveals three robust clusters. Crit Care Med 2018; 46:915–925
21. Wunsch H, Angus DC, Harrison DA, et al. Comparison of medical admissions to intensive care
units in the United States and United Kingdom. Am J Respir Crit Care Med 2011; 183:1666–1673
22. Soares M, Bozza FA, Angus DC, et al. Organizational characteristics, outcomes
, and resource use in 78 Brazilian intensive care
units: The ORCHESTRA study. Intensive Care
Med 2015; 41:2149–2160
23. Harrison DA, Brady AR, Rowan K. Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: The Intensive Care
National Audit & Research Centre Case Mix Programme Database. Crit Care 2004; 8:R99–R111
24. Fortier I, Doiron D, Burton P, et al. Invited commentary: Consolidating data harmonization–how to obtain quality and applicability? Am J Epidemiol 2011; 174:261–264; author reply 265266
26. Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and confounding in critical care health services research involving observational data. J Crit Care 2006; 21:1–7
27. Bingenheimer JB, Raudenbush SW. Statistical and substantive inferences in public health: Issues in the application of multilevel models. Annu Rev Public Health 2004; 25:53–77
28. Walters S, Maringe C, Butler J, et al. Comparability of stage data in cancer registries in six countries: Lessons from the International
Cancer Benchmarking Partnership. Int J Cancer 2013; 132:676–685
29. Ciccolallo L, Capocaccia R, Coleman MP, et al. Survival differences between European and US patients with colorectal cancer: Role of stage at diagnosis and surgery. Gut 2005; 54:268–273
30. Hefter Y, Madahar P, Eisen LA, et al. A time-motion study of ICU workflow and the impact of strain. Crit Care Med 2016; 44:1482–1489
31. Gabler NB, Ratcliffe SJ, Wagner J, et al. Mortality among patients admitted to strained intensive care
units. Am J Respir Crit Care Med 2013; 188:800–806
32. Noritomi DT, Ranzani OT, Monteiro MB, et al. Implementation of a multifaceted sepsis
education program in an emerging country setting: Clinical outcomes
and cost-effectiveness in a long-term follow-up study. Intensive Care
Med 2014; 40:182–191
33. Marshall JC. Global collaboration in acute care clinical research: Opportunities, challenges, and needs. Crit Care Med 2017; 45:311–320
epidemiology; heterogeneity; intensive care; international; outcomes; sepsis
Supplemental Digital Content
Copyright © by 2019 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.