Accuracy Comparison Between Age-Adapted SOFA and SIRS in Predicting in-Hospital Mortality of Infected Children at China's PICU

Abstract

Objectives: 

Sepsis-3 consensus suggests “the need to develop similar updated definitions for pediatric populations.” Sequential organ failure assessment (SOFA) and systemic inflammatory response syndrome (SIRS) criteria are two systems widely used to define the status of infection. However, it is still unclear whether SOFA is more accurate than SIRS in predicting children mortality in low- and middle-income countries. Thus, we validated the accuracy of age-adapted SOFA and SIRS in predicating the poor prognosis of infected children in China's pediatric intensive care unit (PICU).

Methods: 

We performed a retrospective and observational cohort study of children admitted for infection to PICU in the hospital between January 1, 2009 and December 31, 2017. The indexes within 24 h after intensive care unit (ICU) admission were analyzed according to age-adapted SOFA and SIRS, and all data were sourced from the hospital's electronic health record database. The prognosis was illustrated with primary outcome and secondary outcome. Primary outcome referred to in-hospital mortality, and secondary outcome to in-hospital mortality or ICU length of stay ≥ 7 days. The predictive power of age-adapted SOFA and SIRS was compared using crude and adjusted area under the receiver operating characteristic curve (AUROC).

Results: 

Of 1,831 PICU-admitted children due to infection, 164 (9.0%) experienced primary outcome, and 948 (51.8%) secondary outcome. Of 164 deaths, 65.9% were males (median age of 7.53 months, range of 2.67–41.00 months). Children who scored ≥ 2 in age-adapted SOFA or met two SIRS criteria accounted for 92.5% and 73.3%, respectively. In addition, age-adapted SOFA score of ≥2 predicted adverse outcome more accurately than pediatric SIRS (adjusted AUROC, 0.753; 0.713–0.796 vs. 0.674; 0.631–0.702; P < 0.001).

Conclusion: 

Compared with SIRS criteria, age-adapted SOFA score of ≥ 2 enjoys a more accuracy in predicting in-hospital mortality of PICU-admitted children, and a higher sensitivity in identifying children with severe infection.

INTRODUCTION

Sepsis will remain a major threat for global children health in the next decades ⁽¹⁾. Standardizing definition and guidelines of sepsis can guarantee the efficiency of “Surviving Sepsis Campaign” that aims to decrease the sepsis-induced mortality. But the current definition does not encompass all the characteristics of sepsis ^{(2, 3)}, which hampers the consequent efforts to fight against this disease ^{(1, 4)}. Based on the refreshing knowledge on sepsis and emerging big-data analysis, JAMA released Sepsis-3 based on SOFA instead of SIRS in 2016. But this definition was not applicable in children, for it was just modified according to Sepsis-3 ⁽⁵⁾. Therefore, for pediatric sepsis, to make a SOFA-based definition and evaluate its efficiency is an urgent job lying ahead.

Recent studies ⁽⁶⁾ have verified that age-adapted SOFA enjoys a favorable accuracy in predicting the poor prognosis of infected children. Besides, Spesis-3 based on age-adapted SOFA shows a stronger predictive power than Sepsis-2 based on SIRS ⁽⁶⁾. But these advantages are only seen in high-income countries. In other countries with different scales of medical resources, especially low- and middle-income countries challenged with the majority of global sepsis patients, the prognosis of sepsis children shows a different profile ⁽⁷⁾. The large number of children in China makes its medical resources comparatively scarce ⁽⁸⁾. Although inflicted with a morbidity more than two times higher than that in Europe and America ⁽⁹⁾, China still lacks a comprehensive definition of pediatric sepsis. So, before the wide replication of age-adapted SOFA in China, its predictive power should be verified through comparison with SIRS.

This retrospective and observational cohort analysis compared the power of age-adapted SOFA and SIRS in predicting the prognosis of PICU children. The prognosis referred to in-hospital mortality (primary outcome) and in-hospital mortality or ICU length of stay ≥ 7 days (secondary outcome).

METHODS

Design and patients

This retrospective and observational cohort analysis included children admitted by a multidisciplinary PICU (covering internal medicine, surgical medicine, and traumatology) at a grade-A hospital between January 1, 2009 and December 31, 2017. Inclusion criteria: age of ≤18 (neonates excepted), existent or suspected infection at ICU admission. Exclusion criteria: records missed (age, gender, diagnosis), repeated admission by ICU for the same hospital episode, self-decided discharge, and unclear prognosis. This analysis was approved by the ethics committee of Guangzhou Women and Children's Medical Center. No informed consent was available because the analysis only covered the patients’ e-data.

Data collection

The data were collected from the hospital's e-health record database. Physiological and laboratory indexes within 24 h after ICU admission were scored with age-adapted SOFA (0–22) and classified with SIRS criteria (1–4). Missing variables scored 0 or were replaced with normal values in accordance with Raith et al. ⁽¹⁰⁾.

SIRS definition

According to 2005 Pediatric Sepsis Consensus statement, pediatric SIRS was defined as presence of ≥ 2 SIRS criteria, one of which must be abnormal temperature or white cell count ⁽¹¹⁾.

Statistical analysis

Chi-square test was performed for between-group comparison of categorical variables (percentage), and Mann–Whitney U test for between-group comparison of continuous variables (median). Two kinds of prognosis were predicted: primary outcome, secondary outcome. Primary outcome was measured with in-hospital mortality, and secondary outcome with in-hospital mortality or ICU length of stay ≥ 7 days (longer than 3 days, an index set by high-income countries ^{(6, 12, 13)}). AUROC was used to assess the efficiency of two systems, including specificity, sensitivity, positive (negative) predictive value. Delong method ⁽¹⁴⁾ was used to compare AUROC values, and multiple imputation to account for missing data in sensitivity analysis. In age-adapted SOFA, Glasgow coma scale score below 14 was considered the change of conscious status. R 3.4.3 was used for statistical analysis. P < 0.05 was considered statistically significant.

RESULTS

Study population

A total of 1918 children were PICU-admitted for infection or suspected infection, including 87 excluded and 1,831 included by our analysis (Fig. 1). Of these 1,831 children, 1,058 (57.8%) aged < 1 year. Of the total deaths, those aged < 1 year made up 51.8% (85) (Supplementary Figure 1, https://links.lww.com/SHK/A814). Of 1667 survived children, 1,503 (62.6%) were males (median age of 8.64 months, range of 2.93–25.03 months; median stay length of 6.94 days, range of 4.46–11.83 days), 164 (9.0%) experienced primary outcome, and 948 (51.8%) experienced secondary outcome. The 164 deaths included 108 males (65.9%) with a median age of 7.53 m (ranging from 2.67 to 41.00 months) (Table 1, supplementary Figure 2, https://links.lww.com/SHK/A814).

Fig. 1: Flowchart of enrollment in the present study.

Table 1: Demographic and clinic variable of survivors and non-survivors

SIRS criteria

Children meeting different numbers of SIRS criteria showed difference in death and survival (3.00[2.00, 3.00] vs. 2.00[1.00, 3.00]; p < 0.001; Table 1). As shown by Table 2, of all the PICU children, 1343 (73.3%) met ≥ 2 criteria. The mortality rose from 6.2% in children meeting < 2 criteria to 10.0% in children meeting ≥ 2 criteria (Figs. 2B and 3B). The AUROC of pediatric SIRS in predicting the possibility of primary outcome and secondary outcome was 0.578 and 0.538 (supplementary Table 1, https://links.lww.com/SHK/A814).

Table 2: Distribution of SIRS criteria and age-adapted SOFA score

Fig. 2: Distribution of patients by SOFA score and SIRS criteria at ICU admission.

Fig. 3: Mortality predicted by SOFA score and SIRS criteria at ICU admission.

Age-adapted SOFA results

According to age-adapted SOFA, 1693 (92.5%) children scored ≥ 2 (Table 2). The score showed difference between deaths and survivors (9.00 [7.00, 11.00] vs. 5.00[3.00, 7.00]; p < 0.001; Table 1). The mortality rose from 2.3% in children scoring <2 to 9.4% in children scoring ≥2 (Figs. 2A and 3A). Meanwhile, the proportion of children with secondary outcome rose from 23.2% to 54.1% (Supplementary Figure 3A, Supplementary Figure 4A, https://links.lww.com/SHK/A814). Score of ≥2 of age-adapted SOFA had a powerful ability to predict primary outcome and secondary outcome, with an AUROC of 0.738 and 0.687 (Supplementary Table 1, https://links.lww.com/SHK/A814).

Efficiency comparison between age-adapted SOFA and SIRS

Compared with SIRS, age-adapted SOFA was more accurate to predict primary outcome (AUROC, 0.757 vs. 0.603; p < 0.001) and secondary outcome (AUROC, 0.700 vs. 0.554; p < 0.001). In conjunction with baseline, the two systems got the similar accuracy in predicting primary outcome and secondary outcome (AUROC, [0.771, 0.733] vs. [0.698, 0.609], p < 0.001) (Table 3, Fig. 4).

Table 3: Crude and adjusted AUROC for discrimination characteristics of age-adapted SOFA and SIRS criteria on ICU-admitted children with infection

Fig. 4: Discriminatory capacity of SIRS criteria and SOFA score for primary outcome and secondary outcome using AUROC.

Sensitivity analysis

To reduce deviation in the results, sensitivity analysis was performed to adjust the above-mentioned results using different models. Age-adapted SOFA results tested with score of < 14 and score of < 15 were compared. In each condition, age-adapted SOFA was more accurate than SIRS (supplementary Table 2, https://links.lww.com/SHK/A814).

DISCUSSION

This is China's first evaluation on the accuracy of age-adapted SOFA and SIRS in predicting the prognosis of sepsis children. Compared with SIRS criteria, age-adapted SOFA score of ≥2 enjoys a higher accuracy in predicting primary outcome and secondary outcome, and a higher sensitivity in identifying children with severe infection.

Infection remains the biggest health threat in the children of low- or middle-income countries ⁽¹⁵⁾. We have reported a mortality of 9% in children admitted by PICU for serious infection, higher than that (5.8%) reported by Schlapbach et al. ⁽⁶⁾. Children of <5 years face a high risk of sepsis-induced death, and this age even drops in China: children aged <1 year, either patients or deaths, make up over 50% of the total PICU patients and deaths. In this research, the PICU children (> 60% being males) had a median age of just 8 months, and a longer stay length (6.94 days) than that in other two reports (1.7 days and 1.3 days) (6,16). However, all the analysis we reviewed only cover the children in developed countries. The epidemiological data on pediatric sepsis differs largely in countries with varying populations and medical resources ^(17–19). But in China, these data (like median age, gender proportion, and ICU stay length) show no obvious variance ^{(12, 13)}. So, there is a need to clarify the difference in sepsis definition set by different countries.

Accurate definition and diagnosis are the first two steps to decrease sepsis-induced mortality and carry out related researches. Ever since the release of pediatric sepsis definition 10 years ago, no modification has been made, indicating people's stagnant understanding on this disease. This definition needs refreshed, like Sepsis-3 set for adults ⁽²⁰⁾. Studies have suggested that age-adapted SOFA, for its superiority of over SIRS criteria in predicting septic prognosis, could function for children as Sepsis-3 does for adults. But compared with SIRS, age-adapted SOFA is more laboratory-dependent, which confines its replication in underdeveloped countries. In this research, the predictive abilities of two systems were externally validated. Compared with SIRS, age-adapted SOFA was more accurate in predicting primary outcome and secondary outcome; this is consistent with our previously reported results ^{(6, 16)}. Matics and Sanchez-Pinto ⁽¹⁶⁾ modified adult-oriented SOFA into a PELOD2-based SOFA and verified its higher predictive accuracy for in-hospital mortality (AUROC, 0.88). Schlapbach et al. ⁽⁶⁾ also confirmed that PELOD2-based SOFA was more accurate to predict in-hospital mortality than SIRS.

The current definition of pediatric sepsis, set based on SIRS in 2005 ⁽¹¹⁾, has shown an inability to predict poor prognosis ⁽²¹⁾, which may be caused by the inclusion of diseases without any organ dysfunction. Of SIRS-accorded children with fever and in emergency, only <5% need ICU care ⁽²²⁾. But once organ dysfunction is added into SIRS criteria, the 30-day mortality of blood-infected children will increase by 17 times ⁽²³⁾. So, using organ dysfunction indexes rather than SIRS can raise its accuracy in identifying children with severe infection. In our research, age-adapted SOFA (score of ≥2) demonstrated a higher sensitivity in identifying ICU children with severe infection. Our hospital, a leading medical institution in south China, charges over 80% of children in critical condition. With a higher PRISM III score than that reported by Matics (13 vs. 2) ⁽¹⁶⁾, the samples in our research could represent the total patients of this kind in this area. Most of the children admitted by our hospital presented organ dysfunction, and 92.5% of them scored ≥2 according to age-adapted SOFA. So, this system could also be used as a stratifying tool to direct the management of ICU.

Although highly laboratory-dependent, age-adapted SOFA is a feasible tool to define pediatric sepsis in China, especially in advanced hospitals (like Guangzhou Women and Children's Medical Center) capable of carrying out these routine laboratory examinations. The first in low- or middle-income countries, this research also confirms the usefulness of age-adapted SOFA in improving ICU management. In addition, the reliability of this research is guaranteed with two measures: inclusion of all infected children during the 9 years before, and the codiagnosis by two PICU specialists for each child.

This research has also limitations. First, it is a single-center retrospective analysis; multicenter should be covered. Second, some children with severe infection were initially treated at other hospitals, including county or township hospitals. In some cases, the onset of infection was not clarified. Third, in the newly-developed age-adapted SOFAs, cardio-vascular and renal variables are calculated with modified age cutoffs. We did not evaluate the performance of another pediatric SOFA in this study. Studies on SOFA based on different cut-offs should be conducted to assess its clinical utility.

In conclusion, compared with SIRS criteria, age-adapted SOFA score of ≥2 is more accurate to predict in-hospital mortality of China's PICU children, and can be taken as an identifying tool in managing ICU. But whether this system can be transplanted into pediatric sepsis definition needs to be answered with more researches.