Organ Dysfunction in Children With Blood Culture-Proven Sepsis: Comparative Performance of Four Scores in a National Cohort Study

OBJECTIVES: Previous studies applying Sepsis-3 criteria to children were based on retrospective analyses of PICU cohorts. We aimed to compare organ dysfunction criteria in children with blood culture-proven sepsis, including emergency department, PICU, and ward patients, and to assess relevance of organ dysfunctions for mortality prediction. DESIGN: We have carried out a nonprespecified, secondary analysis of a prospective dataset collected from September 2011 to December 2015. SETTING: Emergency departments, wards, and PICUs in 10 tertiary children’s hospitals in Switzerland. PATIENTS: Children younger than 17 years old with blood culture-proven sepsis. We excluded preterm infants and term infants younger than 7 days old. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We compared the 2005 International Pediatric Sepsis Consensus Conference (IPSCC), Pediatric Logistic Organ Dysfunction-2 (PELOD-2), pediatric Sequential Organ Failure Assessment (pSOFA), and Pediatric Organ Dysfunction Information Update Mandate (PODIUM) scores, measured at blood culture sampling, to predict 30-day mortality. We analyzed 877 sepsis episodes in 807 children, with a 30-day mortality of 4.3%. Percentage with organ dysfunction ranged from 32.7% (IPSCC) to 55.3% (pSOFA). In adjusted analyses, the accuracy for identification of 30-day mortality was area under the curve (AUC) 0.87 (95% CI, 0.82–0.92) for IPSCC, 0.83 (0.76–0.89) for PELOD-2, 0.85 (0.78–0.92) for pSOFA, and 0.85 (0.78–0.91) for PODIUM. When restricting scores to neurologic, respiratory, and cardiovascular dysfunction, the adjusted AUC was 0.89 (0.84–0.94) for IPSCC, 0.85 (0.79–0.91) for PELOD-2, 0.87 (0.81–0.93) for pSOFA, and 0.88 (0.83–0.93) for PODIUM. CONCLUSIONS: IPSCC, PELOD-2, pSOFA, and PODIUM performed similarly to predict 30-day mortality. Simplified scores restricted to neurologic, respiratory, and cardiovascular dysfunction yielded comparable performance.


F
ollowing the 1992 and 2001 consensus statement of the American College of Chest Physicians, pediatric sepsis was defined through expert opinion in 2005 by the International Pediatric Sepsis Consensus Conference (IPSCC) as infection in presence of at least two out of four criteria of systemic inflammatory response syndrome (SIRS) (1,2).Severe sepsis was defined as sepsis with organ dysfunction, with more weight given to cardiovascular and respiratory dysfunction (3).These pediatric "Sepsis-2" definitions remain the only ones available for this age group, yet they have not been adequately validated (3).
The recent Sepsis-3 consensus definition demonstrated that infected adult patients with organ dysfunction, as measured by the Sequential Organ Failure Assessment (SOFA) score, had higher mortality than those without organ dysfunction, implying higher utility of the SOFA score compared with SIRS for mortality prediction (4).The 2020 pediatric Surviving Sepsis Campaign accordingly used the terms "sepsis-associated organ dysfunction" and "septic shock" to delineate sepsis, rather than "severe sepsis" (5).Given the limitations of the IPSCC, criteria for pediatric sepsis are currently being revised, and a global survey among pediatricians identified that the concept of sepsis-associated organ dysfunction is widely accepted to distinguish sepsis from infection (6)(7)(8).Yet, controversy surrounds the optimal operationalization of organ dysfunction scoring in children with infection (9).Most previous cohorts on pediatric sepsis included either children with suspected infections, lacking comprehensive microbiological diagnostics, or children with microbiological results that may represent either colonization, co-infection, or primary viral infection (10)(11)(12)(13)(14). Furthermore, most studies were performed in PICUs, and the generalization of findings to less acute cohorts, such as children in emergency department (ED) or ward settings, remains poorly understood (15,16).
Therefore, we have used a curated dataset from 2011 to 2015 population-based multicenter prospective cohort study, with clearly defined, gold standard phenotype of blood culture-proven sepsis (2), to test the predictive performance of existing scoring systems to identify children at higher mortality risk.

Study Design
We have carried out a nonprespecified secondary analysis of data collected by the Swiss Pediatric Sepsis Study, 2011Study, -2015 (17) (17).This prospective, observational multicenter cohort study investigated blood culture-proven sepsis, as defined by IPSCC definitions (2), in children in Switzerland.All ten major children's hospitals in the country participated in the study.These sites accounted for 78% of all pediatric hospital admissions and 98% of all PICU admissions with an International Classification of Diseases, 10th Revision code for pathogen-specific sepsis in children in Switzerland (17).The current analysis was approved by the ethics committees of all participating centers (Cantonal Ethics Committee Bern, approval number KEK-029/11, approval date February 20, 2018, study title "Swiss Pediatric Sepsis Study-Impact of innate immunity on susceptibility to sepsis in neonates and children") and the study was conducted in accordance with the ethical principles described in the Declaration of Helsinki.We followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline (18).

Cohort Characteristics
In 2011-2015, infants and children presenting with blood culture-proven bacterial or fungal sepsis, as defined by IPSCC definitions (2), were eligible if they were recruited to the Swiss Pediatric Sepsis Study with an age between older than 7 days and younger than 17 years at sepsis onset and if patient files to verify organ dysfunction were available.We excluded preterm neonates born younger than 37 weeks and neonates with early-onset sepsis at younger than 7 days of life.• While adult sepsis criteria are based on the SOFA score, pediatric sepsis remains defined by IPSCC criteria, which were crafted in 2005.

RESEARCH IN CONTEXT
• In view of the planned revision of pediatric sepsis criteria, there is a need for robust evaluation of organ dysfunction score performance in children with confirmed sepsis.

Outcome
The primary outcome used in the current analysis was 30-day (in-hospital) mortality.The composite secondary outcome was defined as 30-day (in-hospital) mortality or PICU length of stay of 3 days or longer after blood culture sampling (19).Data on demographics, perinatal and other risk factors, comorbidities, infection site, severity, and outcome were entered prospectively into an online database.For the current analysis, we post hoc categorized comorbidities according to the pediatric complex chronic conditions classification system version 2 (20).

Organ Dysfunction Scores
In the original study (17), investigators prospectively categorized every patient as having sepsis, severe sepsis, or septic shock, according to IPSCC criteria (2).For this curated dataset, we additionally collected the variables required to define, post hoc, individual organ dysfunctions according to IPSCC (2), Pediatric Logistic Organ Dysfunction-2 (PELOD-2) (21), pediatric SOFA (pSOFA) (22), and Pediatric Organ Dysfunction Information Update Mandate (PODIUM) (23) (Supplementary Methods, http://links.lww.com/PCC/C434).For every parameter, the worst value documented within the same calendar day during which the blood culture was obtained was collected.If a laboratory parameter had not been performed (such as lactate or arterial blood gas), it was assumed to be normal, which is aligned with contemporary scoring methodology (21).For PELOD-2, pSOFA, and PODIUM, in addition to using the full scores, we constructed a binary (yes/no) variable for the presence of each individual organ dysfunction, defined as yes if the score summary of all variables contributing to a given organ were greater than 0. Accordingly, we constructed five (PELOD-2), six (IPSCC and pSOFA), and eight (PODIUM) individual organ dysfunctions as binary variables.

Statistics
Descriptive statistics are presented as median and interquartile ranges (IQRs) for continuous variables, and frequencies and percentages for categorical variables.
We assessed the agreement between the four scores in relation to adjudication of specific organs affected by calculating the relative agreement and Krippendorff 's alpha, with 95% CIs derived from 5000 bootstrap samples (24).We considered a Krippendorff 's alpha greater than 0.80 as good and greater than 0.67 as acceptable agreement (24).We then assessed the discriminative power of the scores for the primary and secondary outcome using area under the curve (AUC) of receiver operating characteristic curves (25).To adjust the AUCs for age, sex, and presence of chronic medical conditions, we fitted logistic mixed-effects models with a random intercept for each hospital, using the linear predictor of the fixed effects for further analysis by receiver operating characteristic curves and corresponding AUCs.We estimated 95% CIs using DeLong's method (26).We assessed the calibration of adjusted models with the Hosmer-Lemeshow C*statistic ( 27) and Cox's calibration regression (28), and the overall fit of the models with Brier score (29).We then examined the contribution of individual organs for the prediction of the primary and secondary outcome for the IPSCC, PELOD-2, pSOFA, and PODIUM scores, respectively.For this analysis, we used conditional random forest analyses that are unbiased in the presence of variables with different number of levels (30).The relevance of the individual organs within a score for the prediction of the outcome was judged by the permutation importance (31).We additionally analyzed the IPSCC and the binarized PELOD-2, pSOFA, and PODIUM scores with logic regression models that find the optimal combination of organs via crossvalidation of logic trees.We did sensitivity analyses using only the first sepsis episode in every patient, to account for the fact that a patient could experience more than one sepsis episode.Further details on the statistical analyses are presented in the Supplementary Methods (http://links.lww.com/PCC/C434).All analyses were conducted with R Version 4.1.2(32).

Cohort Description
We analyzed 877 sepsis episodes in 807 patients (Fig. Across all four scores, mortality increased incrementally with higher score values (Fig. 2), as well as with higher number of organs affected (Fig. S3, http:// links.lww.com/PCC/C434).Across all four scores, the proportion of children with the secondary outcome mortality and/or PICU length of stay of greater than or equal to 3 days increased incrementally with higher score values (Fig. S4, http://links.lww.com/PCC/C434), as well as with higher number of organs affected (Fig. S5, http://links.lww.com/PCC/C434).

Contribution of Individual Organ Dysfunctions to Mortality
We then assessed importance of individual organ dysfunctions as defined by IPSCC, PELOD-2, pSOFA, and PODIUM for the primary outcome using conditional random forest analyses.Cardiovascular, respiratory, and neurologic dysfunction were consistently the most relevant organ dysfunctions across all four organ dysfunction scores, while hepatic dysfunction was relevant in IPSCC and PODIUM but not pSOFA (Fig. 4).These results were confirmed in logic regression for IPSCC and binarized PELOD-2, while respiratory, neurologic, and hepatic dysfunction were most relevant for the binarized pSOFA.We could not derive a discriminative model from the binarized PODIUM using logic regression.Results were confirmed in sensitivity analyses (Fig. S9, http://links.lww.com/PCC/C434).
We then simplified IPSCC, PELOD-2, pSOFA, and PODIUM by only considering information on

DISCUSSION
This secondary analysis of a curated dataset from 2011 to 2015 population-based multicenter prospective cohort study of children with blood culture-proven sepsis recruited across ED, ward, and PICU settings confirms an association between organ dysfunction and 30-day mortality.When comparing four pediatric scores for organ dysfunction, discrimination of the primary and secondary outcome did not substantially differ, however, we observed major differences in terms of classification as to whether organ dysfunction was present, and as to which organ was dysfunctional.Cardiovascular, respiratory, and neurologic dysfunction emerged as the most relevant organs contributing to a higher risk of mortality.
The relevance of organ dysfunction to identify children with higher disease severity is biologically evident and has been confirmed in multiple studies primarily in intensive care settings.Yet, direct comparisons of different approaches to adjudicate organ dysfunction, and understanding the impact of differences between scores, have received less attention in critically ill children (9).The Sepsis-3 criteria in adults give all organ dysfunctions similar weight and operationalize organ dysfunction by the SOFA score (4).In contrast, the IPSCC "severe sepsis" criteria ranked cardiovascular and respiratory dysfunction higher than other organ dysfunctions and provided specific criteria for individual organ dysfunction rather than using a standardized score (2).While the original purpose was to delineate children with severe sepsis to be enrolled in an interventional trial from children with sepsis without organ dysfunction, these criteria have been widely used amongst children with sepsis.The IPSCC was formulated by a panel of experts and has never undergone revision nor independent validation (3,33).The Pediatric Sepsis Definition Taskforce is expected to develop and validate new criteria for pediatric sepsis (6,15).Interestingly, the variation in performance across the scores was only modest, despite substantial differences in how these scores were developed and constructed.PELOD-2 was derived from a French-Belgian multicenter study including 3671 children admitted to PICU between 2006 and 2007, and used weighting based on the effect size observed in the model (21).In comparison, the pSOFA was created combining the structured incremental SOFA score widely used in

AT THE BEDSIDE
• In this population-based study, IPSCC, PELOD-2, pSOFA, and PODIUM, assessed at blood culture sampling, achieved a similar prediction of 30-day mortality in children with proven sepsis.
• Although we observed substantial differences in individual adjudication of the type of organ dysfunction between scores, neurologic, respiratory, and cardiovascular dysfunction were consistently most relevant to predict 30-day mortality.
Copyright © 2024 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.Unauthorized reproduction of this article is prohibited

Online Clinical Investigations
Pediatric Critical Care Medicine www.pccmjournal.orge125 adults with PELOD-2 based age-specific cutoffs and has been subsequently validated in additional studies (14,22).Finally, the recent PODIUM criteria represent the product of extensive systematic literature reviews and expert opinion, incorporating some age-specific cutoffs derived from other studies (such as creatinine) (9,23).Importantly, the use of these scores, except for IPSCC, has been largely confined to PICU settings, and validation, where available, was almost exclusively performed in PICU cohorts (15,34).To date, the majority of ED and ward-based studies investigating identification of sicker children have focused on Early Warning Tools, which commonly use vital signs but only rarely provide measures of organ dysfunction (35,36).Similar to the adult quick SOFA score (37), we found that cardiovascular, respiratory, and neurologic dysfunction were the most relevant organ dysfunctions in respect to prediction of the primary outcome across all four organ dysfunction scores.Our observation is supported by a post hoc analysis of a large randomized fluid trial on children with infection (38), and by a secondary analysis of an international pediatric sepsis point prevalence study (39).Interestingly, when we limited organ dysfunction scores to these three organ dysfunctions, the performance in unadjusted and adjusted analyses were surprisingly similar in comparison to the respective full organ dysfunction score.Importantly, cardiovascular, respiratory, and neurologic dysfunctions are readily measurable during clinical examination, which facilitates their assessment in ED and in smaller hospitals without dedicated PICU or in less resourced healthcare settings (6).However, our findings do not preclude that the presence of other organ dysfunction, such as renal failure, in particular, in higher acuity PICU cohorts, may contribute additionally to worse prognosis for short-and longterm outcomes (40).
In contrast to previous epidemiological studies on organ dysfunction in pediatric sepsis, the present study was based on data from a population-based multicenter prospective cohort designed to investigate blood culture-proven sepsis, with all patients meeting IPSCC sepsis criteria.The study captured both community-and hospital-acquired sepsis and is representative of a broad range of comorbidities and clinical settings in a high-income country (11)(12)(13).Sample size in subgroups was small, however, precluding separate analysis of PICU and ED populations.In addition, the focus on blood culture-proven sepsis allowed studying a clearly defined infectious disease phenotype of high relevance.
A number of limitations need to be considered.First, we excluded premature babies and those with early-onset sepsis, given the intrinsic difficulties in applying organ dysfunction criteria to this age group (41).Additionally, patients after allogeneic bone marrow transplants were not included.Second, we did not develop or validate specific thresholds but used published organ dysfunction criteria.Third, not all criteria included in the full PODIUM score were available, which may have decreased its performance.Fourth, we did not consider deaths occurring greater than 30 days, nor functional or quality of health outcomes (42).Fifth, the study only included children with blood culture-proven sepsis and our findings may not generalize to sepsis caused by viral infections or culturenegative sepsis.Finally, our findings date back to a cohort recruited in 2011-2015 and may not be generalizable to other healthcare settings, in particular, to low-or middle-income countries.

CONCLUSIONS
In conclusion, while the performance of different organ dysfunction scores was similar to predict 30-day mortality, the substantial discrepancies in terms of organ dysfunction adjudication implies a lack of comparability and provides a strong rationale for future tailoring of these scores toward a single score informed by the analysis of large international databases (6, 16).
Our findings suggest that derived, simplified organ dysfunction criteria may yield comparable performance, facilitating application toward less resourced and nonintensive care settings.

Figure 1 .
Figure 1.Upset demonstrating adjudication of presence of an organ dysfunction affected by each of the four organ dysfunction scores.Upset (43) displaying intersections of the four organ dysfunction scores (sets).The lower shows a matrix layout for all intersections, with dark circles indicating a sets is part of the intersection (i.e., presence of an organ dysfunction as defined by each of the four organ dysfunction scores).The upper shows the size of intersections.IPSCC = International Pediatric Sepsis Consensus Conference, PELOD-2 = Pediatric Logistic Organ Dysfunction-2, PODIUM = Pediatric Organ Dysfunction Information Update Mandate, pSOFA = pediatric Sequential Organ Failure Assessment.

Figure 2 .
Figure 2. Proportions of patients meeting the primary outcome (30-d mortality) in relation to score value, according to the four organ dysfunction scores used.Proportion of episodes with the primary outcome for International Pediatric Sepsis Consensus Conference (IPSCC), Pediatric Logistic Organ Dysfunction-2 (PELOD-2), pediatric Sequential Organ Failure Assessment (pSOFA), and Pediatric Organ Dysfunction Information Update Mandate (PODIUM).The x-axes denote the number of organs affected (IPSCC) or the total score of each of the four organ dysfunction scores.Numbers above bars indicate number with the primary outcome/total number.

Figure 3 .
Figure 3. Area under the receiver operating characteristics curve analyses to predict the primary outcome (30-d mortality) for each of the four organ dysfunction scores used.Unadjusted (A), and analyses adjusted for age, sex, and presence of comorbidities, with a random effect per study site (B) are shown.IPSCC = International Pediatric Sepsis Consensus Conference, PELOD-2 = Pediatric Logistic Organ Dysfunction-2, PODIUM = Pediatric Organ Dysfunction Information Update Mandate, pSOFA = pediatric Sequential Organ Failure Assessment.

Figure 4 .
Figure 4. Importance of individual organ dysfunctions for the prediction of the primary outcome (30-d mortality) for each of the four organ dysfunction scores used.Permutation importance of individual organ dysfunctions from conditional random forest analyses of 2005 International Pediatric Sepsis Consensus Conference (A), Pediatric Logistic Organ Dysfunction-2 (B), pediatric Sequential Organ Failure Assessment (C), and Pediatric Organ Dysfunction Information Update Mandate (D).A higher value indicates higher importance compared with the other score items.Irrelevant covariates display permutation importance close to 0 or negative values.
In 442 of 877 episodes (50.4%), patients were previously healthy, and 227 of 877 episodes (25.9%) were classified as hospitalacquired.The median length of stay in hospital was 14 days (IQR, 8-28 d).In 289 of 877 episodes (33.0%), admission to PICU was required with a median PICU length of stay of 7 days (IQR, 2-30 d).The primary outcome, 30-day (in-hospital) mortality, occurred in 38 of 877 episodes (4.3%), the secondary outcome, 30-day (in-hospital) mortality or PICU length of stay of 3 days or longer after blood culture sampling, in 226 of 877 episodes(25.8).The median time to death after obtaining the blood culture was 6 days (IQR, 1-16.5 d).

TABLE 1 .
Measures of Calibration and Discrimination of Adjusted Models to Predict the Primary Outcome (30-d Mortality) inChildren With Sepsis

e127
Dr. Heininger received funding from Global Pertussis Initiative and the Central and Eastern Europe Pertussis Awareness Group, Infectopharm, Merck, Moderna, Pfizer, Roche, Sanofi, and Sanofi Pasteur; he disclosed that he is on the Data Monitoring Committee, the phase II study of adjuvanted pandemic influenza vaccine, the Data Monitoring Committee Norovirus Bivalent Virus-like particle Vaccine Program, the Meta Data Safety Monitoring Board for Coalition for Epidemic Preparedness Innovations, the Data Monitoring Committee, the cell culture influenza vaccine, and the Varicella Advisory Board.Dr. Riedel's institution received funding from Sentec AG.Dr. Agyeman's institution received funding from the Swiss National Science Foundation (342730_153158/1 and 320030_201060/1), the Swiss Society of Intensive Care, the Bangerter Foundation, the Vinetum and Borer Foundation, the Foundation for the Health of Children and Adolescents, and the Sanofi-Aventis Suisse; he received support for article research from the Swiss National Science Foundation.The remaining authors have disclosed that they do not have any potential conflicts of interest.Drs.Schlapbach, Goertz, Berger, and Agyeman contributed equally.Drs.Schlapbach, Berger, and Agyeman conceptualized and designed the study, coordinated and supervised data collection, drafted the initial article, and reviewed and revised the article.Dr. Goertz contributed to study design, coordinated data collection, drafted the initial article, and reviewed and revised the article.Dr. Hagenbuch carried out the initial analyses, drafted the initial article, and reviewed and revised the article.Drs.Aubert, Papis, Giannoni, Posfay-Barbe, Stocker, Heininger, Bernhard-Stirnemann, Niederer-Loher, Kahlert, Natalucci, Relly, Riedel, and Aebi contributed to study design, collected data, and reviewed and revised the article.All authors approved the final article as submitted and agree to be accountable for all aspects of the work.