Rapid administration of antibiotics is the most important intervention in sepsis (1) and requires early recognition of infection in the emergency department (ED). However, previous studies have found low accuracy of infection biomarkers in severely ill patients (2). Validated biomarkers for clinical guidance of early antibiotic treatment are therefore highly warranted, especially in ED patients with possible sepsis (3).
Hitherto, no biomarkers are recommended for identification of infection in patients with suspected sepsis (2). The established biomarkers C-reactive protein (CRP) and procalcitonin (PCT) have been well characterized in many patient populations (3,4). However, less is known about their ability to diagnose early infections in severely ill patients in the ED. IL-6, a cytokine with an important role in early inflammation cascades in infection (5), has shown great variation in diagnostic accuracy as a biomarker in ED patients (5,6). Its clinical utility in the ED is therefore still unclear. Calprotectin, a protein complex stored in abundance in neutrophils (7), is released upon activation (8) and could therefore serve as an early marker for acute inflammation as shown in in vitro (9), in vivo (10), and clinical studies (11–13). Biomarkers aid clinical decision-making in combination with clinical findings. However, few studies have so far assessed the added value of biomarkers when combined with clinical findings or judgment by ED physicians in the diagnosis of infections (14–16).
Our hypothesis was that relevant inflammatory biomarkers, alone or in combination, could improve early infection recognition in patients with possible sepsis in the ED, thus improving antibiotics prescription. The primary aims of this study were to compare the diagnostic accuracy of the four infection biomarkers calprotectin, CRP, IL-6, and PCT and to investigate whether these biomarkers could provide added diagnostic value to the clinician treating patients with possible sepsis in the ED. A secondary aim of this study was to investigate the biomarkers' ability to discriminate between different types of infections, as well as their ability to predict sepsis and overall mortality. The tertiary aim was to investigate the dynamics of the biomarkers during the first 7 days of hospitalization.
MATERIALS AND METHODS
Study design and setting
This prospective observational study recruited patients from all admissions assessed by medical or sepsis rapid response teams (RRTs) throughout 2020 in the ED at Oslo University Hospital, a large tertiary hospital (Fig. 1A). Both RRTs assess patients with possible sepsis. Patients assessed in specific RRTs devoted to trauma, surgical emergencies, ST-elevation myocardial infarction, cardiac arrest, and cerebral stroke were not included.
All included patients had objective signs of organ dysfunction or otherwise suspected severe disease (criterion details in Supplementary Table 1, https://links.lww.com/SHK/B502). Study participants were consecutively included, and clinical data were registered in a local database (MedInsight, Oslo, Norway). As the aim of the study was to investigate the diagnostic accuracy of the various biomarkers as well as ED clinician's (EDC's) assessment of infection (defined by the prescription of antibiotics), patients with any biomarker result available to the clinician before administration of antibiotics were excluded from further analysis.
Patients admitted to the intensive care unit (ICU) from April 24 to December 14 and a random selection of hospital ward patients were also included in a cohort with daily sampling during 7 days or until death or discharge (Fig. 1B). Patients discharged at less than 48 h after admission or with more than two missing samples were excluded.
Informed consent was obtained from all patients or next of kin if the patient was unable to consent. The study was approved by the Regional Committee for Medical and Health Research Ethics in Southeastern Norway (2019/306) and registered at ClinicalTrials.gov (NCT03956043). The study is reported according to the STARD (Standards for Reporting Diagnostic Accuracy Studies) guidelines (17).
Cultures were collected from blood and/or tissues on clinical indication. Systemic inflammatory response syndrome criteria, National Early Warning Score, Sequential Organ Failure Assessment (SOFA) and quick SOFA (qSOFA) in the ED, as well as the highest SOFA during the hospitalization, were calculated. Sepsis was defined according to Sepsis-3 as an acute increase in SOFA ≥2 from habitual state with infection likelihood assessed as “probable” or “definite” infection (18). Charlson Comorbidity Index was calculated (19).
The EDCs' decision to use antibiotics was registered. According to the standard operating procedures at the time of inclusion, antibiotics should be administered within 60 min after arrival of patient with suspected sepsis (20), which is before biomarker analysis is completed.
Diagnostic test methods: biomarker measurements
Venous or arterial blood was collected within 15 min of arrival in the ED and centrifuged at 1980g for 10 min. C-reactive protein and calprotectin were with some exceptions (<10 samples) analyzed within 2 h. Plasma was transferred to a refrigerator (4°C) within 2 h and transferred to cryotubes the next working day for storage at −80°C. Procalcitonin (except for 41 samples that were analyzed real-time) and IL-6 were analyzed in thawed plasma according to the manufacturer's instructions. Instruments, reagents, calibrators, and reference intervals are described in Supplementary Table 2, https://links.lww.com/SHK/B502. Creatinine, lactate, platelets, and bilirubin were analyzed as previously described (21).
Outcomes: infection likelihood and mortality
Infection likelihood was retrospectively assessed by an infectious disease specialist after discharge or death and classified as “not likely,” “probable,” or “definite” (Fig. 1C) (22,23). The following criteria were used: “not likely”: no sign of infection and a noninfectious diagnosis established; “probable”: infection was considered to be likely, but without microbiological confirmation; and “definite”: microbiologically confirmed either by detection of strict pathogens (pathogen not part of normal flora) or detection of pathogens in otherwise sterile body fluids or tissues. Date of death was collected from the Norwegian National Population Register.
Statistical analyses were performed using IBM SPSS Statistics version 26 (IBM, Armonk, NY) if not otherwise stated. Patients missing biomarker results were excluded. Comparing the three infection likelihood groups, dichotomous variables were analyzed by Pearson chi-square tests, whereas continuous variables were analyzed by one-way ANOVA (normally distributed) and Kruskal-Wallis (non–normally distributed) tests. The groups “probable” and “definite” infection were merged into the group “infection” for most analyses. The area under the receiver operating characteristic (ROC) curve (AUC) was estimated for patients with and without infection. Youden index was used to determine the optimal biomarker cutoff in our cohort. Sensitivity, specificity, negative (NPV) and positive (PPV) predictive values, and positive (LR+) and negative (LR−) likelihood ratios were calculated using MedCalc Statistical Software version 20.023 (Ostend, Belgium) for the reference and optimal cutoff values and the lowest value reaching PPV of 95% or greater, if possible. Biomarker levels were log10-transformed for regression analyses. Normality was visually assessed. Univariate and multivariate binary logistic regression was applied to assess the combined ability of EDC and biomarkers to differentiate infected and noninfected patients. Model fit was evaluated using the Hosmer-Lemeshow test; models with P > 0.05 were accepted. To identify the best regression model, all four biomarkers and the EDC were included and then backward eliminated using the LR method. The regression models' probability predictions were used to create ROC curves and AUC calculations, whereas predicted group membership (“infection” or “not likely” infection) was used as basis for calculations of sensitivity, specificity, NPV, PPV, LR+, and LR−. Associations between biomarkers in the ED were determined by linear regression. Decision curve analysis was performed in R version 4.1.1 (R Core Team, 2021, Vienna, Austria) (24,25). Longitudinal biomarker level variations were examined using Wilcoxon signed-rank test. Spearman rank correlation coefficient was computed for IL-6 and CRP in the longitudinal samples. Based on AUC from other studies (4,11), a sample size of 418 was needed to differentiate AUCs of PCT and calprotectin (δ = 0.07) with a confidence level of 95% and 80% power (26).
Characteristics of study subjects
A total of 391 admissions to the ED were included in the analysis (Fig. 1A). Based on the infection likelihood assessment, 154 patients were classified as “not likely” (diagnoses listed in Supplementary Table 3, https://links.lww.com/SHK/B502), 135 as “probable,” and 102 as “definite” infection (Fig. 1C). Mortality and sex distribution was comparable across the groups (Supplementary Table 4, https://links.lww.com/SHK/B502). However, the “definite” and “probable” groups were older and had higher Charlson Comorbidity Index, qSOFA, and systemic inflammatory response syndrome criteria and longer hospital stays, and fewer were admitted to the ICU. Sepsis was present in 177 admissions (45%) (Supplementary Table 4, https://links.lww.com/SHK/B502).
None of the biomarkers were associated with 30-day mortality in this cohort (Supplementary Table 5, https://links.lww.com/SHK/B502). Calprotectin, CRP, IL-6, and PCT were all significantly increased in the two infection groups compared with the “not likely” infection group (Fig. 2). In further analyses, the categories “definite” and “probable” infection were merged into the group “infection” (n = 237). In this composite group, 216 cases (91%) were correctly assessed by the EDC to have an assumed infection as defined by the prescription of antibiotics (Table 1, A). However, 43 of the 154 patients (28%) with “not likely” infection were also treated with antibiotics.
Table 1 -
Emergency department clinician and biomarker test abilities
|Test and cutoff values
||AUC (95% CI)
||Sensitivity % (95% CI)
||Specificity % (95% CI)
| >1.69 mg/LRef
| ≥1.96 mg/LO
| ≥4 mg/LRef
| ≥31 mg/LO
| ≥50 mg/LPPV
| ≥7 pg/mLRef
| ≥52 pg/mLO
| ≥194 pg/mLPPV
| ≥0.1 μg/LRef
| ≥0.22 μg/LO
| ≥0.4 μg/LPPV
| EDC + calprotectin†
| EDC + CRP†
| EDC + IL-6†
| EDC + PCT†
| EDC + CRP + IL-6†
Area under the ROC curve, sensitivity, specificity, NPV, PPV, and LR−, and LR+ for the EDC's decision to treat with antibiotics and at different cutoff values for the biomarkers calprotectin, CRP, IL-6, and PCT (A). Emergency department clinician in combination with single biomarkers and the best combination of biomarkers (CRP and IL-6) (B).
*Performed using univariate logistic regression; all values here except AUC are computed from the regression models' predictions.
†Multiple logistic regression; all values here except AUC are computed from the regression models' predictions.
Ref, reference value; O, optimal cutoff value in our cohort as defined by the Youden J index; PPV, lowest cutoff value to reach a PPV of ≥95% (calprotectin could not reach PPV ≥95%).
IL-6 and CRP are best at discriminating patients with and without infection
The biomarkers' ability to discriminate between patients with and without infection was assessed using ROC curves (Fig. 3). IL-6 and CRP had a significantly higher AUC than PCT and Calprotectin (P < 0.001). (Table 1, A) displays each biomarker's optimal cutoff value and diagnostic abilities as biomarkers for infections at different cutoff values. At their optimal cutoff value, the specificities were higher than the EDC for all biomarkers except calprotectin. C-reactive protein (NPV, 93%) and IL-6 (NPV, 97%) could exclude infections with high probability at reference value (CRP <4 mg/L and IL-6 <7 pg/mL), whereas calprotectin and PCT did not reach NPV greater than 75% at any cutoff value. In contrast, PCT (≥0.4 μg/L), IL-6 (≥194 pg/mL), and CRP (≥50 mg/L) could all predict infections (95% PPV). At these high levels, CRP demonstrated higher sensitivity for infection compared with PCT and IL-6. Furthermore, IL-6 and CRP were superior as biomarkers for infection both in patients with qSOFA of 2 or greater (n = 163) and less than 2 (n = 228) in the ED (Supplementary Table 6a, https://links.lww.com/SHK/B502).
Biomarkers improve the clinical diagnostic specificity for infection in the ED
Logistic regression was used to investigate whether single biomarkers could add clinical value to the EDC to treat suspected infection with antibiotics. Univariate logistic regression was applied to compute the ROC curve for the EDC alone, whereas bivariate logistic regression was used for the EDC and each biomarker combined (Fig. 3). Comparing AUCs, each biomarker improved the EDC in discriminating patients with and without infection (P < 0.001 for all biomarkers). However, the specificities and sensitivities were modestly improved (Table 1, B).
Combination of CRP, IL-6, and EDC best identified patients with infections
To identify the best model for predicting the presence of infection, we performed multivariate logistic regression with backward elimination including all four biomarkers and the EDC. The best performing model included the EDC, IL-6, and CRP (Fig. 3 and Table 1, B). The performance of the model was largely unchanged when stratified by qSOFA of less than 2 and 2 or greater (Supplementary Table 7, https://links.lww.com/SHK/B502).
Clinical value of this regression model was further illustrated by decision curve analysis (Fig. 4) (25), a visual analysis of the net benefit of diagnostic approaches against decision thresholds. In this case, the decision was whether to start empirical antibiotic therapy. For example, 0.05 as the decision threshold indicates a willingness to withhold antibiotics if the estimated risk of infection is less than 5%. The biomarkers added net benefit mostly to patients with qSOFA of less than 2 (Fig. 4). With increasing decision thresholds, the net benefit of adding biomarkers to the EDC compared with treating all patients improved, while still being superior to treating based on the EDC alone.
CRP and IL-6 identify undertreated and overtreated patients
As CRP and IL-6 emerged as the best-performing biomarkers, we assessed if their optimal cutoff values could be applied in combination to recognize infection. In fact, CRP and IL-6 was the only pair of biomarkers that did not correlate in the initial ED samples (Supplementary Fig. 1, https://links.lww.com/SHK/B502). If both IL-6 and CRP were greater than the optimal cutoff value (≥52 and ≥31, respectively), infection was almost certain (PPV, 95%). On the other hand, if CRP and IL-6 were less than the optimal cutoff value, the NPV was 90% for all patients and 98% for those with qSOFA of 2 or greater (Supplementary Table 6b, https://links.lww.com/SHK/B502).
According to our hypothesis, we next examined whether the combination of CRP and IL-6 optimal cutoff values could provide added value to the EDC. Based on the post hoc infection likelihood assessment, the EDC undertreated infections in 21 of 391 cases (5%) (Fig. 5). Among these, CRP and IL-6 levels greater than the optimal cutoff values were found in 13, of which 10 (77%) had a “definite” or “probable” infection. Inversely, among the patients who received antibiotics in the ED, 29 patients had CRP and IL-6 values of less than the optimal cutoff values. Among these, infection was “not likely” in 22 (76%) (Fig. 5). In conclusion, CRP and IL-6 identified 32 of 65 patients (49%) who were undertreated or overtreated, implying that these two biomarkers could aid the clinical decision-making process.
Higher levels of calprotectin in gram-positive versus gram-negative bacterial infections
We next compared biomarkers levels in viral, gram-positive and gram-negative bacterial infections (Supplementary Fig. 2, https://links.lww.com/SHK/B502). Interestingly, calprotectin was higher in gram-positive compared with gram-negative bacterial infections (P < 0.001). IL-6 (P = 0.002) and PCT (P = 0.010), but not CRP (P = 0.830) or calprotectin (P = 0.237), were higher in bacterial compared with viral infections. A list of all pathogens is provided in Supplementary Table 8, https://links.lww.com/SHK/B502.
Plasma IL-6 levels peak at admission to ED
We explored biomarker dynamics during the first 7 days of hospitalization in a subgroup of ED patients (n = 52), predominantly admitted to the ICU (Supplementary Fig. 3, https://links.lww.com/SHK/B502). The characteristics of this patient subgroup are available in Supplementary Table 9, https://links.lww.com/SHK/B502. All patients with infection received antibiotics within 24 h of hospitalization. In these patients, IL-6 decreased from the ED (day 1) to day 2 (P = 0.004) and further to day 3 (P < 0.001) (Fig. 6). In contrast, CRP and PCT increased nonsignificantly after ED admission, peaking at day 2. Whereas PCT and calprotectin levels significantly decreased from day 2 (P = 0.006 and P = 0.016, respectively), CRP decreased first from day 3 (P < 0.001) (Fig. 6). There was a positive correlation between IL-6 and CRP from day 2, but not initially in the ED (Supplementary Fig. 4, https://links.lww.com/SHK/B502).
In this study, we hypothesized that relevant inflammatory biomarkers might improve the clinical assessment of infection in patients with possible sepsis in the ED. We investigated the diagnostic performance of biomarkers to identify infections. We also evaluated the precision of the ED clinician to diagnose infections, defined by the prescription of antibiotics (EDC) before biomarkers were analyzed, and finally the add-on effect of the biomarkers to the EDC. The likelihood of infection was assessed by a detailed post hoc evaluation of all patient data. C-reactive protein and IL-6 were the overall best-performing biomarkers in the ED for identifying patients with infection. Whereas PCT, IL-6, and CRP all provided high PPV, IL-6 and CRP also reached high NPVs, able to virtually rule out infection, especially when combined. Longitudinal samples showed that IL-6 and calprotectin peaked at admission to the ED, whereas PCT and CRP peaked the following days. Notably, IL-6 and CRP did not correlate at admission among patients with infection but correlated strongly during subsequent days. The different dynamics of biomarkers in the early phases of infection implies that the use of biomarker combinations may be more appropriate compared with a single biomarker approach, possibly because of symptom course and duration disparities.
According to recent Surviving Sepsis Campaign (SSC) guidelines, patients with septic shock (18) or a high likelihood of sepsis should be treated with antibiotics immediately and at the latest within 1 h of recognition. Thus, for patients in this sepsis category, EDCs should normally not wait until results of biomarkers (2). However, in patients with possible sepsis without shock, the SSC guidelines recommend a time-limited investigation and administration of antibiotics within 3 h if infection is still suspected. Consequently, for patients in this sepsis category, biomarkers may support the ED Clinician's decision-making. Although the SSC recommends against the use of qSOFA as a single screening tool for sepsis (2), it is a well-established predictor of severity and mortality (27). Appreciating that these results must be validated in independent cohorts, our data could nevertheless support a treatment algorithm as illustrated in Figure 7. For patients with qSOFA of 2 or greater, only a very low probability of undertreatment can be accepted, and a diagnostic model would have to perform extremely well to outperform a strategy of treating all patients. Decision curve analysis revealed that treating all patients gave similar net benefit as EDC alone and with biomarkers. However, if both CRP and IL-6 were less than the optimal cutoff value in our cohort, an NPV of 98% might therefore safely argue for discontinuation of antibiotics. Conversely, for patients with qSOFA of less than 2, a higher risk of undertreatment may be acceptable. Using a diagnostic model with CRP and IL-6 together with clinical judgment could be useful both to secure treatment in patients with infections and prevent overtreatment in patients without.
The optimal cutoff value and AUC, sensitivity, and specificity of calprotectin were similar as in previous reports (11,12,28). However, in contrast to these studies, the performance of calprotectin was inferior to the other biomarkers in our data (11,12,28). Calprotectin is known to peak before CRP and PCT but after IL-6 (9,28,29), and thus, a preadmission peak in calprotectin seems unlikely. Nevertheless, it may be used as a biomarker to identify gram-positive infections, possibly guiding antibiotic therapy. We also confirm prior findings that PCT predicts the presence of infections but performs poorly in excluding infections. Our optimal cutoff value is in accordance with the established cutoff value of 0.25 μg/L (30). In most settings, PCT is considered a better diagnostic biomarker for infection than CRP (31,32). However, in this ED cohort, CRP performed better than PCT. This was mostly due to superior sensitivity, as previously shown by de Kruif et al. (14). Sampling in most of the studies included in meta-analyses has been done in the ICU (32,33), clearly at a later stage of disease than in our ED study. Notably, the few patients with confirmed viral infections were not excluded because bacterial coinfection could not be excluded. This could have contributed to CRP's better performance relative to PCT, as PCT was lower in viral compared with bacterial infections in our cohort, and CRP was not.
This study has some limitations. First, the criterion standard of infections is difficult to define. We have based this study on a post hoc clinical evaluation. An alternative approach could be to use a strict definition of microbiological confirmed infection. However, we find this insufficient, as blood cultures have been shown to have a sensitivity as low as 32% in sepsis (34). We therefore argue that our definition of infection is more clinically relevant. Second, we did not decipher the single components, such as clinical examinations and radiology of the clinician's assessment, which also includes clinical gestalt (35). Third, our results have not been verified in a validation cohort. Moreover, only 58% of the patients with infection “not likely” had blood culture drawn in the ED, probably because other diagnosis was evident, but this can also be a source of bias. Lastly, the choice to treat severely ill patients with antibiotics might reflect caution rather than conviction of infection as underlying cause, as discussed by Prescott and Iwashyna (36).
We have shown that the combination of CRP and IL-6 obtained at admission to the ED can aid the clinical decision when assessing infection in patients with possible sepsis. IL-6 and CRP were superior as biomarkers for infection regardless of qSOFA score (optimal cutoff values ≥31 mg/L and ≥52 pg/mL, respectively). These results need to be verified in an independent cohort.
The authors thank all collaborators in the study, especially the staff in the ED, ICU, and department of biochemistry at Oslo University Hospital, Ullevål, for study attentiveness and help. They also thank Per Trygge Flemmen for his guidance in R. Christine Brunborg (Department of Biostatistics, University of Oslo) was consulted for statistical analyses; any errors are the responsibility of the authors.
1. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, Lemeshow S, Osborn T, Terry KM, Levy MM: Time to treatment and mortality during mandated emergency care for sepsis
. N Engl J Med
2. Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, Machado FR, McIntyre L, Ostermann M, Prescott HC, et al.: Surviving Sepsis
Campaign: international guidelines for management of sepsis
and septic shock 2021. Crit Care Med
3. Pierrakos C, Velissaris D, Bisdorff M, Marshall JC, Vincent JL: Biomarkers of sepsis
: time for a reappraisal. Crit Care
4. Tan M, Lu Y, Jiang H, Zhang L: The diagnostic accuracy of procalcitonin and C-reactive protein for sepsis
: a systematic review and meta-analysis. J Cell Biochem
5. Molano Franco D, Arevalo-Rodriguez I, Roqué IFM, Montero Oleas NG, Nuvials X, Zamora J: Plasma interleukin-6 concentration for the diagnosis of sepsis
in critically ill adults. Cochrane Database Syst Rev
6. Ma L, Zhang H, Yin YL, Guo WZ, Ma YQ, Wang YB, Shu C, Dong LQ: Role of interleukin-6 to differentiate sepsis
from non-infectious systemic inflammatory response syndrome. Cytokine
7. Dale I, Fagerhol MK, Naesgaard I: Purification and partial characterization of a highly immunogenic human leukocyte protein, the L1 antigen. Eur J Biochem
8. Foell D, Frosch M, Sorg C, Roth J: Phagocyte-specific calcium-binding S100 proteins as clinical laboratory markers of inflammation. Clin Chim Acta
9. Lipcsey M, Hanslin K, Stålberg J, Smekal D, Larsson A: The time course of calprotectin liberation from human neutrophil granulocytes after Escherichia coli
and endotoxin challenge. Innate Immun
10. van Zoelen MA, Vogl T, Foell D, Van Veen SQ, van Till JW, Florquin S, Tanck MW, Wittebole X, Laterre PF, Boermeester MA, et al.: Expression and role of myeloid-related protein-14 in clinical and experimental sepsis
. Am J Respir Crit Care Med
11. Havelka A, Sejersen K, Venge P, Pauksens K, Larsson A: Calprotectin, a new biomarker for diagnosis of acute respiratory infections. Sci Rep
12. Larsson A, Tydén J, Johansson J, Lipcsey M, Bergquist M, Kultima K, Mandic-Havelka A: Calprotectin is superior to procalcitonin as a sepsis
marker and predictor of 30-day mortality in intensive care patients. Scand J Clin Lab Invest
13. Wollmer M, Wändell P, Rosenqvist M, Larsson A, Melander O, Wessman T, Ärnlöv J, Ruge T: Plasma calprotectin in the emergency department: a potential clinical biomarker for patients with infectious diseases. Scand J Clin Lab Invest
14. De Kruif MD, Limper M, Gerritsen H, Spek CA, Brandjes DP, ten Cate H, Bossuyt PM, Reitsma PH, van Gorp EC: Additional value of procalcitonin for diagnosis of infection in patients with fever at the emergency department. Crit Care Med
15. Henning DJ, Hall MK, Watsjold BK, Bhatraju PK, Kosamo S, Shapiro NI, Liles WC, Wurfel MM: Interleukin-6 improves infection identification when added to physician judgment during evaluation of potentially septic patients. Am J Emerg Med
16. Leticia Fernandez-Carballo B, Escadafal C, MacLean E, Kapasi AJ, Dittrich S: Distinguishing bacterial versus non-bacterial causes of febrile illness—a systematic review of host biomarkers. J Infect
17. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, Irwig L, Levine D, Reitsma JB, de Vet HC, et al.: STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open
18. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, et al.: The third international consensus definitions for sepsis
and septic shock (Sepsis
19. Charlson ME, Pompei P, Ales KL, MacKenzie CR: A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis
20. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, Kumar A, Sevransky JE, Sprung CL, Nunnally ME, et al.: Surviving Sepsis
Campaign: international guidelines for management of sepsis
and septic shock: 2016. Crit Care Med
21. Amundsen EK, Binde C, Christensen EE, Klingenberg O, Kvale D, Holten AR: Prognostic value of nucleated RBCs for patients with suspected sepsis
in the emergency department: a single-center prospective cohort study. Crit Care Explor
22. Calandra T, Cohen J: The international sepsis
forum consensus conference on definitions of infection in the intensive care unit. Crit Care Med
23. Klein Klouwenberg PM, Cremer OL, van Vught LA, Ong DS, Frencken JF, Schultz MJ, Bonten MJ, van der Poll T: Likelihood of infection in patients with presumed sepsis
at the time of intensive care unit admission: a cohort study. Crit Care
24. Memorial Sloan Kettering Cancer Center. Decision Curve Analysis. Available at: https://www.mskcc.org/departments/epidemiology-biostatistics/biostatistics/decision-curve-analysis
. Accessed October 13, 2021.
25. Vickers AJ, van Calster B, Steyerberg EW: A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res
26. Hajian-Tilaki K: Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform
27. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, Rubenfeld G, Kahn JM, Shankar-Hari M, Singer M, et al.: Assessment of clinical criteria for sepsis
: for the third international consensus definitions for sepsis
and septic shock (Sepsis
28. Jonsson N, Nilsen T, Gille-Johnson P, Bell M, Martling CR, Larsson A, Mårtensson J: Calprotectin as an early biomarker of bacterial infections in critically ill patients: an exploratory cohort assessment. Crit Care Resusc
29. Fullerton JHA, Segre E, de Maeyer RP, Maini AA, Gilroy DW: Kinetics of calprotectin, procalcitonin and C-reactive protein in healthy volunteers administered intravenous endotoxin. Presented at the 40th International Symposium on Intensive Care & Emergency Medicine; Brussels, Belgium. Critical Care 2022:474
30. Schuetz P, Beishuizen A, Broyles M, Ferrer R, Gavazzi G, Gluck EH, González Del Castillo J, Jensen JU, Kanizsai PL, Kwa ALH, et al.: Procalcitonin (PCT)–guided antibiotic stewardship: an international experts consensus on optimized clinical use. Clin Chem Lab Med
31. Hausfater P: Biomarkers and infection in the emergency unit. Med Mal Infect
32. Wacker C, Prkno A, Brunkhorst FM, Schlattmann P: Procalcitonin as a diagnostic marker for sepsis
: a systematic review and meta-analysis. Lancet Infect Dis
33. Liu Y, Hou JH, Li Q, Chen KJ, Wang SN, Wang JM: Biomarkers for diagnosis of sepsis
in patients with systemic inflammatory response syndrome: a systematic review and meta-analysis. Springerplus
34. Bernard GR, Ely EW, Wright TJ, Fraiz J, Stasek JE Jr., Russell JA, Mayers I, Rosenfeld BA, Morris PE, Yan SB, et al.: Safety and dose relationship of recombinant human activated protein C for coagulopathy in severe sepsis
. Crit Care Med
35. Dale AP, Marchello C, Ebell MH: Clinical gestalt to diagnose pneumonia, sinusitis, and pharyngitis: a meta-analysis. Br J Gen Pract
36. Prescott HC, Iwashyna TJ: Improving sepsis
treatment by embracing diagnostic uncertainty. Ann Am Thorac Soc