Predicting 30-day mortality in intensive care unit patients with ischaemic stroke or intracerebral haemorrhage

BACKGROUND Stroke patients admitted to an intensive care unit (ICU) follow a particular survival pattern with a high short-term mortality, but if they survive the first 30 days, a relatively favourable subsequent survival is observed. OBJECTIVES The development and validation of two prognostic models predicting 30-day mortality for ICU patients with ischaemic stroke and for ICU patients with intracerebral haemorrhage (ICH), analysed separately, based on parameters readily available within 24 h after ICU admission, and with comparison with the existing Acute Physiology and Chronic Health Evaluation IV (APACHE-IV) model. DESIGN Observational cohort study. SETTING All 85 ICUs participating in the Dutch National Intensive Care Evaluation database. PATIENTS All adult patients with ischaemic stroke or ICH admitted to these ICUs between 2010 and 2019. MAIN OUTCOME MEASURES Models were developed using logistic regressions and compared with the existing APACHE-IV model. Predictive performance was assessed using ROC curves, calibration plots and Brier scores. RESULTS We enrolled 14 303 patients with stroke admitted to ICU: 8422 with ischaemic stroke and 5881 with ICH. Thirty-day mortality was 27% in patients with ischaemic stroke and 41% in patients with ICH. Important factors predicting 30-day mortality in both ischaemic stroke and ICH were age, lowest Glasgow Coma Scale (GCS) score in the first 24 h, acute physiological disturbance (measured using the Acute Physiology Score) and the application of mechanical ventilation. Both prognostic models showed high discrimination with an AUC 0.85 [95% confidence interval (CI), 0.84 to 0.87] for patients with ischaemic stroke and 0.85 (0.83 to 0.86) in ICH. Calibration plots and Brier scores indicated an overall good fit and good predictive performance. The APACHE-IV model predicting 30-day mortality showed similar performance with an AUC of 0.86 (95% CI, 0.85 to 0.87) in ischaemic stroke and 0.87 (0.86 to 0.89) in ICH. CONCLUSION We developed and validated two prognostic models for patients with ischaemic stroke and ICH separately with a high discrimination and good calibration to predict 30-day mortality within 24 h after ICU admission. TRIAL REGISTRATION Trial registration: Dutch Trial Registry (https://www.trialregister.nl/); identifier: NTR7438.


Introduction
The incidence of stroke and the subsequent mortality have declined over the past decades, yet stroke is still among the top five leading causes of death in many countries.[3] Cerebral injury associated with stroke can threaten the function of other vital organ systems, leading to respiratory or circulatory complications. 4,5As a result, patients may need treatment in the intensive care unit (ICU).This has occurred more often during the last decade 6,7 with impaired consciousness and subsequent intubation as the main reason for ICU admission. 8Stroke encompasses several subtypes of which intracerebral haemorrhage (ICH) and ischaemic stroke and are the most frequent. 9tients admitted to ICU with a stroke follow a particular survival pattern with a high short-term mortality, but if they survive the first 30 days, a relatively favourable subsequent survival is observed compared with several other diseases necessitating ICU admission. 103][14][15] Therefore, it is highly relevant to develop a disease-specific model for ICU-admitted stroke patients to be able to predict which patients are most likely to survive the first critical 30 days.Preferably, clinical parameters readily available upon ICU admission or within the first 24 h of ICU admission should be used as possible predictors.
The generic Acute Physiology and Chronic Health Evaluation (APACHE) IV model 16 is an established and widely used prediction model for in-hospital mortality in ICU-admitted patients using parameters readily available within 24 h after ICU admission.Although it can be seen as the benchmark prediction model in ICU medicine, it is unknown if this generic model is suitable in disease-specific situations with a markedly specific mortality pattern such as stroke.In addition, the end point of the APACHE-IV model (i.e.hospital mortality) does not match the apparent transition from high to low mortality risk around 30 days, as has been shown in the subset of ICU-admitted stroke patients. 10 aimed to develop and validate two prognostic models to predict 30-day mortality for ICU patients with either ischaemic stroke or ICH.The two models were developed based on parameters readily available within 24 h after ICU admission, were internally validated and were compared with the existing APACHE-IV model.

Study design
Ethical approval for this study (Ethical Committee N8 W 18_049#18.067) was provided by the Institutional Review Board (IRB) of the Amsterdam University Medical Center (Chairperson Professor J. Swinkels) on 20 February 2018.According to the IRB, the study was not subject to the Medical Research Involving Human Subjects Act and therefore the necessity to obtain informed consent was waived.This cohort study was performed in all 85 Dutch ICUs participating in the National Intensive Care Evaluation (NICE) database between 2010 and 2019, comprising over 90% of all Dutch ICUs. 17Inclusion criteria were ICU admission due to ischaemic stroke or ICH according to the APACHE-IV classification system 16 and fulfilling the APACHE-IV inclusion criteria (therefore, excluding ICU readmissions and ICU admission <4 h).Patient survival data were obtained by linking the NICE database to the national medical insurance claim database (Vektis, Zeist, The Netherlands) using a deterministic linkage algorithm. 18The only exclusion criterion was a linkage discrepancy between the NICE and Vektis database.
The NICE database is registered according to the Dutch Data Protection Act, and all data collection was performed in accordance with General Data Protection Regulations.The study is registered in the Dutch Trial Register as NTR7438.

Data collection
Potential determinants recorded within 24 h of ICU admission and available in the NICE database were selected based on literature, clinical experience, and empirical knowledge.All potential determinants are shown in Supplemental Digital Content 1, http://links.lww.com/EJA/A876, which demonstrates definitions and data measurement scales of all potential determinants, and whether determinants are incorporated in the APACHE-IV model.

SOPHISTIC-2 study 137
Eur J Anaesthesiol 2024; 41:136-145 Age, sex, comorbidities (some new and some incorporated in the APACHE-IV model), and acute physiological disturbance quantified by the APACHE-III Acute Physiology Score (APS) 19 were included.The APACHE-III APS was used without points assigned for Glasgow Coma Scale (GCS) score.In this manner, the influence of the acute physiologic derangement could be assessed separately from the impaired level of consciousness in this neurologically compromised population.The APS without points for GCS, therefore ranges from 0 to 204 points (the full range including points for GCS is 0 to 252).Supplemental Digital Content 2, http://links.lww.com/EJA/A877 elaborates on all elements of the APACHE-III APS and demonstrates its calculation.
The GCS score was incorporated by using the lowest GCS score recorded during the first 24 h of ICU admission (see complementary explanation in Supplemental Digital Content 1, http://links.lww.com/EJA/A876, which demonstrates all potential determinants).The GCS was a priori categorised into three groups [GCS score 3 to 8 (low); 9 to 12 (moderate) and 13 to 15 (high) as adapted from traumatic brain injury literature]. 20,21Additionally, whether admission took place in a specialised neurosurgical centre was taken into account.Furthermore, the application of mechanical ventilation, vasoactive medication and the occurrence of acute renal failure or confirmed infection within 24 h of ICU admission were incorporated as potential predictors.

Outcome
The primary outcome was 30-day mortality after ICU admission.Logistic regression models were used to analyse the association between potential determinants and 30-day mortality.

Models' development
All models were built separately for both types of stroke.The dataset was evenly split at random to create a training and validation dataset.Model development, including a preliminary descriptive step, was performed in the training data.
First, a descriptive step was taken by dividing all potential determinants into five categories to assess the contribution of different steps in the collection of clinically relevant information.These categories were based on the chronological order in which information is assessed in common clinical practice: age and sex, chronic health condition, level of consciousness using GCS score as specified above, acute physiological disturbance using the APACHE-III APS (without GCS), and treatment and complications (as specified above and in Supplemental Digital Content 1, http://links.lww.com/EJA/A876, which demonstrates all potential determinants).All models also included calendar year of ICU admission, time of admission, and type of hospital.
We assessed the results of logistic regression models for each type of stroke (ischaemic stroke and ICH) separately, with increasing numbers of potential determinants, irrespective of statistical significance or other measure of variable selection in a five-step fashion with category 1, 2, 3, 4, and lastly 5.In each model, hereafter called the descriptive models, the new category was included together with all potential determinants of the previous categories.Hence, the fifth model incorporated all above mentioned potential determinants, and is hereafter called the full model.
Second, the full models for ischaemic stroke and ICH were simplified with backward selection based on the Akaike Information Criterion (AIC), 22 selecting the bestfit model to predict 30-day mortality with the greatest amount of variation explained while using the fewest possible number of determinants, hereafter called the simplified model.
Third, the prediction of the primary outcome was assessed for both types of stroke using the existing APACHE-IV model 16 (specification is shown in Supplemental Digital Content 1, http://links.lww.com/EJA/A876), which includes age, some comorbidities, APACHE-III APS and admission diagnosis.

Models' validation and comparison
The predictive performance of the newly derived simplified models was assessed in the validation dataset.Consequently, both simplified models were compared with the APACHE-IV model for each type of stroke separately.Discrimination was measured by Receiver Operating Characteristic (ROC) analyses with the Area Under the Curve (AUC) as performance measure.
Calibration curves were plotted for both simplified models and the APACHE-IV model to assess accuracy between the estimated and observed number of events.Furthermore, the Brier score 23 (range 0 to 1) was used to assess the overall performance index reflecting both discrimination and calibration of both simplified models, with lower scores indicating better performance.Differences in predictive performance between the simplified and APACHE-IV models were assessed based on differences between corresponding AUCs with 95% confidence interval (95% CI).

Sensitivity analyses
Our primary outcome could have been influenced by withdrawal of life sustaining therapy (WLST) decisions, which were not registered in this study.In the literature, the use of mortality within 72 h after ICU discharge as a proxy for WLST decisions has been described. 24,25herefore, sensitivity analyses were performed in the subset of patients who survived more than 72 h after ICU discharge in order to assess the potential impact of WLST All statistical analyses were performed using statistical environment R, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria).The performance and reporting of this study were in accordance with the TRIPOD statement and checklist 26 (see Supplemental Digital Content 3, http://links.lww.com/EJA/A878, which demonstrates the completed TRIPOD checklist).

Results
During the study period (2010 to 2019), 650 815 patients were admitted to the participating ICUs who fulfilled the APACHE-IV criteria and for whom valid record linkage with the Vektis database could be established.The 30day mortality for all admitted ICU patients after ICU admission was 13%.Of the total ICU population, 14 303 patients (2.2%) were admitted for stroke, 8422 with ischaemic stroke and 5881 with ICH (Fig. 1).Thirtyday mortality after ICU admission was 27% in patients with ischaemic stroke and 41% in patients with ICH.Descriptive characteristics of both patient groups are shown in Table 1.

Descriptive models
The descriptive models showed increasing AUCs for every consecutive model (Table 2).The largest increase in discriminative performance occurred with the addition of GCS score in each type of stroke: from AUC 0.64 (95% CI, 0.62 to 0.66) to AUC 0.83 (0.82 to 0.85) in ischaemic stroke, and from AUC 0.62 (0.60 to 0.64) to AUC 0.82 (0.80 to 0.83) in ICH.In the following steps, discriminative performance increased only minimally.

Development of simplified models for ischaemic stroke and intracerebral haemorrhage
Table 3 shows the simplified models predicting 30-day mortality in patients admitted to ICU with ischaemic stroke and ICH, respectively.Only eight predictors remained in each model.Important contributors in both models were age, GCS score, acute physiological disturbance (as defined using APACHE-III APS; without GCS), the application of mechanical ventilation, and the occurrence of acute renal failure.Every increasing point in the APACHE-III APS score increased the odds of mortality 1.03-fold (95% CI, 1.03 to 1.04) in patients with both ischaemic stroke and ICH.A GCS score in the lowest category (GCS 3 to 8) increased 30-day mortality SOPHISTIC-2 study 139       observed and expected mortality corresponded very well with the black dots lying closely to the 458 line representing perfect match.Hence, there was an overall good calibration for both simplified models and APACHE-IV model in both types of stroke.Brier scores were 0.13 for the simplified model for ischaemic stroke and 0.16 for ICH, indicating an overall good predictive performance in both simplified models (Table 3).

Sensitivity analyses
Sensitivity analyses excluding patients with assumed WLST, occurring in 418 (5.0%) patients with ischaemic stroke and 285 (4.8%) in patients with ICH, demonstrated wider CIs due to smaller groups, but similar effect estimates and model performance in mortality prediction (Supplemental Digital Content 4, http://links.lww.com/EJA/A879, which demonstrates the results of the sensitivity analyses).
142 van Valburg et al.In this nationwide study, including over 14 000 stroke patients admitted to the ICU, we developed and validated two prognostic models for patients with ischaemic stroke and intracerebral haemorrhage, analysed separately, to predict 30-day mortality after ICU admission with high discrimination and good calibration.Main predictors of 30-day mortality were age, GCS score (lowest during the first 24 h), acute physiological disturbance as described by the APACHE-III APS score, the application of mechanical ventilation and the occurrence of acute renal failure.The newly derived models did not differ in performance from the APACHE-IV model in predicting 30-day mortality.Major new contributors compared with the APACHE-IV model were the use of vasoactive medication, and the occurrence of acute renal failure.
][14][15] Because of the size of our study, we were able to build models for ischaemic stroke and ICH separately.The two models demonstrated overlapping predictors although differences were found, mainly in effects of comorbidities.][15]27 The strengths of this study are the large number of patients in a multicentre study design and the comparison to the existing APACHE-IV model, which also has been validated in the Dutch ICU population. 28Furthermore, we limited the candidate predictors to determinants present within 24 h of ICU admission and we chose a literaturebased endpoint of 30-day mortality taking the specific mortality pattern of ICU-admitted stroke patients into consideration.Additionally, in accordance with the Joint Statement regarding Prognostication in Neurocritical Care, not only did we include admission data, but also data from the subsequent 24 h, and we could incorporate many comorbidities and complications, which is often lacking in prognostic tools for mainly patients with ICH. 29 Additionally, by design, we incorporated acute physiological derangement by using the well known APS, but eliminated the points assigned for GCS score to separate acute physiological derangement from impaired level of consciousness.Using this approach, we could assess impaired level of consciousness as a separate step in the preliminary descriptive models.Moreover, by taking GCS score into account using the lowest value in the first 24 h of ICU admission, not only did we incorporate a single moment (such as on admission), but created a more robust view of the patients' level of consciousness during the first 24 h of ICU admission.
Some limitations need to be addressed.Firstly, our results pertain to the subset of acute stroke patients admitted to the ICU.No extrapolation is possible to patients with ischaemic stroke or ICH admitted from the emergency department to the ward or stroke unit.1][32][33][34][35][36] This can lead to increased numbers of ICU-admitted stroke patients in the future 6,7 but would not affect the validity of our study results.
Secondly, the administrative NICE database creates the possibility of analysing prognostic factors for 30-day mortality in a large and extensive cohort of ICU-admitted stroke patients, but unfortunately does not contain data concerning functional outcome.Therefore, this study pertains only to mortality risk of ICU-admitted stroke patients and not to their functional independence or quality of life.This certainly warrants more research with longer follow-up using the above mentioned outcome measurements.It also restrains our ability to implement this new prognostic model in clinical decision-making, especially with regards to counselling families on continued ICU care versus WLST.Unfortunately, we could not collect data concerning National Institutes of Health Stroke Scale (NIHSS) scores 37,38 or clinical neurological examination, ICH score, 39 information on affected vascular territories, subtype of stroke, or performed neurosurgical interventions as possible predictors via the NICE database.The latter would have been particularly interesting as we found that having been admitted to a specialised neurosurgical centre was protective, but we did not have specific information on type of neurosurgery performed nor on the decision-making process pertaining to a neurosurgical intervention.Preferably, this should be subject of future studies.We expect that with this crucial stroke-specific information, the likelihood will increase that our model might outperform the already decent performing APACHE-IV model.
Third, all-cause mortality was used as outcome measure, and, therefore, not taking reason of death into account including WLST decisions.Unfortunately, due to legal data protection regulations, we were not able to assess the exact proportion of WLST decisions by reviewing each individual patient report.Insight in this proportion would have improved international generalisability.An increasing shift in European ICUs towards WLST practices has been shown. 40Recent literature stated that prediction models in acute brain injury in general are not suited for WLST decision-making 29,41 as WLST decisions may influence the results in prognostication for severely ill patients such as ICU-admitted stroke patients. 24,25,42,43aking all this into account, we attempted to incorporate Lastly, with the development of the simplified prognostic models, we also aimed to explore whether using fewer parameters would enable an improvement in clinical bedside use compared with, for example, the APACHE-IV model.Only eight predictors were incorporated in each simplified model.Unfortunately, the incorporation of the APACHE-III APS as a determinant, hampers practical bedside use, as the complexity of the APS still requires computational capabilities.
However, the descriptive models in this study demonstrated that the inclusion of age, sex, six comorbidities and the lowest GCS score during the first 24 h of ICU admission as the only predictors returns an AUC of over 0.8 in both types of stroke, implying good discriminatory ability with only these few and easy assessable variables.This information may assist healthcare providers in daily clinical practice in decision-making for stroke patients admitted to ICU.Ongoing exploration of practical prognostication in this population may, therefore, focus on predictors concerning impaired level of consciousness or more applicable implementations of acute physiologic disturbance measures.

Conclusion
Prognostication in ICU-admitted stroke patients is commonly performed in clinical practice using different approaches such as clinical impressions based on knowledge and experience, as well as empirical prediction tools.We developed and validated prognostic models with a high discrimination and good calibration to predict 30-day mortality within 24 h for patients who were admitted to the ICU because of ischaemic stroke or ICH, analysed separately.Major predictors were age, GCS score, acute physiologic disturbance, the application of mechanical ventilation and the occurrence of acute renal failure.The two models demonstrated overlapping predictors, although differences were found: presence of cardiovascular comorbidities as NYHA class IV or diabetes were predictive in patients with ischaemic stroke, while ICU admission to a specialised neurosurgical centre was protective in patients with ICH.
The APACHE-IV model predicting 30-day mortality showed similar performance in both populations but is quite complex and includes more variables.The descriptive models in this study showed that even with a few and easy assessable predictors (age, sex, six comorbidities, and lowest GCS score during the first 24 h), a good model performance can be achieved.This may help clinicians in their daily work in decision-making for ICU-admitted stroke patients.
Further research should focus on practical bedside prediction modalities with, preferably, functional independence and quality of life as outcome measures while keeping the specific mortality pattern of ICU-admitted stroke patients in mind.Such studies should incorporate information on clinical neurological examination including brainstem function, and subtype of stroke and vascular territories within the stroke populations (such as brainstem infarction versus a hemispheric stroke) to assess possible differences in prognosis among the subtypes of ischaemic stroke and ICH.Other factors that might be examined are, the impact of WLST decisions, and organisational aspects (such as whether admission to a specialised neurosurgical centre, which was protective in patients with an ICH in our analyses, might imply a more interventional approach).An ongoing collaboration between neurologists and intensivists may shift the focus of current datasets to clinical severity scores and incorporation of functional status variables to individualise patient care in a multidisciplinary approach.

Fig. 2
Fig. 2 Receiver operating characteristic curves of full, simplified and Acute Physiology and Chronic Health Evaluation-IV models for (a) ischaemic stroke and (b) intracerebral haemorrhage.

Fig. 3
Fig.3Calibration plots of simplified and Acute Physiology and Chronic Health Evaluation-IV models for ischaemic stroke and intracerebral haemorrhage, respectively.

SOPHISTIC- 2
study 143 Eur J Anaesthesiol 2024; 41:136-145 this important subject by performing sensitivity analyses, excluding the subset of patients in which WLST decisions were made based on a literature-based definition.These sensitivity analyses demonstrated similar effect estimates for 30-day mortality prediction.

Table 1
Characteristics of all 14 303 patients admitted to the ICU after ischaemic stroke or intracerebral haemorrhage Data are presented as: n (%) or median[IQR].APACHE, Acute Physiology and Chronic Health Evaluation; APS, acute physiology score; COPD, chronic obstructive pulmonary disease; GCS, Glasgow Coma Scale; h, hour; ICU, intensive care unit; IQR, interquartile range; n, number; NYHA, New York Heart Association.

Table 2
Descriptive models of five categories for 30-day mortality in patients admitted to an intensive care unit with ischaemic stroke or intracerebral haemorrhage APACHE, Acute Physiology and Chronic Health Evaluation; APS, acute physiology score; AUC, area under the curve; CI, confidence interval.a Comorbidities include chronic respiratory insufficiency/COPD, NYHA class IV, diabetes, chronic renal insufficiency or dialysis dependency, immunologic deficiency or compromised status and malignancies.b APACHE to III APS was calculated using all parameters except for GCS (as is also shown in Supplemental Digital Content 2, http://links.lww.com/EJA/A877), as this was accounted for in category 3 in order to separate acute physiological derangement and impaired level of consciousness.Further explanation is found in the Methods section.c Complications or ICU treatments include the use of mechanical ventilation or vasoactive medication and occurrence of acute renal failure or confirmed infection.Calendar year of ICU admission (2010 to 2019), time and type of admission and type of hospital are taken into account in all descriptive models.Eur J Anaesthesiol 2024; 41:136-145

Table 3
Simplified models for predicting 30-day mortality in ICU-admitted patients with ischaemic stroke or intracerebral haemorrhage 16C, Akaike Information Criterion; APACHE, Acute Physiology and Chronic Health Evaluation; APS, acute physiology score; AUC, area under the curve; CI, confidence interval; GCS, Glasgow Coma Scale; ICU, intensive care unit; NA, not admittable, this determinant is listed as it is predictive for the other type of stroke; NYHA, New York Heart Association; OR, odds ratio.aPredictor is present in the APACHE to IV model.See Supplemental Digital Content 1, http://links.lww.com/EJA/A876 for an overview of all parameters incorporated in the APACHE-IV model.16bReference ¼ no.