Respiratory ECMO Survival Prediction (RESP) Score for COVID-19 Patients Treated with ECMO

Abstract

Veno-venous extracorporeal membrane oxygenation (VV ECMO) has been used as a life-supporting modality for patients with severe respiratory failure because of coronavirus disease 2019 (COVID-19). We aim to evaluate the performance of the RESP score in predicting the hospital survival of COVID-19 patients undergoing VV ECMO. We performed retrospective analysis of the extracorporeal life support organization (ELSO) dataset for COVID-19 patients requiring ECMO support to evaluate the performance of RESP score in predicting in hospital survival.

All adult (age ≥18) COVID-19 patients receiving VV ECMO for acute respiratory failure enrolled in the ELSO database from March to August 2020 were included in the analysis.

A total of 1985 patients from the ELSO registry were identified and analyzed based on pre-ECMO variables. Median RESP score of survivors was 3 (IQR 1–5) compared to 2 (IQR 0–4) in deceased. A logistic model including RESP score variables poorly discriminated survival and death with AUC (area under curve) 0.61 (95% confidence interval: 0.59–0.64). In-hospital survival for COVID-19 patients based on RESP score class from I to V was 69.7%, 59.3%, 45.7%, 42.5%, and 32.3%, respectively. Patients with immunosuppression (relative risk = 0.43) and pre-ECMO cardiac arrest (relative risk = 0.48) had lower survival.

RESP score is a poor predictor of survival in COVID-19 patients undergoing ECMO. Compared to the original cohort used for RESP score creation, COVID-19 patients in RESP class I–III had worse survival whereas the patients in RESP class IV–V had better survival.

In 2019, severe acute respiratory syndrome coronavirus 2 (SARS-CoV 2) was identified as a new infectious agent in China leading to the coronavirus disease-19 (COVID-19) pandemic, which resulted in significant stress on health care systems worldwide. As of the time of writing, there have been 103 million COVID-19 cases worldwide and a death count of 2.2 million.¹ Extracorporeal membrane oxygenation (ECMO) is a possible therapeutic option for patients with refractory severe acute respiratory failure.^2,3 During the past decade, after the 2009 influenza outbreak, ECMO has been established as a rescue modality for severe refractory acute respiratory failure.⁴ ECMO is a resource-intensive treatment which requires careful consideration before implementation, particularly in a pandemic situation.

There are a few scores which assist with predicting mortality on veno-venous ECMO (VV ECMO) patients based on pre-ECMO clinical criteria, such as Respiratory ECMO Survival Prediction (RESP) score or PRedicting dEath for SEvere acute respiratory distress syndrome (ARDS) on VV-ECMO (PRESERVE) score.^5,6 Clinical judgment along with these scores assists with evaluating candidacy for ECMO in refractory acute severe respiratory failure.

RESP score is one of the scores developed from data from the extracorporeal life support organization (ELSO) registry in 2014 for predicting hospital survival.⁵ This score uses 12 pre-ECMO clinical variables and based on the total score, patients are divided into five risk classes (I–V) (Table 1). As risk class increases, hospital survival decreases (92%, 76%, 57%, 33%, and 18%, respectively, for risk class I–V). In the original article that examined RESP score and survival, researchers showed that RESP score might be more meaningful than routine ICU severity score in decision making for patients with severe acute respiratory failure requiring ECMO and estimating their prognosis.⁵

Table 1. - Respiratory extracorporeal membrane oxygenation survival prediction Score Values for Calculation

Characteristic	Score
Age (years)
18–49	0
50–59	−2
≥60	−3
Mechanical ventilation (hours)
0–47	3
48–167	1
≥168	0
Cardiac arrest before ECMO
PaCo2, mm Hg	−2
<75	0
≥75	−1
Peak inspiratory pressure, cm H2O
<42	0
≥42	−1
Immunocompromised	−2
CNS dysfunction	−7
Nonrespiratory infection	−3
Neuromuscular blocker before ECMO	1
Nitric oxide before ECMO	−1
Bicarbonate before ECMO	−2
Viral pneumonia (COVID-19 positive)	3
Total RESP score	Risk Class
≥6	I
3–5	II
−1 to 2	III
−5 to −2	IV
≤−6	V

The utility of the RESP score in predicting in-hospital survival of COVID-19 patients requiring VV ECMO is yet unknown. The goal of this study is to evaluate the RESP score’s ability to predict hospital survival in COVID-19 patients undergoing VV ECMO.

Methods

We performed a retrospective analysis of COVID-19 patients in the ELSO registry who were enrolled between March 1, 2020 and August 30, 2020. ELSO maintains the registry of patients undergoing ECMO support from many centers around the world. ELSO is a global organization, which maintains a comprehensive registry of patients undergoing extracorporeal support from ECMO centers around the world. Data including demographics, pre-ECMO clinical variables (including respiratory support before mechanical ventilation and ECMO), International Classification of Diseases-10 (ICD 10) codes, procedure and complication codes, hospital outcome, use of rescue/adjuvant therapies (such as prone position, neuromuscular blocker, and inhaled pulmonary vasodilators) were collected.

We defined pre-ECMO variables as follows and refer to supplemental material, Supplemental Digital Content 1, https://links.lww.com/ASAIO/A773 for ICD 10 codes.:

• 1. Acute associated nonpulmonary infection – Acute extrapulmonary infection caused by bacteria, fungus, or virus.
• 2. Cardiac dysfunction – Acute and chronic heart failure.
• 3. Chronic respiratory dysfunction – Chronic obstructive pulmonary disease, asthma, and other chronic respiratory conditions.
• 4. Central nervous system (CNS) dysfunction – Stroke, neuro-trauma, encephalopathy, cerebral embolism, and seizures.
• 5. Immunocompromised state – Solid organ transplant, HIV, hematological malignancy, solid tumor, and cirrhosis.
• 6. Obesity – Body mass index >30 kg/m².
• 7. Renal dysfunction – we further delineated this criterion as (a) acute or chronic and (b) with renal replacement or without renal replacement therapy (see attached supplemental material, Supplemental Digital Content 1, https://links.lww.com/ASAIO/A773).
• Because of the nature of the study, Institutional Review Board waived the need for informed consent.

Data Collection

Data were extracted from the ELSO registry for adult patients with COVID-19 undergoing VV ECMO for acute respiratory failure. Data included demographics, pre-ECMO support and procedures, ICD-10 diagnosis codes, culture site, organism, and status at hospital discharge. From the ELSO registry, it is possible to retrieve all the variables needed to calculate RESP score (i.e., age, use of intravenous bicarbonate, neuromuscular blockers, and pulmonary vasodilators, duration of mechanical ventilation before initiation of ECMO, nonpulmonary acute infection before ECMO, partial pressure of carbon dioxide [pCO₂], peak inspiratory pressure [PIP], history of CNS dysfunction, cardiac arrest, and immunocompromised status.). RESP scores were calculated using the weights as recommended in Schmidt et al. (2014)⁵ and categorized into five risk classes (I: ≥6; II: 3–5; III: −1 to 2; IV: −5 to −2; V: ≤−6). All cases in the sample were scored as having viral pneumonia. Though acute kidney injury, cardiac dysfunction, pH, PaO₂/FIO₂ ratio, SaO₂ were not included in RESP score calculation, Schmidt et al. (2014) identified these as putative factors in survival.⁵ Also, prone positioning is known to affect the outcome in Moderate-Severe ARDS whereas steroids and chronic respiratory disease are known to affect the outcome in COVID-19 pneumonia.^7–10 As they were available in the ELSO registry, they were included in our multiple logistic model. All patient and hospital information were de-identified. The study period was from March 2020 to August 2020. The primary outcome was survival at hospital discharge.

Statistical Analysis

Comparisons between patients who survived and died were conducted using independent t-tests for continuous variables and χ² tests for categorical variables. All variables included in the RESP score and identified by Schmidt et al. (2014) were included in analyses. A multiple logistic regression model predicting survival was estimated with 11 variables included in the RESP score and variables related to survival (p < 0.10) in bivariate analyses. The final multiple logistic regression model contained all RESP variables and additional variables that retained significance at p < 0.05. Variables were categorized (per Schmidt et al., 2014) before being entered into the multiple logistic regression model.⁵ Specifically, duration of mechanical ventilation before initiation of ECMO (<48 hours, 48 hours to 7 days, >7 days), PaCO₂ (≥75 mm Hg), and PIP (≥42 cm H₂O) were categorized. FiO₂ (equal to 100, <100), pH (7.35–7.45 or other value), and SaO₂ (≥94, <94) were dichotomized. Age was entered as a continuous variable. The remaining variables were binary (present/absent).

Area under curve (AUC) the receiver operating characteristics (ROC) was estimated as an indicator of how well the model distinguished between patients who survived and died. AUC was estimated for a model containing only RESP score and for the second model containing RESP variables in addition to variables retaining significance (p < 0.05). Observed survival per RESP risk class was calculated for cases without missing values for RESP variables (n = 1206). The predicted probability of survival (and 95% confidence interval [CI]) was estimated and plotted against the RESP score.

Missing data for several measures required consideration. Specifically, duration of mechanical ventilation (10.2%), FiO₂ (12.3%), PIP (29.9%), pH (10.7%), PaCO₂ (11.3%) and SaO₂ (26.8%) had missing data rates above 10%. To reduce bias introduced by listwise deletion, multiple imputations was used for missing data. Thirty sets of imputed data were created to replace missing values (fully conditional specification method).¹¹ Fully conditional specification model is appropriate when imputed variables are discrete. All 11 variables in RESP score, 13 additional putative variables, survival, and hours on ECMO were included in the multiple imputation model. All analyses for this study were generated using SAS 9.4 software. Copyright © 2016 SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC.

Results

The sample included all patients who had data available for pre-extracorporeal life support, COVID-related information, and diagnosis codes (n = 2005). Cases that were discharged as ‘On-ECMO’ to other facilities were excluded from analyses (n = 20). The sample included 1985 patients with COVID-19 who had a mean age of 48.2 years (SD = 11.5, median = 49.2) and were 72.2% (1434/1985) male. The sample comprised 13.0% (257/1985) Asian, 14.1% (280/1985) Black, 28.2% (559/1985) Hispanic, 4.6% (92/1985) Middle Eastern or North African, 4.8% (95/1985) multiple races, 2.5% (49/1985) Native American, 27.5% (546/1985) White, and 5.4% (107/1985) other or unknown race.

Overall survival rate was 52.9% (1051/1985); median RESP score was 3 (range = −12, 7); and median time on ECMO was 354 hours. Before ECMO, 77.1% (1530/1985) of subjects were administered neuromuscular blockers, 35.1% (696/1985) of patients received pulmonary vasodilators, and 8.0% (158/1985) of patients received bicarbonate infusion. As shown in Table 2, several variables included in calculating the RESP score were significantly related to survival in patients with COVID-19 in bivariate analyses. Specifically, age, immunocompromised, time on mechanical ventilation, bicarbonate infusion, history of cardiac arrest before ECMO, PIP, and PaCO₂ were significantly related to survival. In addition to variables included in the RESP score, FIO₂, pH, SaO₂, acute kidney injury, prone positioning, and steroids were significantly related to survival. Survivors had a significantly higher RESP score than nonsurvivors.

Table 2. - Summary of Bivariate Tests Between Survival for Patients with COVID-19 on extracorporeal membrane oxygenation (ECMO) and Patient Characteristics, Pre-ECMO Measures, and ECMO-Related Measures

Variable	All Patients (n = 1985)	Status at Discharge
		Survived (n = 1051)	Died (n = 934)	p value
Age, median (IQ)	49.2 (40.0, 56.6)	46.5 (37.1, 53.7)	51.8 (43.4, 59.1)	<0.001
Immunocompromised, n (%)	79 (4.0)	16 (1.5)	63 (6.8)	<0.001
Hours on mechanical ventilation before ECMO, median (IQ)	89 (32, 149)	83 (30.5, 143)	98 (33, 162)	0.046
CNS dysfunction, n (%)	207 (10.4)	101 (9.6)	106 (11.4)	0.21
Nonpulmonary infection, n (%)	139 (7.0)	71 (6.8)	68 (7.3)	0.65
Neuromuscular blockers, n (%)	1530 (77.1)	813 (77.4)	717 (76.8)	0.76
Pulmonary vasodilator, n (%)	696 (35.1)	368 (35.0)	328 (35.1)	0.96
Bicarbonate (intravenous), n (%)	158 (8.0)	66 (6.3)	92 (9.9)	0.003
Cardiac arrest, n (%)	106 (5.3)	31 (3.0)	75 (8.0)	<0.001
Pre-ECMO ventilator settings, median (IQ)
PaO2/FIO2 ratio	70.0 (58.2, 92.0)	72(60, 95.2)	68.1 (57.0, 88.8)	0.08
FIO2	100 (90,100)	100 (90, 100)	100 (100, 100)	<0.001
PIP, cm H2	34 (30, 38)	33.5 (30, 38)	35 (30, 39)	0.04
MAP, cm H2	22 (19, 26)	22 (19, 26)	22 (19, 26)	0.32
PEEP	14 (12, 16)	14 (12, 16)	14 (11, 16)	0.21
Pre-ECMO blood gas, median (IQ)
pH	7.3 (7.2, 7.4)	7.3 (7.2, 7.4)	7.3 (7.2, 7.4)	<0.001
PaCO2, mm Hg	60 (50, 74)	58 (49, 72)	61.6 (50.3, 75.0)	<0.001
PaO2, mm Hg	67 (57, 81)	67 (57, 83)	66.4 (55.0, 79.0)	0.85
SaO2, %	90 (86, 94)	91 (87, 95)	90 (85, 94)	<0.001
Acute kidney injury, n (%)	434 (21.9)	207 (19.7)	227 (24.3)	0.01
Prone positioning, n (%)	1280 (64.5)	657 (62.5)	623 (66.7)	0.05
Steroids, n (%)	441 (22.2)	191 (18.2)	250 (26.8)	<0.001
Chronic respiratory disease, n (%)	110 (5.5)	59 (5.6)	51 (5.5)	0.88
Heart dysfunction, n (%)	81 (4.1)	38 (3.6)	43 (4.6)	0.27
Obesity, n (%)	1150 (61.8)	620 (62.8)	530 (60.7)	0.35
RESP score, median (IQ)	3 (1, 4)	3 (1, 5)	2 (0, 4)	<0.001
Hours on ECMO, median (IQ)	354 (193, 608)	333 (203, 551)	383 (185, 653)	0.02

CNS, central nervous system; ECMO, extracorporeal membrane oxygenation; FIO₂, fraction of inspired oxygen; IQ, interquartile range; MAP, mean airway pressure;PaO₂, partial pressure of oxygen; PaCO₂, partial pressure of carbon dioxide; PIP, peak inspiratory pressure; PEEP, positive end-expiratory pressure; RESP, respiratory ECMO Survival Prediction; SaO₂, oxygen saturation

As shown in Table 3, survival rates declined across the RESP risk classes, ranging from 68.6% for class I to 35.0% for class V. Figure 1 contains predicted probabilities (and 95% CI) for survival by RESP score.

Table 3. - Survival Rate by RESP Risk Class

RESP Risk Class	Complete Case Survival (n = 1206)	Original Resp score model5
I	68.6% (81/118)	92% (151/164)
II	59.9% (315/526)	76% (428/563)
III	46.7% (184/394)	57% (589/1033)
IV	43.0% (55/128)	33% (148/449)
V	35.0% (14/40)	18% (26/146)

For the Complete Case Sample (n = 1206 sample), all cases had complete data for variables comprising RESP score.

Absolute number in original RESP score model derived from percentage.

RESP, respiratory extracorporeal membrane oxygenation survival prediction.

Figure 1.: Predicted probability of survival by RESP score for COVID-19 patients. Note: Plots for 30 imputations were inspected. A representative plot is shown. For all, the predicted probabilities ranged from just below 0.2 to just above 0.6 and RESP = 0 coincided with about 0.5 probability. RESP, respiratory extracorporeal membrane oxygenation (ECMO) survival prediction.

In multiple logistic regression analysis, the following nine variables retained significance in relation to survival: age, immunocompromised, time on mechanical ventilation, prior cardiac arrest, FiO₂, pH, SaO₂, prone position, and steroids (see Table 4). Acute kidney injury and PaO₂/FIO₂ ratio were included in the multiple logistic regression model but were removed because of nonsignificance. The relative risk of survival for immunocompromised patients and patients with prior cardiac arrest were 0.43 and 0.48, respectively, compared to those without these risk factors.

Table 4. - Summary of Multiple Logistic Analysis Predicting Survival from Patient Characteristics, Pre-Extracorporeal Membrane Oxygenation (ECMO) Measures, and ECMO-Related Measures.

Parameter	Odds Ratio	95% CI		p value
Age	0.96	0.95	0.96	<0.001
Immunocompromised	0.23	0.13	0.41	<0.001
Hours on mechanical ventilation before ECMO
<48 hours	(reference)	(reference)	(reference)	0.19
48 hours to 7 days	0.86	0.68	1.09	0.002
≥7 days	0.62	0.46	0.85
CNS dysfunction	1.00	0.73	1.37	0.98
Nonpulmonary infection	0.95	0.65	1.38	0.79
Neuromuscular blockers	1.00	0.79	1.27	0.98
Pulmonary vasodilator	1.10	0.90	1.35	0.35
Bicarbonate (intravenous)	0.93	0.64	1.34	0.69
Cardiac arrest	0.30	0.19	0.48	<0.001
PaCO2 ≥75 vs PaCO2 <75	0.88	0.69	1.13	0.32
PIP ≥42 vs PIP <42	0.79	0.56	1.13	0.20
FiO2 = 100 vs FiO2 <100	0.77	0.62	0.96	0.02
pH 7.35 – 7.45 vs other	1.50	1.18	1.91	0.001
SaO2 ≥94 vs SaO2 <94	1.29	1.02	1.64	0.04
Prone position	0.78	0.64	0.97	0.02
Steroids	0.60	0.47	0.76	<0.001

CNS, central nervous system; ECMO, extracorporeal membrane oxygenation; FIO₂, fraction of inspired oxygen; PIP, peak inspiratory pressure; PaCO₂, partial pressure of carbon dioxide; RESP, respiratory ECMO Survival Prediction; SaO₂, oxygen saturation.

The multiple logistic regression model fairly discriminated between patients with COVID-19 on ECMO who survived and died, AUC = 0.70 (95% CI: 0.68–0.72, see Figure 2). The logistic model includes only RESP score variables poorly discriminated between patients that survived and died, AUC = 0.61 (95% CI: 0.59–0.64).¹² In the original study by Schmidt et al.⁵ RESP score AUC for viral pneumonia groups was 0.73 (95% CI: 0.65–0.80).⁵

Figure 2.: Receiver operating characteristics curve for RESP variables and pre-ECMO support variables predicting survival. Note: A representative plot is included with AUC equivalent to average AUC across 30 imputed datasets. AUC, area under curve; RESP, respiratory extracorporeal membrane oxygenation (ECMO) survival prediction.

Discussion

In the current study, the RESP score was not as accurate in predicting hospital survival for COVID-19 patients compared to the cohort of patients used for RESP score creation.⁵ We offer several possible explanations for this difference in predictive value. First, as diseases and treatments evolve, especially in the necessarily-fluid context of care during a pandemic, perhaps the RESP score’s calibration and predictive value have shifted in ways that have not yet been measured.¹³ This finding has important implications and suggests that a recalibration of the RESP score may be necessary for different types of respiratory diseases and different treatment regimes. Second, differences in predictive value between the Schmidt et al. study and our study might be based on the lower survival rates for COVID-19 compared to other viral pneumonia diagnoses. Specifically, mortality of our cohort is 47% which is nearly 150% higher than mortality of viral pneumonia patients in the study by Schmidt et al.(30%, 77/260).⁵. Again, this finding points to the importance of a recalibration of the RESP score based on the type (and projected mortality rate) of the specific disease.

The increased mortality in this cohort could be caused by several factors, including the mortality of COVID-19 as compared to other viral pneumonias, especially for certain populations.¹⁴ Our cohort of patients had a higher obesity prevalence (60% vs. 3%) than those in Schmidt et al.’s study.⁵ Obese people are more prone to severe COVID-19 and poor outcomes.¹⁵ Also, our COVID-19 patients were on mechanical ventilation for longer time periods than the patients in Schmidt et al.’s study (89 hours vs. 57 hours) before initiation of ECMO, and they remained longer on ECMO support (354 hours vs 168 hours).⁵ Additionally, strains on health care resources (e.g., availability of providers and ICU beds) and alterations to care protocols (e.g., use of PPE, limiting investigations such as bronchoscopy) during the pandemic might have led to poorer outcomes for ECMO patients.¹⁶ Thus, patients who might have survived in nonpandemic situations might have expired during the pandemic because of myriad strains on the health care system.

Similar to Schmidt et al.’s findings, age, immunosuppression, duration of mechanical ventilation before ECMO, and cardiac arrest were significantly related to survival. In contrast, PIP (≥42), use of neuromuscular blockers, inhaled pulmonary vasodilators, CNS dysfunction, PaCO₂ (≥75), bicarbonate use, acute nonpulmonary associated infection were not associated with survival. Studies exploring the relationship between renal dysfunction or acute kidney injury and survival have produced mixed findings. A recent ELSO registry report found acute kidney injury was associated with higher mortality.⁹ In contrast, although renal dysfunction which involved both acute and chronic renal failure was identified as a potential candidate for inclusion in calculating the RESP score, it was not independently associated with survival and therefore was not included in the final RESP score.^5,14 Similarly, in our study, acute renal failure was a predictor of survival in bivariate analysis but failed to retain significance in multivariate analysis. A mixed set of findings have also been reported for the P/F ratio.⁹ Schmidt et al. report that the P/F ratio was not associated with hospital survival once a decision was made to put the patient on ECMO.⁵ In contrast, as per ELSO data on COVID-19, the P/F ratio was noted to have decreased hazard ratio 0.68 (0.57–0.81) for every doubling.⁹ Meanwhile, in our study, FIO₂ needed to maintain SaO₂ ≥94 was associated with survival.

Surprisingly, prone positioning and steroids were related to decreased survival. Randomized controlled trials have shown the beneficial effect of proning in the ARDS population,⁷ whereas the RECOVERY trial showed the beneficial effect of dexamethasone for patient needing oxygen support for COVID-19⁸. Likely, prone positioning and receiving steroids were confounded with higher severity of illness, which may account at least somewhat for the decreased survival rate. Our study comprised of patients from pre-RECOVERY and post-RECOVERY periods. As such, patients in the pre-RECOVERY period might have received corticosteroids on a compassionate basis and without following any protocol.⁸ This might have led to selection bias of the most severe patients in sub-cohort of patients treated with corticosteroids. Also, the original dataset for Schmidt did not have prone position available in the ELSO registry. Moreover, this finding indicates that the severity of illness was not fully captured by the set of measures included in either the RESP score or other variables included in this study.

Limitation

Our study is a retrospective review of registry data and does not take into account the various pharmacological treatments the patients received. Our goal was to evaluate RESP score for its performance based on registry data. The data are provided by a subset of health care centers that have resources to submit data. During the pandemic, these resources were severely limited and there might be bias in submitted patients’ data whereby centers that were overwhelmed might not have been able to submit full data. As a result, the ELSO database might have missing patient data leading to unknown bias. At the beginning of the pandemic, in the most severely affected areas, there were very limited resources to carry out ECMO except for select healthy young adults. Data from centers in these areas may have been severely skewed, to comprise only the fittest patients treated with ECMO. Despite this selection bias, COVID-19 patients’ survival is not better than viral pneumonia patients needing ECMO.

In the absence of randomized controlled trials (or prospective studies), ELSO registry data can provide guidance on utilizing RESP scores in patients with COVID-19. The distribution of COVID-19 patients across various classes of RESP was similar to Schmidt et al.’s reported distribution of RESP scores, with the majority of patients in classes II and III.⁵

In comparison to Schmidt et al. study, our study showed that survival rates were not as good for the RESP classes I–III, but they were better for classes IV–V⁵. Moreover, both the means and ranges in survival rates deviated greatly from the original RESP study, ranging from a low of 32% to a high of 69%. This suggests that even in patients with high RESP class, survival estimates predict that one in three patients will survive.

Conclusion

RESP score is not a robust predictor of survival for COVID-19 patients needing VV ECMO. The severity of illness of patients with COVID-19 was not fully captured by the set of measures included in the RESP score and other variables included in this study. Importantly, the use of RESP score is not recommended to be a sole measure of assessing candidacy for ECMO support for COVID-19 respiratory failure as it was created on patients already on ECMO. Rather, it is intended for use in conjunction with clinical judgment.¹⁷ The current study might help in prognosticating patients and discussing with family members possible outcomes for this novel disease.