Since 1976, extracorporeal membrane oxygenation (ECMO) has continued to aide in the treatment of severe respiratory disease in pediatric patients.1–4 Early utilization of ECMO technology was frequently associated with complications, particularly with oxygenator failure during the era of silicone (and to a lesser extent polypropylene hollow fiber) oxygenators and risks for bleeding. From 1980 to 2004, the average neonatal ECMO run was just under 200 hours.5 With the advent of newer technologies, such as the polymethylpentene oxygenator and improved centrifugal pumps, more centers are utilizing ECMO for expanding indications and for longer periods of time. The average neonatal respiratory run from 2005 to 2013 was 243 hours, and the average of the longest recorded runs each year increased from 512 hours to 615 hours. Further, an analysis of the ELSO registry by era (Figure 1A), with the modern era starting in 2005, demonstrates a significant improvement (p < 0.05) in survival for patients requiring long (>250 hours) ECMO runs. Despite this increased survival for longer ECMO runs, there is a 10% per decade decline in overall neonatal VA ECMO survival (Figure 1B). We hypothesize that this may be related to an increase in the severity of illness in this population, but no risk adjustment index is currently available. Risk adjustment has been used in other areas of pediatric critical care medicine to facilitate comparison of patient populations and to provide the ability of institutions to benchmark their outcomes against peer institutions.6–9 The goal of this work was to analyze the ELSO data registry for neonatal ECMO patients on VA ECMO for respiratory disease and develop risk-adjustment scores based primarily on pre-ECMO factors to allow for more robust comparisons between eras and benchmarking for institutions.
Data were obtained from the ELSO registry for all neonatal VA Respiratory ECMO patients from 2000 to 2010. This era was chosen to be the modern era where high frequency oscillator ventilation and nitric oxide therapy have been deployed in efforts to salvage patients before ECMO and spanned the introduction of newer oxygenator and pump technology. The analysis of VA as the only modality was performed because it is the most widely used mode of ECMO in this patient population.10 Data were imported into a statistical software package for analysis (SPSS v21, IBM). A total of 5,409 records were reviewed. Because the ELSO registry is not an adjudicated data set, the weight-normalized flow at 4 hours and 24 hours on ECMO was used as quality control metric. Four and twenty-four hour ECMO flows that were greater than 250 ml/kg/min were eliminated from further analysis. Likewise, 4 hour ECMO flows that were less than 10 ml/kg/min were eliminated from further analysis. This primary triage resulted in the removal of 554 patients from analysis.
To determine the probability of survival, the methodology outlined by Orr et al.9 was used. Briefly, a univariate analysis was performed on the pre-ECMO variables and on-ECMO variables to identify factors associated with survival to discharge. Fisher’s exact test was used for categorical variables, and an independent sample Mann–Whitney U test was used to compare survivors to nonsurvivors. Those parameters with p < 0.25 were entered into a logistic regression model for survival to hospital discharge with forward entry (p < 0.01) and removal (p > 0.05) criteria. The model was built from a random sampling of the 4,855 patients using a Bernoulli function to create a training set containing 70% of the data. Patients not selected for the training set were used as the validation set. Univariate analysis was performed to ensure that there was no significant difference in any of the variables between training and validation sets (data not shown).
After the first logistic regression, continuous variables that remained in the model were examined using supervised optimal binning techniques.11 Optimal binning is a process by which a continuous variable is discretized against a binary outcome using entropy weighting to determine whether there is a point (or points) in the continuous variable where the variable can be grouped according to its impact on the outcome variable. The impact of discretizing the continuous data points was analyzed by looking at the receiver operator curves (ROCs) for models derived from the continuous and discrete variables (Figure 2). The binary variables replaced the continuous variables in a forward-entry logistic regression model with the same training set to derive a scoring model. Resulting beta coefficients were used to derive a points system for pre-ECMO variables to be added together to create the Pittsburgh Index for Pre-ECMO Risk (PIPER). The PIPER score was then used as the sole input to a logistic regression model, creating a linear equation, r =a × PIPER + constant, to give the probability for death before hospital discharge as defined by Pr(death) = er/(1 + er). A receiver operating characteristic (ROC) for the probability of death versus survival in the validation data set was computed to determine the accuracy of the iterations of the PIPER model to predict hospital mortality. On-ECMO data, including length of time on ECMO and associated complications were examined with respect to the PIPER score. An augmented score (PIPER+) was created to include broad classifications of complications (e.g., mechanical, neurologic, hematologic). Logistic regression with the PIPER+ was performed and ROC generated. Comparisons of the area under ROC (AUROC) for each model tested were performed.12
Univariate analysis of the pre-ECMO data revealed 26 variables related to hospital survival (p < 0.25; Table 1). Variables that did not appear related (p > 0.25) were not included in the table or further analyses. Because of the large N for the dataset, many variables were found to be significantly associated despite having similar medians, which is the reason that a multivariate model was used to isolate the independent factors impacting survival. All variables in Table 1 were placed in the model except race, sex, and SVO2, and SPO2. Although they appear to show some relationship to patient survival, race and sex were excluded because they are not clinical variables that indicate the current severity of illness of a patient. SVO2 and SPO2 were excluded because these variables were recorded in only a small percentage of the data set, and significantly limited the number of patients that could be entered into the regression. Of the 3,410 patients that were selected as the training set, 1,501 had complete data and were used to generate the model.
After the first binary logistic regression, the pre-ECMO variables that remained in the model were as follows: Apgar at 5 minutes, birth weight, age, FiO2, pH, pO2, inhaled nitric oxide (iNO), Hours from intubation until ECMO, and mean arterial pressure. This model had a Hosmer-Lemeshow result of p = 0.921, which indicated a good fit. For the continuous variables in the model, the optimal binning algorithm was performed for each variable against survival to discharge, and the results are found in Table, Supplemental Digital Content 1 (http://links.lww.com/ASAIO/A100). Optimal binning algorithms were unsuccessful for FiO2 and time from Intubation to ECMO, which indicated that the algorithm could not find a specific point where each variable conferred a survival advantage.
Forward-entry logistic regression was then performed on the binned variables and the remaining continuous variables (FiO2 and time from intubation to ECMO), which resulted in a model that retained all the binned variables. The continuous variables FiO2 and time from intubation to ECMO were found to be no longer sufficient to remain in the model when the binned variables were employed. The Hosmer-Lemeshow statistic (p = 0.166) indicated that the model fit was adequate. Furthermore, AUROC for the original continuous variable model (0.74, 0.72–0.77) and the model containing the optimally binned variables (0.73, 0.71–0.76) were not statistically different (Figure 2), which indicated that we were able to preserve the sensitivity and specificity of the model without these two variables. The beta coefficients of each of the variables in the model (see Table, Supplemental Digital Content 2, http://links.lww.com/ASAIO/A101) were used to create the PIPER score (Table 2).
Logistic regression on the PIPER score as the lone variable yielded the equation rPIPER = 0.065 × PIPER − 2.416, and the probability of death is Pr(Death) = erPIPER/(1 + erPIPER). The Hosmer-Lemeshow test for this model (p = 0.51) demonstrated a good fit. The AUROC for the PIPER Score was 0.73 (95% confidence interval [CI]: 0.70–0.75), and was not statistically different from the AUROC for the continuous variables or the binned variables (Figure 2). Plotting the quartiles of the PIPER score against the percent mortality in each quartile revealed an increase in mortality of approximately 15% per PIPER quartile from a baseline mortality of 12% to a peak mortality of 60% (Figure 3). In addition, the predicted mortality for the upper edge of each quartile was computed using the equation above and added to Figure 3. Comparing this to the all-comers mortality of 37% demonstrates the ability of the PIPER score to stratify expected mortality over the general population mortality.
The relationship of on-ECMO variables to survival was also evaluated (see Table, Supplemental Digital Content 3, http://links.lww.com/ASAIO/A102). The length of the ECMO was significantly associated with mortality (p ≤ 0.001), and ECMO duration positively correlated with the PIPER score (ρ = 0.366, p < 0.001). The odds ratio (with 95% CI) of death with any complication was 3.3 (2.8–3.9). The odds ratio for each complication category is found in Table 3. Complications were prevalent on ECMO, with 69% of patients experiencing at least one complication. There was also a 5% increase in the incidence of complications per PIPER quartile (see Figure, Supplemental Digital Content 1, http://links.lww.com/ASAIO/A99). The percentage of patients within in each complication category listed in Table, Supplemental Digital Content 3, http://links.lww.com/ASAIO/A102, increased with PIPER quartiles for hemorrhagic, mechanical, neurologic, metabolic, pulmonary, and renal complications (data not shown). There was also a statistically significant relationship between the median length of ECMO and the occurrence of all complications except neurologic complications (see Table, Supplemental Digital Content 4, http://links.lww.com/ASAIO/A103). Neurologic complications were significantly associated (p < 0.05) with lower birth weight (<3 kg). Lower birth weight had an odds ratio of 1.2 (95% CI: 1.1–1.5) for neurologic complications.
Because ECMO duration and the incidence of complications were both found to be associated with mortality, a combined logistic regression model was created that included the PIPER score, ECMO duration, and presence of each of the complication categories above as independent variables. The model resulted with a good fit (Hosmer-Lemeshow p = 0.86) with the following variables remaining in the model: PIPER, hours on ECMO, hematologic complication, neurologic complication, metabolic complication, pulmonary complication, and renal complication. The AUROC for this combined model was 0.79 (95% CI: 0.77–0.81). This AUROC was significantly better (p < 0.05) than the PIPER model alone (Figure 5).
A composite score (PIPER+) was created based upon the original PIPER score and the beta coefficients of these parameters from the model (Table 4) and was placed into a binary logistic regression model as the sole parameter. The resultant model (r = 0.054 × PIPER ± 3.62) fit the data well (Hosmer-Lemeshow p = 0.312) and had AUROC of 0.81 (95% CI: 0.79–0.83), which similar to that of the composite model containing the individual components (Figure 5). Quartiles for this composite score were similarly associated with increased mortality (Figure 6), with each quartile adding 21% risk of mortality from a baseline of 7% to a peak of 71%. Predicted mortality for the average point in each quartile was also computed to show the accuracy of the algorithm to the actual patient values.
Utilizing pre-ECMO data from the ELSO registry, we created the PIPER score and found a clear increase in mortality with PIPER quartiles (Figure 4). Thus, the PIPER score appears to provide an adequate metric for risk-adjusting patients before initiation of ECMO. Increasing PIPER scores were also associated with longer ECMO runs and a higher propensity for complications, both of which were associated with higher mortality rates and prompted the creation of the PIPER+ score. The PIPER+ score had better sensitivity and specificity for hospital mortality (Figure 5). The increased AUROC occurred from the adjustment of mortality upwards at the highest quartile for all-comers. In the PIPER+ model, mortality reached 70% (Figure 6) compared with the PIPER model, which showed a peak mortality of 60% for all-comers (Figure 3). This is likely a reflection of the increasingly negative impact of complications that result from longer ECMO runs. Further, the predicted probability for each at the midpoint of the quartiles was found to closely match the actual mortality at the first three quartiles. In both scoring systems, the mortality of the highest quartile was over-predicted by approximately 15%. Future work to refine the prediction at this level is needed to bring it into closer alignment with the actual data and was likely driven by the lower numbers of patients that are found at such extremes.
Because the assigned points for each parameter in the PIPER and PIPER+ scores are based on the beta coefficients from the multivariate analysis, they also represent a glimpse into the relative impact of various parameters on the mortality of these ECMO patients. Those parameters with higher points have a greater impact on the outcomes. Although many of the parameters that make up the PIPER and PIPER+ scores have previously been found to be negatively associated with survival, none have been put together in a risk stratification score. Congenital diaphragmatic hernia (CDH), which has long been associated with higher mortality,5,13–15 had the highest impact of the pre-ECMO variables. Forty-eight percent of the patients in our dataset had CDH, with a 57% hospital mortality rate that was similar to the highest mortality rate in the upper PIPER quartile. At first glance, CDH may appear to have dominated the dataset. Figure 4 shows that CDH patients are found in three of the four quartiles and provided a greater proportion of the patients in the highest mortality group. However, the mortality rates of CDH, non-CDH, and the combined cohort for each PIPER quartile were similar (Figure 3). Thus, we can conclude that the PIPER score offers risk stratification over simple CDH classification and provides evidence that not all CDH patients should be expected to have such a high mortality.
Patients placed on ECMO at day of life 10 or later had an equally stiff penalty as the CDH patients. The older age of patients upon initiation of ECMO being related to decreased survival has also been shown in an ELSO registry from the same time period.16 The patients in that study cannulated for ECMO 7 days after birth had poorer outcomes and were associated with higher ventilator settings. The increased ventilator settings may be an indication that greater lung injury was created as a result of trying to keep the patient off ECMO, or it may be indicative of a refractory lung injury to which ECMO is the next logical treatment. Because the possible reasons for the delay between birth and ECMO vary in their underlying etiology, this area should be further investigated with a study specifically powered to examine it.
Poor Apgar scores, lower pH, and lower oxygen indexes have all been previously described as playing a role in the increased mortality of ECMO patients.17–19 These parameters are all indicative of poor respiratory reserve or lack of responsiveness to conventional therapy and which may lead a patient toward ECMO therapy. Whether they represent a true increase in mortality or point to areas that deserver further study on aggressive intervention is unclear and should be investigated further. Similarly, our finding that the use of iNO was protective in these patients is counterintuitive and no apparent relationships within the dataset could provide sufficient explanation of this phenomenon. Inhaled nitric oxide has been successfully used to treat respiratory failure and related pulmonary hypertension in term and near-term neonates,20 and may be useful in reducing the need for ECMO.14 As might be expected in a population that did go onto ECMO, the overall usage rate of iNO was very high, with 84% of patients receiving iNO pre-ECMO. We do not find that the use of iNO was associated with the underlying disease (CDH patients were only 4% less likely to receive it than all other patients). Instead, we hypothesize that it may be related to the rapid progression of the disease. We found that there was a reduction in the length of time from intubation to ECMO for patients who did not receive iNO (median 21 hours vs. 27 hours, p < 0.05). Again, because the length of time from intubation to ECMO was not independently associated with mortality by the multivariate regression, we can only speculate as to the reasons for the association and further studies should be undertaken to examine this phenomenon specifically.
In addition to the pre-ECMO parameters, consideration must also be given to events on ECMO that influence outcomes. Increasing PIPER scores were not only associated with higher mortality, but were also associated with longer ECMO runs and higher frequencies of complications. Each of these factors has previously been associated with higher mortality.13,15,16,21–26 The one complication category that was not associated with time on ECMO was neurological complications. It is likely that neurologic complications occurred within the first 7 days (medians were 175 hours) and may have resulted in redirection of care compared with other complications, which may have occurred later or did not cause redirection of care. Neurologic complications were more likely in patients with lower birth weights, which may be associated with younger gestational ages.16 Because gestational age was not available in this data set, lower birthweight may therefore be acting as a surrogate for this parameter.
One of the primary limitations of this study is that the ELSO Registry from which these data are drawn is a self-reported database with some overlapping and unclear definitions for complications and interventions. We attempted to limit the impact of this by performing a triage of data using unrealistic flow rates as a surrogate for improper data entry. However, for complications the absence of data is considered the same as no complication; something which we hope that planned updates to the database will address. Another limitation is that we examined VA ECMO only. We chose VA ECMO as the sole modality for analysis because it is historically the most widely practiced form of ECMO in this patient population.10 Although venovenous (VV) ECMO is possible in neonates,27 it is utilized in approximately 30% of the patients per year.10 The decision to use VA versus VV can often be based on institutional experience and policies, but it is increasingly chosen for patients who have singular pulmonary dysfunction. Given the findings of our study, subsequent analysis of the PIPER and PIPER+ scores in the VV ECMO population should certainly be undertaken. This is of particular interest given that one of the benefits of VV ECMO is its potential for fewer neurologic events, which provides a significant penalty in the PIPER+ scoring system.
It is our hope that using a pre-ECMO scoring system can be used for propensity matching of patients to compare treatment modalities (equipment or medical management strategies) and to offer families a realistic expectation of the chance for survival going into ECMO. Although the major limitations of this study were that it relied on a nonadjudicated, retrospective data set and included a specific modality of ECMO, multiple pre-ECMO variables were able to be regressed against survival to discharge and we could determine a subset that were independently associated with in-hospital mortality. Because the data required to compute the PIPER and PIPER+ models is already collected for the ELSO registry, validation of the models could be readily completed through a multicenter research study or in a single high volume center. Future work will also attempt to incorporate a larger patient population (pediatric and VV ECMO) as well as other pre-ECMO variables not currently included in the ELSO registry. These include lactate levels, length of CPR before cannulation, as well variables found in other scoring indices such as the Pediatric Risk of Mortality (PRISM) and the Pediatric Multiple Organ Dysfunction Score (P-MOD).
The authors thank the help of Peter Rycus at ELSO for his role in maintaining the ELSO Registry and providing the data for this manuscript.
1. Bartlett RH, Roloff DW, Cornell RG, Andrews AF, Dillon PW, Zwischenberger JB: Extracorporeal circulation in neonatal respiratory failure: A prospective randomized study. Pediatrics 1985.76: 479–487.
2. Peek GJ, Elbourne D, Mugford M, et al: Randomised controlled trial and parallel economic evaluation of conventional ventilatory support versus extracorporeal membrane oxygenation for severe adult respiratory failure (CESAR). Health Technol Assess 2010.14: 1–46.
3. McNally H, Bennett CC, Elbourne D, Field DJ; UK Collaborative ECMO Trial Group: United Kingdom collaborative randomized trial of neonatal extracorporeal membrane oxygenation: Follow-up to age 7 years. Pediatrics 2006.117: e845–e854.
4. UK Collaborative ECMO Trial GroupUK collaborative randomised trial of neonatal extracorporeal membrane oxygenation. Lancet 348: 75–82–1996.
5. ELSO Registry Report: 2013.Ann Arbor, MI: Extracorporeal Life Support Organization.
6. Pollack MM, Ruttimann UE, Getson PR: Pediatric risk of mortality (PRISM) score. Crit Care Med 1988.16: 1110–1116.
7. Pollack MM, Patel KM, Ruttimann UE: PRISM III: An updated pediatric risk of mortality score. Crit Care Med 1996.24: 743–752.
8. Boyd CR, Tolson MA, Copes WS: Evaluating trauma care: The TRISS method. Trauma score and the injury severity score. J Trauma 1987.27: 370–378.
9. Orr RA, Venkataraman ST, McCloskey KA, et al: Measurement of pediatric illness severity using simple pretransport variables. Prehosp Emerg Care 2001.5: 127–133.
10. ECLS Registry Report: US Summary. 2015.Ann Arbor, MI: Extracorporeal Life Support Organization.
11. Fayyad UM, Irani KB. Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning. Proceedings of the International Joint Conference on Uncertainty in AI. 1993. Morgan Kaufmann, San Francisco, 1022–1027.
12. Hanley JA, McNeil BJ: A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983.148: 839–843.
13. Jaillard S, Pierrat V, Truffert P, et al: Two years’ follow-up of newborn infants after extracorporeal membrane oxygenation (ECMO). Eur J Cardiothorac Surg 200018: 328–333.
14. Muraca MC, Negro S, Sun B, Buonocore G: Nitric oxide in neonatal hypoxemic respiratory failure. J Matern Fetal Neonatal Med 25(suppl 1): 47–50, 2012.
15. Prodhan P, Stroud M, El-Hassan N, et al: Prolonged extracorporeal membrane oxygenator support among neonates with acute respiratory failure: A review of the Extracorporeal Life Support Organization registry. ASAIO J 2014.60: 63–69.
16. Smith KM, McMullan DM, Bratton SL, Rycus P, Kinsella JP, Brogan TV: Is age at initiation of extracorporeal life support associated with mortality and intraventricular hemorrhage in neonates with respiratory failure? J Perinatol 2014.34: 386–391.
17. Green TP, Timmons OD, Fackler JC, Moler FW, Thompson AE, Sweeney MF: The impact of extracorporeal membrane oxygenation on survival in pediatric patients with acute respiratory failure. Pediatric Critical Care Study Group. Crit Care Med 1996.24: 323–329.
18. Mehta NM, Turner D, Walsh B, et al: Factors associated with survival in pediatric extracorporeal membrane oxygenation–a single-center experience. J Pediatr Surg 2010.45: 1995–2003.
19. Barrett CS, Jaggers JJ, Cook EF, et al: Outcomes of neonates undergoing extracorporeal membrane oxygenation support using centrifugal versus roller blood pumps. Ann Thorac Surg 2012.94: 1635–1641.
20. DiBlasi RM, Myers TR, Hess DR: Evidence-based clinical practice guideline: inhaled nitric oxide for neonates with acute hypoxic respiratory failure. Respir Care 2010.55: 1717–1745.
21. Nehra D, Goldstein AM, Doody DP, Ryan DP, Chang Y, Masiakos PT: Extracorporeal membrane oxygenation for nonneonatal acute respiratory failure: The Massachusetts General Hospital experience from 1990 to 2008. Arch Surg 2009144: 427–432–XXX.
22. Mehta A, Ibsen LM: Neurologic complications and neurodevelopmental outcome with extracorporeal life support. World J Crit Care Med 2013.2: 40–47.
23. Bartlett RH, Gattinoni L: Current status of extracorporeal life support (ECMO) for cardiopulmonary failure. Minerva Anestesiol 2010.76: 534–540.
24. Donn SM: ECMO indications and complications. Hosp Pract (Off) 1990.25: 143–146, 149–150, 153–157.
25. Geven WB, Nabuurs-Korhman LA, Van Kessel-Feddema JM, Festen C: Follow-up in neonatal extracorporeal membrane oxygenation (ECMO). Int J Artif Organs 1995.18: 584–588.
26. Zwischenberger JB, Cox CS Jr: Extracorporeal membrane oxygenation (ECMO) for neonatal respiratory failure. Thorac Cardiovasc Surg 1992.40: 316–322.
27. Lazar DA, Cass DL, Olutoye OO, et al: Venovenous cannulation for extracorporeal membrane oxygenation using a bicaval dual-lumen catheter in neonates. J Pediatr Surg 201247(2): 430–434.
extracorporeal membrane oxygenation; neonatal; outcomes; risk adjustment
Supplemental Digital Content
Copyright © 2016 by the American Society for Artificial Internal Organs