Mortality and long-term outcome in children with congenital heart disease (CHD) have significantly improved over past decades because of the major improvements in the diagnostics, as well as the medical and surgical management of complex heart diseases.1,2 The majority of those patients are now living into adulthood, creating a new cohort of children and adults with CHD undergoing noncardiac surgical or invasive procedures.3
An increased incidence of major intra- and postoperative complications has been reported in this highly vulnerable population, as well as a higher mortality rate.4,5 In a recent study, we reported that children with major or severe CHD, as defined by a functional classification system, undergoing noncardiac surgery are at higher risk of mortality and have higher incidences of postoperative cardiorespiratory events, when compared with a matched group of children without CHD undergoing surgical procedures of comparable complexity.6 In that study, we also demonstrated that patients with minor CHD were not at increased risk when compared with children without CHD. The ability to determine differences in the risk of adverse outcomes among children with major and severe CHD is important for daily clinical decision making, and for the development of multidisciplinary strategies aimed at allocating perioperative material (eg, diagnostics and monitoring) and human (eg, cardiac versus pediatric anesthesiologist) resources.7
The objective of this study was to identify the predictors for postoperative mortality in children with major or severe CHD undergoing noncardiac surgery, and to develop a risk stratification score.
This study was performed using data from the 2012, 2013, and 2014 pediatric databases of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The ACS NSQIP Pediatric database includes de-identified data on children less than 18 years of age undergoing noncardiac surgery and includes 129 variables, including preoperative risk factors, intraoperative characteristics, 30-day postoperative outcomes, and in-hospital mortality in both the inpatient and outpatient settings.8 Adverse events and comorbidities reported in the database are determined by strict inclusion criteria. To ensure the quality of the data collected, the ACS NSQIP Pediatric conducts interrater reliability audits of selected participating sites.9 The results of the audits completed to date reveal an overall disagreement rate of approximately 2% for all assessed program variables. For the databases, exclusion criteria included: patients ≥ 18 years old, trauma cases, solid-organ transplantation, and cardiac surgery. Children undergoing multiple procedures performed by different surgical teams under the same anesthesia were also excluded from the database. In addition, cases coming from hospitals with an interrater reliability audit disagreement rate > 5%, or a 30-day follow-up rate < 80%, were excluded.
We included all children with major or severe CHD as defined and recorded in the 2012 and 2013 ACS NSQIP Pediatric databases in a derivation cohort, and those recorded in the 2014 database in a validation cohort. Based on the ACS NSQIP Pediatric database, children with major CHD are defined as children with a repaired CHD and a residual hemodynamic abnormality with or without medications (eg, tetralogy of Fallot with wide open pulmonary insufficiency, hypoplastic left heart syndrome including stage 1 repair), while children with severe CHD are defined as children with uncorrected CHD, children with documented pulmonary hypertension, children with ventricular dysfunction requiring medications, or children listed for heart transplant.6 We also used the International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes to categorize cardiac lesions into 3 subgroups: (1) septal defects and left ventricle outflow tract obstruction, (2) single-ventricle physiology (SVP), and (3) right ventricle outflow tract obstruction. The primary outcome variable for our analysis was the incidence of in-hospital mortality.
Children were stratified into 5 age groups: < 6 months, ≥ 6–12 months, ≥ 1–6 years, ≥ 6–12 years, and ≥ 12 years old. The following preoperative characteristics were included: age, sex, American Society of Anesthesiologists (ASA) physical status classification, prematurity (<24 weeks, 24–36 weeks, and >36 weeks), type of procedure (elective versus emergent surgery), surgical specialty (eg, thoracic and vascular surgery, general surgery, plastic and neurosurgery), preoperative respiratory disease (eg, asthma, chronic lung or airway diseases, cystic fibrosis), preoperative oxygen supplementation, tracheostomy, acute or chronic kidney disease, neurologic disease (eg, mental retardation, cerebral palsy, central nervous system disease, intracerebral hemorrhage), stroke, seizure, immune disease, preoperative use of steroids, preoperative inotropic support, preoperative mechanical ventilation, preoperative cardiopulmonary resuscitation (CPR), previous surgery within 30 days before the current procedure, and preoperative transfusion (defined as transfusion of whole blood or red blood cells during the 48 hours before surgery). A preoperative event was defined as an event occurring between the day of hospital admission and the day of the surgical procedure. Surgical complexity was assessed using each procedure’s relative value unit based on current procedure terminology codes.10 Relative value units have replaced the original ACS NSQIP complexity score as a measure of surgical complexity, and have been shown in database analyses to independently predict postoperative morbidity following general surgery.11,12
Categorical variables are expressed as number and percentage, and continuous variables are expressed as median and interquartile range. Preoperative variables were compared between survivors and nonsurvivors using χ2 or a Mann-Whitney U test. To control for possible confounding among variables, we used multivariable logistic regression using backward selection to determine the independent predictors for in-hospital mortality using a univariate cutoff of P< .10 for inclusion and P > .05 for removal. The results are expressed as regression coefficient (B) and standard error, the odds ratio (OR) as a measure of risk, the 95% confidence interval (CI), and P values obtained from the Wald test.13 The area under the receiver operating characteristics (ROC) curve was used as a measure of discrimination between those with and without in-hospital mortality. An area ≥ 0.750 was considered a good predictive accuracy.14
On the basis of the predictors obtained from multivariable logistic regression, we designed a risk stratification score, as a simplified algorithm to predict the incidence of in-hospital mortality. We considered a number of approaches to define the optimal scoring system. Because rounding to the nearest 0.5 integer was not associated with a significant difference in terms of discrimination and calibration, we decided to use a risk score of +1, +2, +3, +4, obtained by multiplying the regression coefficient by 2 and rounding to the nearest integer. This approach allowed us to define a score ranging from 0 to 10. We first assessed the discriminative ability of the risk score in the study cohort (2012–2013) based on the area under the ROC curve and then validated the risk score in our validation cohort (2014). Performance was evaluated by assessing the calibration and discrimination of both models. Calibration was assessed graphically by plotting the observed outcome against the predicted mortality. A smooth nonparametric calibration line was created with the locally weighted scatterplot-smoothing (LOESS) algorithm to estimate the observed probabilities in relation to the predicted probabilities.15,16 Discrimination was quantified by calculating the area under the ROC curve. Although area under the curve (AUC) is a commonly reported, optimistic estimation of the AUC is a frequent problem because of overfitting of statistical models. With the aim to correct for optimism in the derivation cohort (ACS NSQIP 2012–2013), we used a bootstrap procedure with a stratification for the type cardiac by repeatedly (repetition = 500 with replacement from the original data set) taking a large number of samples (sample size = 400, corresponding to the number of patients included in the smallest cardiac lesion subgroup). Our model applied to the bootstrap samples was used to determine the average “bootstrap” AUC. The AUC calculated when our model was applied to the derivation cohort was considered the “optimistic” AUC. The AUC when our model was applied to the validation cohort (ACS NSQIP 2014) was defined as the “naïve” AUC. We used the difference between “naïve” AUC and the “bootstrap” AUC to calculate “optimism.” We then calculated the optimism-corrected AUC as (“optimistic” AUC – “optimism”).17
Statistical analysis was performed using STATA (version 14.1 for Mac OS, Stata Corp, College Station, TX) with 2-tailed P value < .05 considered statistically significant for all analyses.
Among the 183,423 children included in the 2012, 2013, and 2014 ACS NSQIP database, and after exclusion of the patient without CHD and those with minor CHD, we included 4375 children with major or severe CHD in the derivation cohort and 2869 in the validation cohort. The incidence of mortality was 4.7% (204/4375) in the derivation cohort and 4.0% (115/2869) in the validation cohort.
Demographic and surgical characteristics as well as preoperative comorbidities are described in Table 1. Children who died were younger (P < .001), had higher ASA physical status classification (P < .001), had SVP (P < .001), and had a higher incidence of severe CHD (P < .001). They also underwent more emergency (P < .001) and complex procedures (P = .022), more general surgical procedures (P < .001), and fewer laparoscopic procedures (P = .004). In addition, nonsurvivors had more preoperative comorbidities (eg, respiratory diseases, oxygen supplementation, preoperative mechanical ventilation, acute or chronic kidney injury, preoperative CPR, inotropic support, previous surgery within 30 days, and preoperative transfusion) than those who survived (all P < .001).
Of those preoperative predictors for in-hospital mortality, 8 were retained in the final multivariable logistic regression model (Table 2): emergency procedure (OR: 1.66, 95% CI: 1.19–2.31, P = .003), severe CHD (OR: 1.65, 95% CI: 1.15–2.39, P = .007), SVP (OR: 1.83, 95% CI: 1.10–3.06, P = .020), previous surgery within 30 days (OR: 2.01, 95% CI: 1.40–2.89, P < .001), preoperative inotropic support (OR: 2.05, 95% CI: 1.40–3.01, P < .001), preoperative CPR (OR: 2.46, 95% CI: 1.32–4.57, P < .004), acute or chronic kidney injury (OR: 4.42, 95% CI: 2.00–9.75, P < .001), and mechanical ventilation (OR: 7.80, 95% CI: 5.42–11.21, P < .001). The multivariable logistic regression model showed a very good discrimination between those with and without in-hospital mortality (AUC of 0.837 [95% CI: 0.806–0.868]).
Based on the results obtained from multivariable logistic regression, we created a risk stratification score ranging from 0 to 10. The distribution of the risk stratification score and the associated observed in-hospital mortality in the derivation cohort are reported in Figure 1. Scores ≤ 3 are associated with a low risk of mortality (OR: 1.54, 95% CI: 0.78–3.04), scores ranging from 4 to 6 are associated with medium risk (OR: 4.19, 95% CI: 2.56–6.87), and scores ≥ 7 are associated with high risk (OR: 22.15, 95% CI: 15.06–32.59). The apparent “optimistic” AUC obtained in the derivation cohort (ACS NSQIP 2012–2013) was 0.836 (95% CI: 0.805–0.867). In the validation cohort (ACS NSQIP 2014), the score showed a very good discrimination with an area under the “naïve” ROC curve of 0.831 (95% CI: 0.787–0.875). Assessment of calibration showed good calibration (Figure 2), with a high concordance between the predicted probabilities obtained from the logistic regression and the observed frequencies using the risk stratification score. We also used a bootstrap procedure that validates the model for the population on which the population was drawn (ACS NSQIP 2012–2013), with an area under the bootstrap AUC of 0.841 (95% CI: 0.0.781–0.900) (Figure 3). The optimism in apparent performance was 0.010, corresponding to an optimism-corrected area of 0.826.
In our study, we were able to identify 8 predictors for in-hospital mortality in children with major and severe CHD undergoing noncardiac surgery. In addition to preoperative markers of critical illness (eg, inotropic support, mechanical ventilation, preoperative CPR, and acute or chronic kidney injury), we observed that the type of lesion (eg, SVP) and the functional severity of heart disease (eg, severe CHD) were excellent predictors of in-hospital mortality. SVP children appear to be at increased risk of in-hospital mortality regardless of functional status as defined by this database while children with non-SVP and disease classified as severe are at increased risk as compared with those with disease classified as major. Interestingly, other preoperative comorbidities (eg, respiratory or neurologic diseases) as well as the type and the complexity of the surgical procedure performed were not correlated with the incidence of in-hospital mortality.
Among the patients with CHD, children with SVP have been identified to be at high risk for perioperative complications.4,5,18 In a recent retrospective analysis of children with SVP undergoing noncardiac surgery, the incidence of intraoperative and early postoperative adverse events was as high as 11.8%.19 In the Pediatric Perioperative Cardiac Arrest Registry, the incidence of cardiac arrests among children with CHD was 34%, with 13% of the cardiac arrests occurring during noncardiac procedures performed in patients with SVP.18 In an analysis of the National Inpatient Sample, the incidence of in-hospital mortality among children with SVP undergoing noncardiac surgery varied between 17% and 22%.20 Even though mortality in children with SVP who undergo staged palliation has decreased significantly over the past decades, Fontan physiology and its long-term complications (eg, arrhythmias, ventricular failure) continue to pose significant challenges in the management of children requiring noncardiac surgical or invasive procedures.21 In children with a non–single-ventricle cardiac lesion, we also observed that the functional severity of the CHD should be considered an important predictor for in-hospital mortality. Indeed, children with uncorrected CHD, children with documented pulmonary hypertension, those with ventricular dysfunction requiring medications, and those listed for heart transplant (severe CHD) suffered a significantly higher incidence of in-hospital mortality when compared with children with repaired CHD and a residual hemodynamic abnormality (major CHD). Therefore, both the type of CHD and the functional severity of the lesion should be part of the preoperative risk stratification.
Among the 8 predictors for in-hospital mortality, 4 were markers of critical illness (eg, inotropic support, mechanical ventilation, preoperative CPR, and acute or chronic kidney injury). While these findings might seem intuitive and clinically consistent,22 our study failed to find a significant impact of other preoperative conditions (eg, chronic respiratory or neurologic diseases) on in-hospital mortality. Furthermore, our study failed to identify a relationship between the type and the complexity of the noncardiac surgical procedure performed and the incidence of in-hospital mortality.
The diversity of structural malformations in CHD, each with specific physiologic perturbations, hemodynamic consequences, and functional limitations, makes the development of general guidelines for perioperative management challenging.23 Children with CHD, particularly those with a residual lesion burden and compromised cardiovascular status, require an individualized approach to anesthetic and surgical care delivered by trained, multidisciplinary teams.24 To aid decision making and the development of guidelines for the perioperative management of this high-risk population, we have developed a risk stratification score that could be used to predict in-hospital mortality in children with major or severe CHD undergoing a noncardiac surgical or invasive procedure.
Despite the strengths of our methodology and the consistency of our findings, this study presents some major limitations. Analyses were performed using a large multi-institutional database that likely includes missing data, miscoded diagnoses, and miscoded procedures. However, ACS NSQIP is both a well-designed and well-administered database, which has rigorous quality controls built in. It is likely that the use of a clinical database such as ACS NSQIP provides more accurate information than an administrative database.25 It is also important to note that the ACS NSQIP does not allow the identification for the type and the characteristics of the hospital (eg, teaching, children’s hospital, bed-size), and the perioperative management strategies. In addition, all reported values and incidences are absolute values measured from the data set, and cannot be considered representative of national prevalence. Despite the use of multivariable logistic regression analysis, we could not exclude that some other potential confounding factors were not included in our analysis. Finally, use of International Classification of Diseases, Ninth Revision, Clinical Modification codes in conjunction with the ACS NSQIP Pediatric database allows identification of patients with SVP but does not provide accurate information on their specific stage of palliation.
In conclusion, our study demonstrates that, in addition to preoperative markers of critical illness (eg, inotropic support, mechanical ventilation, preoperative CPR, and acute or chronic kidney injury), the type of lesion (eg, SVP), and the functional severity of the heart disease (eg, severe CHD) are excellent predictors for in-hospital mortality in children undergoing noncardiac surgery. To improve decision making and to guide the development of perioperative management strategies in this high-risk population, we developed a risk stratification score that could be used to predict in-hospital mortality. Further studies are needed to assess whether the use of this risk stratification strategy could improve perioperative planning and management of children with major and severe CHD undergoing noncardiac surgery.
Name: David Faraoni, MD, PhD, FCCP.
Contribution: This author helped design the study, perform the statistical analysis, and write the manuscript.
Name: Daniel Vo, MD, FAAP.
Contribution: This author helped design the study and write the manuscript.
Name: Viviane G. Nasr, MD.
Contribution: This author helped design the study and write the manuscript.
Name: James A. DiNardo, MD, FAAP.
Contribution: This author helped design the study and write the manuscript.
This manuscript was handled by: Scott Beattie, MD, PhD.
1. Jortveit J, Øyen N, Leirgul E, et al. Trends in mortality of congenital heart defects. Congenit Heart Dis. 2016;11:160168.
2. Mazor Dray E, Marelli AJ. Adult congenital heart disease: scope of the problem. Cardiol Clin. 2015;33:503512, vii.
3. Sulkowski JP, Cooper JN, McConnell PI, et al. Variability in noncardiac surgical procedures in children with congenital heart disease. J Pediatr Surg. 2014;49:15641569.
4. Watkins SC, McNew BS, Donahue BS. Risks of noncardiac operations and other procedures in children with complex congenital heart disease. Ann Thorac Surg. 2013;95:204211.
5. Christensen RE, Gholami AS, Reynolds PI, Malviya S. Anaesthetic management and outcomes after noncardiac surgery in patients with hypoplastic left heart syndrome: a retrospective review. Eur J Anaesthesiol. 2012;29:425430.
6. Faraoni D, Zurakowski D, Vo D, et al. Post-operative outcomes in children with and without congenital heart disease undergoing noncardiac surgery. J Am Coll Cardiol. 2016;67:793801.
7. Benkwitz C, Watkins SC, Donahue BS. Assessing the risks of noncardiac surgery for children with congenital heart disease. J Am Coll Cardiol. 2016;67:802803.
9. Sellers MM, Reinke CE, Kreider S, et al. American College of Surgeons NSQIP: quality in-training initiative pilot study. J Am Coll Surg. 2013;217:827832.
10. Davenport DL, Henderson WG, Khuri SF, Mentzer RM Jr. Preoperative risk factors and surgical complexity are more predictive of costs than postoperative complications: a case study using the National Surgical Quality Improvement Program (NSQIP) database. Ann Surg. 2005;242:463468.
11. Johnson RG, Arozullah AM, Neumayer L, Henderson WG, Hosokawa P, Khuri SF. Multivariable predictors of postoperative respiratory failure after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg. 2007;204:11881198.
12. Neumayer L, Hosokawa P, Itani K, El-Tamer M, Henderson WG, Khuri SF. Multivariable predictors of postoperative surgical site infection after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg. 2007;204:11781187.
13. Katz MH. Interpreting the results. Multivariable Analysis: A Practical Guide for Clinicians and Public Health Researchers
. 2011Cambridge: Cambridge University Press.
14. Goobie SM, Zurakowski D, Proctor MR, et al. Predictors of clinically significant postoperative events after open craniosynostosis surgery. Anesthesiology. 2015;122:10211032.
15. Le Manach Y, Collins G, Rodseth R, et al. Preoperative score to predict postoperative mortality (POSPOM): derivation and validation. Anesthesiology. 2016;124:570579.
16. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD). Ann Intern Med. 2015;162:735736.
17. Smith GC, Seaman SR, Wood AM, Royston P, White IR. Correcting for optimistic prediction in small data sets. Am J Epidemiol. 2014;180:318324.
18. Ramamoorthy C, Haberkern CM, Bhananker SM, et al. Anesthesia-related cardiac arrest in children with heart disease: data from the Pediatric Perioperative Cardiac Arrest (POCA) registry. Anesth Analg. 2010;110:13761382.
19. Brown ML, DiNardo JA, Odegard KC. Patients with single ventricle physiology undergoing noncardiac surgery are at high risk for adverse events. Paediatr Anaesth. 2015;25:846851.
20. Torres A Jr, DiLiberti J, Pearl RH, et al. Noncardiac surgery in children with hypoplastic left heart syndrome. J Pediatr Surg. 2002;37:13991403.
21. Pundi KN, Johnson JN, Dearani JA, et al. 40-year follow-up after the Fontan operation: long-term outcomes of 1,052 patients. J Am Coll Cardiol. 2015;66:17001710.
22. Shi WY, Rouse M, Weintraub RG, et al. Predictors of outcomes in children awaiting heart transplantation: an experience from a National Paediatric Heart Transplantation Programme. Eur J Cardiothorac Surg. 2016:ezv465.
23. Benson DW Jr. Changing profile of congenital heart disease. Pediatrics. 1989;83:790791.
24. Gottlieb EA, Andropoulos DB. Anesthesia for the patient with congenital heart disease presenting for noncardiac surgery. Curr Opin Anaesthesiol. 2013;26:318326.
Copyright © 2016 International Anesthesia Research Society
25. Bohl DD, Russo GS, Basques BA, et al. Variations in data collection methods between national databases affect study results: a comparison of the nationwide inpatient sample and national surgical quality improvement program databases for lumbar spine fusion procedures. J Bone Joint Surg Am. 2014;96:e193.