Analysis of outcomes after congenital cardiac surgery is a complex problem and requires a reliable and reproducible process. With the advent of many surgical techniques for congenital heart disease the complexity stratification tool has become extremely useful for the analysis of outcome [1,2]. Though complexity stratification may not be the only factor determining outcome it remains an important tool for mortality and morbidity assessment post cardiac surgery.
The Risk Adjustment for Congenital Heart Surgery (RACHS-1) method was developed by the Children's Hospital Boston team and allocated 207 surgical procedures into 6 different categories having similar risk for hospital mortality . Later in 1999 Lacour Gayet and a committee of experts created a tool for stratification of complexity called as Aristotle Basic Complexity (ABC) scores . The score was developed by a group of 50 surgeons from 23 countries who postulated the complexity of a procedure as sum of 3 factors: potential for operative mortality; potential for operative morbidity and technical difficulty of the surgery. Each surgical procedure received a score ranging from 0.5 to 15 and was divided into categories according to the score: Level 1 (1.5–5.9); Level 2 (6.0–7.9); Level 3 (8.0–9.9) and Level 4 (10.0–15.0). This was followed by Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery (STS-EACTS) mortality score, published in 2008 which was developed primarily using the real database from 77,294 patients (33,360 patients from the EACTS and 43,934 patients from the STS between 2002 and 2007 . Using Bayesian statistics that fit the data for small denominators, mortality rates were calculated for each procedure. Each procedure received a score ranging from 0.1 to 5.0, based on the estimated mortality and the procedures were grouped into 5 categories.
The objective of the current study was to compare the RACHS-1, ABC and STS-EACTS complexity scoring models for predicting the length of PICU stay and hospital mortality after surgery for congenital heart disease.
2. Material and methods
This retrospective study was conducted in 65-bedded cardiac PICU of a tertiary care cardiac hospital. Patient ≤18 years of age undergoing cardiac surgery for congenital heart disease on cardio-pulmonary bypass between June and November 2015 were enrolled in the study. Data were collected from the case records and PICU charts of the patients that included demographic parameter, operative diagnosis, and the procedure performed. Peri-operative data including cardio-pulmonary bypass (CPB) time and Aortic cross clamp (AXC) time were also recorded.
Patients were allocated to risk categories according to the RACHS-1 system. ABC scores and STS-EACTS scores were calculated, and patients were allotted to risk categories based on standard charts. Operations involving two or more procedures done concurrently were categorized for the procedure with the higher risk category. Primary outcome indicator in the study was PICU length of stay (PLOS) and hospital mortality. For further analysis PLOS was dichotomized as upper (worst) 25th percentile versus lower (best) 75th percentile.
Statistical analysis was performed using SPSS software. Continuous data were expressed as mean ± SD and categorical data as absolute number and percentage. The analysis was done for overall performance including the extent to which the model accurately predicts the dependent variable, which indicates the goodness of fit (calibration) and ability to separate subjects who experienced the outcome event, from the others (discrimination). Discriminative power of scoring systems was assessed using area under the curve (AUC) of receiver operating characteristic curves (ROC), and z-statistics was applied to compare AUC of individual ROC curves. Calibration was measured using Hosmer-Lemeshow modification of chi-square test. P < 0.05 was considered as significant.
The study included 920 patients with a mean age of 46.2 ± 48.2 months. The baseline characteristics of the study population are shown in Table 1. Commonest procedure performed was Ventricular septal defect closure and Intra-cardiac repair of Tetralogy of Fallot. None of the procedure was classified in category 5 and 6 of RACHS-1 model or category 5 of STS-EACTS scoring model. Median PLOS of the study population was 60 h and the cut-off for prolonged PLOS was calculated as 87 h (upper 25th percentile). Patients with prolonged PLOS were of younger age and had longer CPB and AXC time. Overall hospital mortality of the cohort was 3.04% (n-28).
The distribution of scores over the entire cohort is shown in Table 2. An increase in the percentage of patients requiring prolonged PLOS and increasing mortality was noted with the increase in complexity grade of the procedures. When modeled for multivariate logistic regression analysis, the RACHS-1 (χ2 – 12.54; p – 0.129), ABC (χ2 – 6.302; p – 0.613) and STS-EACTS (χ2 – 9.096; p – 0.334) models showed good fit for prolonged PLOS on calibration. To predict hospital mortality the multivariate logistic regression analysis also showed good fit for RACHS-1 (χ2 – 7.85; p – 0.449), ABC (χ2 – 6.37; p – 0.606) and STS-EACTS (χ2 – 7.85; p – 0.837) models on calibration.
The AUC of the ROC curves for the scores to predict prolonged PLOS (Fig. 1) and hospital mortality is shown in Table 3. STS-EACTS outclassed RACHS-1 and ABC model with AUC of 0.759 for prolonged PLOS and 0.870 for hospital mortality. RACHS-1 model was found to have the worst predictive value for both prolonged PLOS (0.701) and hospital mortality (0.766). As depicted in Table 4 statistically significant difference was found between the AUC of ROC curves of STS-EACTS and RACHS-1 model for both prolonged PLOS (p – 0.046) and hospital mortality (p – 0.015). No significant difference was observed between the AUC of ROC curves of other complexity scoring models.
In the present cohort, STS-EACTS model was found to be the best predictor of both PLOS and hospital mortality followed by ABC which performed better than the RACHS-1 model. AUC of ROC curve for STS-EACTS model was found to be significantly larger than that of RACHS-1; however no significant difference was found with ABC model. Although all 3 complexity scoring models have been validated to predict the mortality and outcome post cardiac surgery no clear superiority of one over another exists. When applied to the STS database both ABC and RACHS-1 were found to be slightly different with ABC method allows classification of more operations, whereas the RACHS-1 system discriminates better at the higher end of complexity . STS-EACTS model is in fact considered superior to the earlier two models due to the fact that they stratify the mortality according to real data for each surgical procedure based on the STS-EACTS multicenter database . The earlier models were developed by the panel of experts based on the clinical judgment and experience and were highly subjective.
The Al-Radi et al. compared the predictive value of the RACHS-1 and ABC model for the hospital mortality outcome and found RACHS-1 to be better than the ABC scoring model . In a study done by Joshi et al. ABC model was found to be better than RACHS-1 with net reclassification improvement of 43% from RACHS-1 to ABC . O'Brien et al. with the introduction of STS-EACTS model compared the new model with its predecessors and found the discriminatory capacity of STS-EACTS to be higher than the RACHS-1 and ABC . Cavalcanti et al. compared the RACHS-1, ABC and STS-EACTS model and found no significant difference in predicting mortality . Another study by Vasdev et al. found the ability to predict mortality of the RACHS-1 similar to that of the STS-EACS with both better compared to the ABC score .
Although the complexity stratification of procedures are common and has been validated important limitations exists. Each child and the corresponding procedure are unique and multiple factors determine the outcome. These include patient related and patient independent factors. Three additional clinical factors (age, prematurity and non-cardiac congenital structural abnormalities) were used in the RACHS-1 score to complement the model and, when used, increase the discriminatory power of the model . The Aristotle score also received some refinements and the so-called “Comprehensive Aristotle (ACC) Score” adds some procedure-dependent factors, including anatomical factors, associated procedures, and age at procedure and procedure independent factors, including general factors such as weight and prematurity, clinical factors such as preoperative sepsis or renal failure, extracardiac and surgical factors .
DeCampli et al. found ACC to have better predictive value than ABC and RACHS-1 for both mortality and length of hospital stay . Joshi et al. found ACC to have better discriminatory power than both ABC and unadjusted RACHS-1 with net reclassification improvement index of 34.76% of cases from ABC to ACC and 30% from RACHS-1 to ACC with significant statistical importance . Bojan et al. found ACC score to be better than the unadjusted RACHS-1 score in predicting hospital mortality; however the difference in the discriminatory capacity was lost when using adjusted RACHS-1 score .
The overall mortality rate of the cohort was 3% with the mortality rate in each category consistent with the predicted mortality by models [3,4]. Majority of the procedure in the current study were stratified as low risk which could explain the good outcome in the cohort. We did not include the surgeries done without CPB which excluded the majority of palliative procedures who are likely to have poor outcome post-operatively.
The main limitation of this study is that it was single institution based and had a small sample size. Due of the retrospective collection of data the variables accounting for the comprehensive complexity could not be included and missing data cannot be ruled out. A more collaborative effort is needed by the primer pediatric cardiac institute of the country to make a data registry of the procedures and outcome. More studies are needed to validate these scoring systems which are based on data from developed countries before applying it to patients in a developing country like ours.
Risk stratification for pediatric heart surgery is a useful tool to predict the outcome. Although all 3 models are effective in predicting the outcome STS-EACTS risk stratification model has the best discriminative power.
Source(s) of support
Presentation at a meeting
 Jacobs ML, Jacobs JP, Jenkins KJ, Gauvreau K, Clarke DR, Lacour-Gayet F. Stratification of complexity: the risk adjustment for congenital heart surgery-1 method and the Aristotle complexity score-past, present, and future. Cardiol Young 2008 Dec;18(Suppl 2):163-168.
 Clarke DR, Lacour-Gayet F, Jacobs JP, Jacobs ML, Maruszewski B, Pizarro C, et al. The assessment of complexity in congenital cardiac surgery based on objective data. Cardiol Young 2008 Dec;18(Suppl 2):169-176.
 Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123(1):110-118. Review.
 Lacour-Gayet F, Clarke D, Jacobs J, Comas J, Daebritz S, Daenen W, et al. The Aristotle score: a complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004 Jun;25(6):911-924.
 Jacobs JP, Jacobs ML, Maruszewski B, Lacour-Gayet FG, Tchervenkov CI, Tobota Z, et al. Initial application in the EACTS and STS Congenital Heart Surgery Databases of an empirically derived methodology of complexity adjustment to evaluate surgical case mix and results. Eur J Cardiothorac Surg 2012;42(5):775-779. discussion 779-80.
 Jacobs JP, Jacobs ML, Lacour-Gayet FG, Jenkins KJ, Gauvreau K, Bacha E, et al. Stratification of complexity improves the utility and accuracy of outcomes analysis in a Multi-Institutional Congenital Heart Surgery Database: Application of the Risk Adjustment in Congenital Heart Surgery (RACHS-1) and Aristotle Systems in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database. Pediatr Cardiol 2009 Nov;30(8):1117-1130.
 Al-Radi OO, Harrell Jr FE, Caldarone CA, McCrindle BW, Jacobs JP, Williams MG, et al. Case complexity scores in congenital heart surgery: a comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg 2007 Apr;133(4):865-875.
 Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh AM, Borde DP, et al. Predicting mortality after congenital heart surgeries: evaluation of the Aristotle and Risk Adjustment in Congenital Heart Surgery-1 risk prediction scoring systems: a retrospective single center analysis of 1150 patients. Ann Card Anaesth 2014;17(4):266-270.
 O'Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro C, et al. An empirically based tool for analyzing mortality associated with congenital heart surgery. J Thorac Cardiovasc Surg 2009 Nov;138(5):1139-1153.
 Cavalcanti PE, Sá MP, Santos CA, Esmeraldo IM, Chaves ML, Lins RF, et al. Stratification of complexity in congenital heart surgery: comparative study of the Risk Adjustment for Congenital Heart Surgery (RACHS-1) method, Aristotle basic score and Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery (STS-EACTS) mortality score. Rev Bras Cir Cardiovasc 2015;30(2):148-158.
 Vasdev S, Chauhan S, Malik M, Talwar S, Velayoudham D, Kiran U. Congenital heart surgery outcome analysis: Indian experience. Asian Cardiovasc Thorac Ann 2013 Dec;21(6):675-682.
 DeCampli WM, Burke RP. Interinstitutional comparison of risk-adjusted mortality and length of stay in congenital heart surgery. Ann Thorac Surg 2009 Jul;88(1):151-156.
 Bojan M, Gerelli S, Gioanni S, Pouard P, Vouhé P. Comparative study of the Aristotle comprehensive complexity and the risk adjustment in congenital heart surgery scores. Ann Thorac Surg 2011 Sep;92(3):949-956.