Pump Type and Strategy
At 90 days, the score predicted mortality similarly for axial (AUC 61%) and centrifugal pumps (AUC 62%) (Figure 2A). For 1 year mortality, the prediction for centrifugal pumps (AUC 61%) was slightly better than axial (AUC 59%) pumps. The prediction performance at 90 days was comparable for BTT (AUC 61%) versus DT (AUC 60%) indications and was slightly lower at 1 year with AUC for BTT and DT at 59% and 58%, respectively.
Multiple Regression Analysis of HMRS Variables on INTERMACS Data
When we entered the variables used for multiple regression analysis for calculating HMRS into a similar analysis on INTERMACS variables, a significantly different list of variables emerged as being statistically significant (Table 2). For example, both INR and center volume were very significant (p < 0.001) in HMRS data but not very significant (p = 0.94 and 0.45, respectively) in the INTERMACS cohort. Alternately, the INTERMACS dataset identified five additional significant variables from the same list that were significantly predictive of 90 day mortality (gender, LVAD indication, preoperative ventilator or intraaortic balloon pump, platelet count, and total bilirubin, all p < 0.05).
Our data clearly demonstrates that the HMRS performance is poor at best when applied to a contemporary, comprehensive, multicenter LVAD database. Originally derived from analyzing outcomes in patients enrolled in the HeartMate clinical trial, HMRS initially gained traction for clinical use given the relative ease of obtaining its five composite variables. However, it should be noted that several potential assumptions were made during its production, which explain its limited applicability that was observed in this study.
Relation Between Variables and Outcomes
HeartMate II risk score was derived from multivariate analysis, and thus assumes a linear and independent relationship between each of the clinical variables and mortality. In other words, it is a weighted sum of individual variables reflecting the cumulative effect on the outcome, and fails to take into account the interdependency among the individual variables and the relationship to mortality. For example, in clinical practice, it is well recognized that preoperative right ventricular dysfunction is associated with a higher mortality risk after LVAD implantation. This risk association is not only based on derangement of hemodynamic variables, but also from the resulting liver congestion and renal dysfunction, all of which are interrelated, and potentially synergistic. However, if right ventricular flow is optimized preoperatively by measures such as diuretic, inotropic, or pulmonary vasodilator therapy, it may result in improvement in measures of liver and renal function, which in turn will improve the risk score, without necessarily affecting the patient’s risk for an adverse post-LVAD outcome. These dynamics play a considerable role in clinical/bed-side decision-making but cannot be captured by a traditional risk score that assume linear relationships between variables and outcomes.
Number of Variables in HMRS
The limited number of variables that go into calculating HMRS fail to capture the complex, heterogeneous risk profile of patients with end-stage HF. For example, INR is one of the five predictive variables in HMRS, with high levels reflective of liver dysfunction and contributing to a high-risk categorization. In routine practice, end-stage HF patients may have normal liver function but have a high INR by virtue of being on anticoagulation. Similarly, the absolute value of creatinine used to define HMRS may be low (suggesting low-risk category), but the patient may be on dialysis (a high-risk feature). Other confounding factors can be attributed to the derivation from clinical trial data and the limited number of variables considered, which fails to account for factors such as nutrition, frailty, socioeconomic dynamics, and psychosocial parameters.
To provide a real-life example using a representative case studies, patient A, a 60 year old male who is INTERMACS level 1, with New York Heart Association class IV symptoms, in acute renal failure with a creatinine level of 2.0 mg/dl, on a ventilator and intraaortic balloon pump would be given a low risk (8% 90 day mortality) by HMRS categorization. Similarly, patient B, a frail 70 year old female with chronic kidney disease (creatinine of 3.3 mg/dl), INTERMACS level 3, with New York Heart Association class IV symptoms would have a medium risk of mortality (11% at 90 days). In clinical practice, both these patients would be considered fairly high risk for an LVAD implantation because of their multiple medical comorbidities that were not captured by HMRS. It is also to be noted that the predictors of 1 year survival when conditioned to a 90 day survival in HMRS calculator were limited to only two variables: age and implant center experience. This is one of the main reasons why clinical use of HMRS in routine practice is limited in application.
We must remember that HMRS was calculated from the HeartMate II clinical trial data, which had specific inclusion and exclusion criteria. The clinical and surgical experience across institutions implanting VADs has evolved and improved since then. When we applied the variables that were entered into a multivariate risk model to calculate HMRS in the much larger patient cohort in INTERMACS, we found different set of variables to be predictive of 90 day mortality. The main reason for this observed difference is that INTERMACS is a much larger, heterogeneous, and comprehensive database with differences in baseline patient characteristics from the HeartMate II clinical trial, and represents the “real life” advanced HF population encountered in clinical practices in the United States. This highlights the fact that risk scores, when derived from specific data sets, may not be applicable to a wider patient population encountered in clinical practice.
Application to Single Centers
The lack of robustness of these scores is corroborated by other single-center validation studies. Thomas et al.9 applied the HMRS to 205 patients who received a HeartMate II pump at their center to report a poor prediction of 90 day mortality by ROC (AUC = 0.56). They concluded that the score’s generalizability as a universal prognostic score may be limited. Another single-center study retrospectively validated the score’s validity to predict short-term mortality in the critically sick INTERMACS class 1 patients.10 However, for a score to be of value to clinicians, it needs to not only predict poor outcomes in critical patients but also be applicable across the heterogeneous HF population at different stages of their disease. In addition, the score’s applicability becomes even more limited to single-center analysis given that one of the five variables (center volume) would remain constant when applied to an individual center.
It could be argued that the HRMS discrimination for predicting mortality at 90 day with an AUC of 0.61, although considerably lower than noted in the original study, is still modest. However, given that an LVAD implantation is a high-risk, long-term, and expensive intervention associated with significant morbidity and mortality,11 a truly predictive risk score needs to be better than ‘modest’ and complement human decision making.
Recently, our group has reported the use of Bayesian methods of analysis to overcome some of these limitations in predicting post-LVAD implant mortality.12 The Bayesian method clearly outperforms conventional risk scores in predictive accuracy, and we believe that this method could allow clinicians to reliably select patients who are likely to achieve “optimal” clinical outcomes with LVAD implantation. This undoubtedly will go a long way toward cost-effectively using this technology in the management of advanced HF.
We acknowledge several limitations of this study, including those that apply to the premise of a retrospective cohort analysis. Age and center experience were calculated as a median from the range of options provided to us. However, our study used the most comprehensive and robust registry of LVAD recipients that is currently available.
The decision to implant an LVAD in an ambulatory patient with advanced HF may be a daunting task for both patients and physicians for multiple reasons. Prognostic risk models have the potential to enhance a clinician’s ability to appropriately select patients for this advanced HF therapy and reduce the likelihood of adverse events. Yet, information gleaned from presently available risk scores models such as HMRS is inconsistent and may be of limited utility for day-to-day clinical practice. There is a critical need for accurate, flexible, and improved predictive model to account for the heterogeneity of end-stage HF patients.
The authors thank the Data Access, Analysis, and Publications Committee of INTERMACS for allowing us to use their registry for the study, and we specifically thank Dr. James Kirklin, Dr. Francis Pagani, and Dr. David Naftel. The authors also thank Susan Meyers and Grant Studdard for administrative, database, and statistical assistance with INTERMACS.
1. Kirklin JK, Naftel DC, Pagani FD, et al. Seventh INTERMACS
annual report: 15,000 patients and counting. J Heart Lung Transplant 2015.34: 1495–1504.
2. Miller LW, Guglin M. Patient selection for ventricular assist devices: A moving target. J Am Coll Cardiol 2013.61: 1209–1221.
3. Feldman D, Pamboukian SV, Teuteberg JJ, et al; International Society for Heart and Lung Transplantation: The 2013 International Society for Heart and Lung Transplantation Guidelines for mechanical circulatory support: executive summary. J Heart Lung Transplant 2013.32: 157–187.
4. Slaughter MS, Pagani FD, Rogers JG, et al; HeartMate II Clinical Investigators: Clinical management of continuous-flow left ventricular assist devices in advanced heart failure. J Heart Lung Transplant 2010.29 (suppl 4): S1–39.
5. Lietz K, Long JW, Kfoury AG, et al. Outcomes of left ventricular assist device implantation as destination therapy in the post-REMATCH era: Implications for patient selection. Circulation 2007.116: 497–505.
6. Imamura T, Kinugawa K, Shiga T, et al. Novel risk scoring system with preoperative objective parameters gives a good prediction of 1-year mortality in patients with a left ventricular assist device. Circ J 2012.76: 1895–1903.
7. Boyle AJ, Ascheim DD, Russo MJ, et al. Clinical outcomes for continuous-flow left ventricular assist device patients stratified by pre-operative INTERMACS
classification. J Heart Lung Transplant 2011.30: 402–407.
8. Cowger J, Sundareswaran K, Rogers JG, et al. Predicting survival in patients receiving continuous flow left ventricular assist devices: The HeartMate II risk score
. J Am Coll Cardiol 2013.61: 313–321.
9. Thomas SS, Nahumi N, Han J, et al. Pre-operative mortality risk assessment in patients with continuous-flow left ventricular assist devices: Application of the HeartMate II risk score
. J Heart Lung Transplant 2014.33: 675–681.
10. Adamo L, Nassif M, Tibrewala A, et al. The HeartMate Risk Score
predicts morbidity and mortality in unselected left ventricular assist device recipients and risk stratifies INTERMACS
class 1 patients. JACC Heart Fail 2015.3: 283–290.
11. Long EF, Swain GW, Mangi AA. Comparative survival and cost-effectiveness of advanced therapies for end-stage heart failure. Circ Heart Fail 2014.7: 470–478.
12. Loghmanpour NA, Kanwar MK, Druzdzel MJ, Benza RL, Murali S, Antaki JF. A new Bayesian network-based risk stratification model for prediction of short-term and long-term LVAD mortality. ASAIO J 2015.61: 313–323.
Keywords:Copyright © 2017 by the American Society for Artificial Internal Organs
risk score; INTERMACS; HMRS