Durable left ventricular assist device (LVAD) is a widely accepted therapy for medically refractory advanced heart failure and significantly improves survival and quality of life.1 Despite improvements over successive generations of LVAD technology, LVAD therapy remains saddled with a high burden of adverse events, including right heart failure (RHF). RHF as a complication after LVAD implantation affects 9–42% of LVAD recipients and is associated with significantly increased mortality and morbidity.2,3 The high burden of RHF has motivated considerable research, but its prediction, prevention, and treatment remain largely unsolved.
The development of RHF after LVAD is likely polymechanistic and via various anatomic, mechanical, physiologic, and inflammatory pathways.3,4 It is clear that preimplant right ventricular function alone is only very modestly associated with postimplant RHF.5 In a host as complicated as an advanced heart failure patient, a myriad of preimplant, intraoperative, and postoperative factors potentially interact together and contribute to the development of RHF. Previous research has identified a number of RHF risk factors such as inotrope dependency, right atrial pressure (RAP), RAP/pulmonary capillary wedge pressure (PCWP) ratio, right ventricular ejection fraction, and renal impairment.5–9 However, a number of problems remain. First, most previous studies focused on a few risk factors at a time chosen by expert judgement. As a result, important but unknown risk factors may have been missed and the relative importance of the risk factors is unknown. Second, as in most complex biological systems, RHF risk factors likely do not increase risk in a linear fashion and there might be interactions among risk factors. These nonlinear relationships and interactions have not been previously identified. Finally and perhaps most importantly, the mere identification of risk factors is not enough to formulate actionable strategies to optimize patients before LVAD implantation to minimize the risk of RHF postimplant. Modifiable risk factors with realistic and impactful targets are necessary for this purpose. To address these shortcomings, we applied novel explainable machine learning (ML) methods to analyze risk factors of RHF after LVAD implantation.
ML methods have some important advantages over classical statistical methods in analyzing complex, high-dimensional problems. In the Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS) registry, for example, there are hundreds of preimplant patient factors that could theoretically be associated with RHF.1 Commonly used multivariate analytical methods such as linear, logistic, and Cox regression have difficulty handling this large number of explanatory variables, do not by default accommodate nonlinear relationships, nor model interactions without manual prespecification, which is impractical for hundreds of variables.10 ML, on the other hand, offers a number of algorithms that are far more flexible and are well equipped for high dimensional, nonlinear, and interacting relationships.11 Traditionally, a major disadvantage of most ML models for medical use is that they are “black boxes” that offer no explanation, and thus no insight or transparency. However, in recent years, novel “explainable ML” methods have been developed to offer rich interpretations of ML models, and they have proven useful in medical research.12–14 In this work, we aimed to leverage explainable ML to derive rich insights into risk factors for RHF after LVAD implantation and inform potential patient optimization strategies to minimize the risk of RHF postimplant.
Methods
Overview
Patients in the INTERMACS registry who were implanted with a continuous-flow durable LVAD were included. Preimplant patient factors including demographic, medical history, hemodynamic, laboratory, treatment, and other variables were entered into a gradient-boosted decision tree ML algorithm (LightGBM). The outcome of interest was 30 days of severe RHF. After derivation of the LightGBM model, an explainable ML method, Shapley additive explanations (SHAP), was applied to determine the most important variables for the development of RHF, the shape of relationships of these variables and their interactions, and risk inflection points of modifiable variables.
Study Population, and Definition of RHF
INTERMACS is a multicenter North American registry of patients with FDA-approved mechanical circulatory support devices initiated in 2005. We acquired INTERMACS data from the NHLBI BioLINCC (National Heart, Lung, and Brain Institute Biologic Specimen and Data Repository Information Coordinating Center), which contained INTERMACS data through December 2017. For this study, we included only patients who received their first durable continuous-flow LVAD (excluding LVAD exchanges) with no other concurrent device implantation. The primary outcome was 30 days of severe RHF defined as a composite of death from RHF, right ventricular assist device implantation, or use of inotropes for more than 14 days.
Data Processing
As we aimed to perform a comprehensive, unbiased analysis of preimplant factors, we included almost all available preimplant variables in INTERMACS, including demographic, medical/surgical history, hemodynamic, laboratory, recent adverse event, and other variables. The small number of variables excluded were redundant variables, variables that had no values, variables that had the same value for almost all patients, and a few that applied to extremely rare cases (such as HIV medications). As gradient-boosted decision tree algorithms natively handle missing values well, we did not exclude missing data or perform imputation. Finally, a number of additional, potentially relevant variables were derived based on existing variables, including pulmonary artery pulsatility index (PAPI), RAP/PCWP ratio, Model for End-stage Liver Disease (MELD) score, estimated glomerular filtration rate, furosemide-equivalent dose, body mass index, blood urea nitrogen/Creatinine ratio, mean arterial pressure, mean pulmonary artery pressure, pulmonary artery pulse pressure, number of inotropes, and number of preimplant hospitalization adverse events.
Model Training and Validation
A gradient-boosted decision tree algorithm was chosen because its many advantages are particularly suitable for this study. It is a mature ML algorithm that has high predictive accuracy, does not make linear relationship assumption, models interactions well, and has good native algorithm that handles missing values well.12,15 Gradient-boosted decision trees have been widely and successfully used in cardiovascular research in recent years. Of the gradient-boosted decision tree algorithms, LightGBM was chosen for its excellent accuracy, fast training speed, and ability to handle large amount of data.15
The dataset was split into an 80% training set and a 20% held-out validation set using random stratified selection. The training set was used to derive the model and threefold cross-validation was used to fine turn model hyperparameters to avoid over-fitting. The final, derived model was then validated on the held-out validation set using c-statistic and calibration curve.
Model Explanation and Variable Analysis
After the model was developed, variable importance was quantified by calculating the mean absolute Shapley value. The Shapley value is a game theory concept developed by economist Professor Lloyd Shapley in 1951, for which he was awarded the 2012 Nobel Prize for Economics. It is a value that assigns the relative contribution of each “player” who works in cooperation with other players to achieve an overall gain. The SHAP method adapts the Shapley theory to ML models by calculating Shapley values for each variable, and it is beginning to be used in the medical field.16–18
Given that some variables may exhibit colinearity that could artificially blunt the importance of these variables when calculated together, we used recursive feature elimination method to avoid this issue by eliminating the least important variable one by one. At each elimination step, we plotted the number of remaining variables against model accuracy to derive a short list of top variables that had the most impact on RHF. For these top variables, we plotted them on a summary plot showing their associations with the risk of RHF.
To visualize nonlinear relationships, we constructed partial dependence plots of each of the top variables. To identify interaction effects, we calculated the SHAP interaction values for all pair of variables and ranked them by mean absolute interaction value. For the top pairs, we plotted partial dependence plots colored by interaction values.
Results
Baseline Characteristics
A total of 19,595 patients who received an LVAD met inclusion criteria and were included. The baseline characteristics of the patients are listed in Table 1. Approximately half of the patients had ischemic cardiomyopathy. Most patients were of INTERMACS profiles 2 or 3 at the time of LVAD implantation. Approximately half of the patients were undergoing LVAD as a potential bridge to transplant. Of the included patients, 19.1% developed severe RHF within 30 days, similar to prior studies.
Table 1. -
Patient Baseline Characteristics
| Variable |
No RHF (n = 15862; 81%) |
RHF (n = 3733; 19%) |
Missing (n) |
| Age (years) |
56.9 (12.9) |
56.9 (12.9) |
2 |
| Sex (male) |
79.3 |
76.5 |
21 |
| Race |
| American Indian |
0.7 |
0.5 |
441 |
| Asian |
1.6 |
1.4 |
| African American |
22.8 |
29.7 |
| Pacific Islander |
0.4 |
0.2 |
| White |
68.6 |
60.1 |
| Height (cm) |
175.2 (9.8) |
174.3 (9.9) |
117 |
| Weight (kg) |
87.8 (22.2) |
87.8 (23.6) |
45 |
| BMI (kg/m2) |
28.5 (6.7) |
28.8 (7.1) |
138 |
| Primary cardiomyopathy diagnosis |
| Dilated NICM |
50.0 |
52.4 |
0 |
| Ischemic cardiomyopathy |
46.0 |
42.9 |
| Other cardiomyopathy |
3.3 |
4.0 |
| INTERMACS profile |
| 1 |
13.6 |
22.4 |
0 |
| 2 |
35.0 |
41.6 |
| 3 |
33.7 |
26.5 |
| 4 |
13.9 |
7.7 |
| 5 |
2.4 |
1.2 |
| 6 |
0.9 |
0.3 |
| 7 |
0.4 |
0.3 |
| Bridge to transplant |
50.0 |
50.0 |
7 |
| Events during hospitalization |
| Dialysis |
2.0 |
4.3 |
0 |
| Intubation |
9.0 |
13.5 |
| Major myocardial infarction |
2.9 |
3.3 |
| Major infection |
5.2 |
7.8 |
Values represent mean (standard deviation) for continuous variables and percentages for categorical variables.
BMI, body mass index; INTERMACS, Interagency Registry for Mechanically Assisted Circulatory Support; NICM, nonischemic cardiomyopathy.
Model Performance
As specified in Methods, a total of 186 preimplant patient variables were identified in INTERMACS and entered into the LightGBM model. The model achieved a c-statistic of 0.81 on the derivation cohort and 0.67 on the held-out validation cohort. The model showed excellent calibration (Figure S1, Supplemental Digital Content 1, https://links.lww.com/ASAIO/A885). Recursive feature elimination using SHAP values showed stable model performance until approximately 30 variables were left, meaning that these variables accounted for most of the model’s predictive power and were most associated with the development of acute severe RHF (Figure S2, Supplemental Digital Content 1, https://links.lww.com/ASAIO/A885).
Most Important Variables
The above-mentioned top 30 variables, along with the association of their values with the log of odds of RHF, are summarized in Figure 1. In this figure, variables are listed from top to bottom-ranked by their importance, or in other words their contribution to RHF. The five most important variables are INTERMACS profile, MELD score, the number of inotropic infusions before implant, hemoglobin, and race. For each variable in this figure, the variable value, illustrated by color (red: high; blue: low), is plotted against the odds of RHF on the x-axis (to the right: more likely to develop RHF; to the left: less likely to develop RHF). Each dot in the figure represents one patient. For example, for INTERMACS profile, numerically high values (less sick patients) indicated by red dots are on the left of the zero line, indicating less risk of RHF. High MELD scores, on the other hand, are associated with elevated risk of RHF.
;)
Figure 1.: Summary plot of the 30 most important variables associated with RHF after LVAD implantation. Variables ranked from top to bottom by importance in the model. For each variable, each patient is represented by a single point. The color of the point represents the variable value which is plotted against the log of odds of RHF on the x-axis. AST, aspartate aminotransferase; BMI, body mass index; BUN, blood urea nitrogen; CKD, chronic kidney disease; GFR, glomerular filtration rate; LDH, lactate dehydrogenase; LVED, left ventricular end-diastolic diameter; MELD, Model for End-stage Liver Disease; Na, sodium; PAPI, pulmonary artery pressure index; PCWP, pulmonary capillary wedge pressure; PDE-5, phosphodiesterase-5; RAP, right atrial pressure; RVEF, right ventricular ejection fraction; WBC, white blood cells.
Nonlinear Relationships and Risk Inflection Points
Most of the top 30 variables exhibited nonlinear associations with the risk of RHF. The partial dependence plots of four representative and potentially modifiable variables are displayed in Figure 2. Different patterns of nonlinearity exist. For example, for INTERMACS profile (Figure 2A), profiles 1 and 2 are associated with similarly increased risk of RHF, whereas profiles 3–7 are associated with similarly decreased risk of RHF. So the separation of risk is principally between INTERMACS profiles 1–2 and 3–7. For RAP (Figure 2B), the risk of RHF increases as RAP increases with a sharp step-up in risk once RAP exceeds 15 mmHg. For PAPI (Figure 2C), the risk of RHF is low until PAPI decreases below 5, and there is a risk inflection point around 3, below which the risk of RHF increases exponentially. Finally, for prealbumin (Figure 2D), the risk of RHF is similarly low for prealbumin greater than or equal to 23 mg/dl and similarly high for prealbumin less than 23 mg/dl. Partial dependence plots of the rest of the variables are included in the supplemental material (Figure S3, Supplemental Digital Content 1, https://links.lww.com/ASAIO/A885).
Figure 2.: Nonlinear relationships between variables and risk of RHF. Panel A: INTERMACS profile; B: RAP; C: RAPI; D: Prealbumin. The log of odds of RHF is plotted against variable value. Each point in the figure represents a single patient. The arrows depict the risk inflection points. INTERMACS, Interagency Registry for Mechanically Assisted Circulatory Support; PAPI, pulmonary artery pulsatility index; RAP, right atrial pressure.
Variable Interactions
Not surprisingly, many variables were found to interact with each other in their associations with the risk of RHF after LVAD implantation. The top 15 pairs of interacting variables are listed in Table 2. A large majority of the top interactions (9 out of 15) involved INTERMACS profile or the number of inotropes before LVAD implantation, suggesting that the acuity of patients strongly influences how other variables affect RHF risk. For example, in patients with more severe INTERMACS profiles, MELD score was a more important risk factor of RHF than in patients with less severe INTERMAC profiles. In other words, severe INTERMACS profiles (1 and 2) accentuated the effect of MELD on RHF (Figure 3A). Similarly, higher number of inotropes accentuated the effect of RAP on RHF (Figure 3B).
Table 2. -
Top Variable Interactions
| Rank |
Variable 1 |
Variable 2 |
| 1 |
INTERMACS profile |
GFR |
| 2 |
INTERMACS profile |
Race |
| 3 |
INTERMACS profile |
MELD |
| 4 |
Number of inotropes |
RAP |
| 5 |
Number of inotropes |
MELD |
| 6 |
Hemoglobin |
Race |
| 7 |
INTERMACS profile |
Number of hospitalization events |
| 8 |
INTERMACS profile |
Hemoglobin |
| 9 |
Lymphocyte percentage |
Prealbumin |
| 10 |
BUN |
GFR |
| 11 |
Number of inotropes |
Albumin |
| 12 |
Height |
Furosemide-equivalent dose |
| 13 |
INTERMACS profile |
AST |
| 14 |
GFR |
Prealbumin |
| 15 |
Race |
WBC |
AST, aspartate aminotransferase; BUN, blood urea nitrogen; GFR, glomerular filtration rate; INTERMACS, Interagency Registry for Mechanically Assisted Circulatory Support; MELD, Model for End-stage Liver Disease; RAP, right atrial pressure; WBC, white blood cells.
Figure 3.: Variable interactions. Panel A: MELD-INTERMACS profile interaction; B: RAP-Number of inotropes interaction. The log of odds of RHF is plotted against variable value. Interaction with the second variable is depicted by showing the second variable on a color scale. Each point in the figure represents a single patient. MELD, Model for End-stage Liver Disease; INTERMACS, Interagency Registry for Mechanically Assisted Circulatory Support; RAP, right atrial pressure.
Discussion
In this study, we performed an explainable ML analysis of preimplant patient factors and their association with acute severe RHF in patients implanted with LVADs in the INTERMACS registry. Out of 186 preimplant patient factors, we identified 30 most important variables and their relationships with the risk of RHF, including key risk inflection points of potentially modifiable variables. To the best of our knowledge, this study is the first comprehensive, unbiased analysis of this topic, using all relevant preimplant variables in a large multicenter registry. Our results also showcase the promise of explainable ML methods as a new avenue for gaining insights into complex medical problems.
For the current problem at hand, RHF after LVAD implantation, previously identified risk factors, such as renal function, RAP, RAP/PCWP ratio, PAPI, and lymphocyte percentage, were also shown among the top risk factors in our study. We additionally identified some previously under-recognized variables such as race, height, prealbumin, cholesterol, serum sodium, platelet, and the number of preimplant adverse events. A notable finding is that race is among the top five most important variables, with White race being protective and African American race being a risk factor for acute RHF after LVAD. While race has been repeatedly shown to be associated with outcomes in heart failure and heart transplant, its association with acute RHF after LVAD has not been shown before.19–21 Race’s association with this early inpatient adverse event is particularly intriguing and deserves further investigation.
The nonlinear relationships, variable interactions, and risk inflection points shown in our study have clear clinical implications. Most biological systems are inherently nonlinear, and our results offer clinicians a more realistic and nuanced understanding of RHF risk factors than traditional linear representations of risk. A better understanding may help with LVAD patient selection, but more importantly, our results could potentially improve the way we optimize patients before LVAD implantation. Currently, patient optimization before LVAD is an important but sometimes nebulous concept. Merely knowing risk factors for an adverse outcome does not automatically translate to a testable strategy to prevent such outcome. For example, RAP is a recognized risk factor for RHF after LVAD, yet driving down the RAP to zero is neither realistic nor safe in most cases. Our results, on the other hand, offer realistic targets and goals for intervention based on modifiable risk factors’ risk inflection points. Using the same example as above, RAP has a risk inflection point around 15 mmHg, which is a plausible and clinically testable goal. A patient optimization strategy can then be developed by combining several modifiable risk factors in a similar fashion, further fine-tuned based on important interactions such as INTERMACS profiles. Proposed patient optimization strategies can then be simulated using our model to estimate their impact and suitability for clinical testing.
Our study is subject to some limitations. First, we acknowledge that, as is the case for any observational study, our ML analysis reveals associations and not causations. Second, we do not suggest that RHF after LVAD implantation is only dependent on preimplant patient factors. In fact, operative factors and postoperative patient management likely have a large impact on the development of RHF as well. We focused on preimplant factors in this study because they are measurable and potentially intervenable before LVAD implantation. Third, we did not perform separate analyses for axial versus centrifugal devices and it is possible that there are some differences in the RHF risk factors for these two types of devices.
In conclusion, an explainable ML method revealed top patient factors associated with RHF after LVAD implantation, their risk inflection points and interactions. These insights could aid the development of patient optimization strategies before LVAD implantation.
References
1. Kirklin JK, Pagani FD, Kormos RL, et al.: Eighth annual INTERMACS report: Special focus on framing the impact of adverse events. J Hear Lung Transplant 36: 1080–1086, 2017.
2. LaRue SJ, Raymer DS, Pierce BR, Nassif ME, Sparrow CT, Vader JM: Clinical outcomes associated with INTERMACS-defined
right heart failure after left ventricular assist device implantation. J Hear Lung Transplant 36: 475–477, 2017.
3. Lampert BC, Teuteberg JJ: Right ventricular failure after left ventricular assist devices. J Hear Lung Transplant 34: 1123–1130, 2015.
4. Pienta MJ, Noly P-E, Janda AM, et al.: Rescuing the right ventricle: A conceptual framework to target new interventions for patients receiving a durable left ventricular assist device therapy [Online ahead of print]. J Thorac Cardiovasc Surg, 2022. doi: 10.1016/j.jtcvs.2022.03.025.
5. Frankfurter C, Molinero M, Vishram-Nielsen JKK, et al.: Predicting the risk of right ventricular failure in patients undergoing left ventricular assist device implantation. Circulation Hear Fail 13: CIRCHEARTFAILURE120006994, 2020.
6. Kang G, Ha R, Banerjee D: Pulmonary artery pulsatility index predicts right ventricular failure after left ventricular assist device implantation. J Hear Lung Transplant 35: 67–73, 2016.
7. Bellavia D, Iacovoni A, Scardulla C, et al.: Prediction of right ventricular failure after ventricular assist device implant: Systematic review and meta‐analysis of observational studies. Eur J Heart Fail 19: 926–946, 2017.
8. Gulati G, Grandin EW, Kennedy K, et al.: Preimplant phosphodiesterase-5 inhibitor use is associated with higher rates of severe early
right heart failure after left ventricular assist device implantation. Circulation Hear Fail 12: e005537, 2019.
9. Yost GL, Joseph CR, Tatooles AJ, Bhat G: Neutrophil-to-lymphocyte ratio predicts outcomes in patients implanted with left ventricular assist devices. ASAIO J 61: 664–669, 2015.
10. LaValley MP: Logistic regression. Circulation 117: 2395–2399, 2008.
11. Li S, Hickey GW, Lander MM, Kanwar MK: Artificial intelligence and mechanical circulatory support. Heart Fail Clin 18: 301–309, 2022.
12. Lundberg SM, Erion G, Chen H, et al.: From local explanations to global understanding with
explainable AI for trees. Nat Mach Intell 2: 56–67, 2020.
13. Lundberg SM, Nair B, Vavilala MS, et al.:
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng 2: 749–760, 2018.
14. Lundberg SM, Lee S-I: A unified approach to interpreting model predictions. in Advances in Neural Information Processing Systems, Vol 30. Red Hook, NY, USA, Curran Associates, Inc., 2017, pp. 4768–4777.
15. Ke G, Meng Q, Finley T, et al.: LightGBM: A highly efficient gradient boosting decision tree. in Advances in Neural Information Processing Systems, Vol 30. Red Hook, NY, USA, Curran Associates, Inc., 2017, pp. 3149–3157.
16. Tseng P-Y, Chen Y-T, Wang C-H, et al.: Prediction of the development of acute kidney injury following cardiac surgery by
machine learning. Crit Care 24: 478, 2020.
17. Chen T, Xu J, Ying H, et al.: Prediction of extubation failure for intensive care unit patients using light gradient boosting machine. IEEE Access 7: 150960–150968, 2019.
18. Zein JG, Wu C-P, Attaway AH, Zhang P, Nazha A: Novel
machine learning can predict acute asthma exacerbation. Chest 159: 1747–1757, 2021.
19. Morris AA, Kransdorf EP, Coleman BL, Colvin M: Racial and ethnic disparities in outcomes after heart transplantation: A systematic review of contributing factors and future directions to close the outcomes gap. J Hear Lung Transplant 35: 953–961, 2016.
20. Kilic A, Higgins RSD, Whitson BA, Kilic A: Racial disparities in outcomes of adult heart transplantation. Circulation 131: 882–889, 2015.
21. Okoh AK, Selevanny M, Singh S, et al.: Racial disparities and outcomes of left ventricular assist device implantation as a bridge to heart transplantation. Esc Hear Fail 7: 2744–2751, 2020.
