Secondary Logo

Journal Logo

Original Articles

Individual mortality risk predictive system of patients with acute-on-chronic liver failure based on a random survival forest model

Zhang, Zhi-Qiao1; He, Gang2; Luo, Zhao-Wen3; Cheng, Can-Chang3; Wang, Peng1; Li, Jing1; Zhu, Ming-Gu3; Ming, Lang1; He, Ting-Shan1; Ouyang, Yan-Ling1; Huang, Yi-Yan1; Wu, Xing-Liu2; Ye, Yi-Nong4

Editor(s): Lyu, Peng

Author Information
doi: 10.1097/CM9.0000000000001539

Abstract

Introduction

Chronic hepatitis B is one of the most prevalent threats to liver health in the world.[1] Acute-on-chronic liver failure (ACLF) is the acute decompensation of liver function based on chronic liver diseases under the actions of different liver attack events.[2] Due to poor basic liver function and multiple organ failure, 60% to 70% of ACLF patients experience rapid aggravation and die within 3 months.[2,3] There is an urgent requirement for a prognostic model to identify ACLF patients with a high mortality risk, who are in urgent need of liver transplantation in the short term. A few prognostic models could provide mortality risk prediction information for ACLF patients.[4–6] However, these prognostic models could only provide predicted mortality for a special group of patients with similar clinical characteristics at the group level,[7,8] but failed to predict individual mortality risk predictive information for individual patients at the individual level.

The random survival forest (RSF) algorithm is a nonparametric algorithm with great clinical application value that has been recommended for prognostic prediction.[9,10] The RSF algorithm can avoid the influence of multicollinearity and can provide objective evaluations of the interactions between different variables.[11] The RSF algorithm can automatically calculate and order the importance of different variables.[12,13] In addition, the RSF method can address the impact of noise caused by missing or incorrect values.[14] The RSF method has been used to develop prognostic models for different diseases.[15–17] As an effective survival analysis method taking nonlinearity into account, the RSF model was superior to the Cox proportional model for prognostic prediction.[18] Recently, several studies developed online mortality risk predictive tools for different tumors, providing individual mortality risk predictive curves at the individual level.[19–21] For clinicians and patients, individual mortality risk predictive curves at the individual level can provide more valuable reference information for individualized treatment decisions.

Therefore, the present study aimed to develop an online individual mortality risk predictive tool for ACLF patients based on an RSF algorithm, which could predict the individual mortality risk predictive curve at the individual level.

Methods

Study population

The current study retrospectively enrolled ACLF patients from the Department of Infectious Diseases of The First People's Hospital of Foshan, Shunde Hospital of Southern Medical University, and Jiangmen Central Hospital (n = 391). The last follow-up time of the enrolled patients was September 10, 2018. Inclusion criteria: 1. ACLF was diagnosed according to the guidelines of the Asian Pacific Association for the Study of the Liver; 2. Hepatitis B surface antigen (HBsAg) positivity for >6 months or with a clear history of chronic hepatitis B; and 3. Adequate survival information. Exclusion criteria: 1. Other hepatitis viruses (hepatitis A, hepatitis C, hepatitis E, and hepatitis D); 2. Liver cancer or other malignant tumors; 3. Autoimmune liver disease; 4. Liver failure caused by alcoholic liver disease, drug-induced hepatitis, hyperthyroidism, poisoning, and other reasons; 5. Unstable period of cardio-cerebral infarction; 6. Accompanied with kidney diseases; 7. Pregnancy; 8. Patients with follow-up time <1 month after discharge were not included in the final survival analysis to eliminate the influence of confounding factors; and 9. Patients without critical baseline information (ie, hepatic encephalopathy (HE), age, serum sodium level, acute kidney injury (AKI), red cell distribution width [RDW], and international normalization index [INR]) were not included in the final survival analysis. Two hundred seventy-six ACLF patients were included in the final survival analysis as the model dataset. We performed the present research according to the Declaration of Helsinki. This study was approved by the Ethics Committee of the Shunde Hospital, Southern Medical University (No. 20171108). As a retrospective study and data analysis were performed anonymously, this study was exempt from the informed consent from patients. The current study eliminated all privacy information that could identify the individual information of patients to protect the privacy of the enrolled patients.

Diagnostic criteria and references

The diseases and complications were diagnosed according to the original studies: ACLF,[22] AKI,[23] hepatorenal syndrome (HRS),[24] HE,[25] pulmonary infection (PI),[26] and gastrointestinal hemorrhage (GH).[27]

Prognostic models

Three prognostic models were calculated according to the previous formula: model for end-stage liver disease (MELD) = 9.57 × loge (creatinine [mg/dL]) + 3.78 × loge (bilirubin [mg/dL]) + 11.2 × loge (INR) + 6.43 × (etiology: 0 for cholestatic or alcoholic; 1 for otherwise);[4] International normalized ratio and creatinine score (ABIC) = (age × 0.1) + (serum bilirubin (mg/dL) × 0.08) + (serum creatine (mg/dL) × 0.3) + (INR × 0.8);[6] Integrated MELD (iMELD) = MELD + [age (year) × 0.3] − 0.7 × Na (mmol/L)] + 100[5]

Validation cohort based on the bootstrap resampling method

The bootstrap resampling technique is a statistical sampling method with replacement from the original cohort, which is suitable for internal validation of prognostic models.[28,29] Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis proposed the bootstrapping resampling method to be a prerequisite of prognostic model development in case the external original dataset was not available.[30] The current study constructed a validation cohort by drawing patients from the model dataset based on the resampling method (n = 276).

Statistical analysis

The statistical analyses were carried out by R software (version 3.6.1). Continuous variables are depicted as the mean ± standard deviation or median (first quartile, third quartile). Continuous variables were compared by t test or the Kruskal-Wallis H test. Categorical variables were compared by the chi-squared test or Fisher exact test. The RSF model in the current study was conducted with reference to the model method in several articles performed by other researchers. The RSF model is an ensemble tree-based algorithm for variable selection in high-dimensional datasets. RSF performs well in calculation efficiency and predictive performance with low generalization error. When there are complex interactions between covariate Zs, the RSF model is particularly suitable for variable selection.[9,10] The Brier score was used to evaluate the diagnostic accuracy of prognostic models according to the original studies.[31,32] The predictive accuracy of the model with a smaller Brier score is superior to that of the model with a higher Brier score.[31] The weighted mean rank estimation method was used to compare the differences between the areas under the time-dependent ROC curves (AUROCs).[33,34] A P value < 0.05 was defined as statistically significant.

Results

Study datasets

There were 276 ACLF patients in the model cohort. The validation cohort contained 276 ACLF patients who were drawn from the original model cohort by the bootstrapping resampling method. The comparisons of baseline characteristics of patients in the model and validation cohorts are summarized in Table 1.

Table 1 - Baseline characteristics of patients in the model and validation groups.
Model group Validation group Group difference
Variable N = 276 N = 276 Test value P-value
Overall survival (months) 19.3 (0.9, 42) 21.8 (1.1, 43.2) −1.022 0.307
Age (years) 43 (36, 53) 40 (35, 50) −1.412 0.158
Creatinine (μmol/L) 71 (60, 83) 72 (61, 83) −0.861 0.389
Uric acid (μmol/L) 201.4 (201.4, 201.4) 201.4 (197, 201.4) −0.281 0.779
Fasting plasma glucose (mmol/L) 5.8 (4.2, 6.8) 5.6 (4.2, 6.6) −0.712 0.477
Direct bilirubin (μmol/L) 188.5 (103.2, 280.4) 190.8 (105.4, 285.5) −0.516 0.606
Albumin (g/L) 30.8 ± 5.7 31.1 ± 5.3 −0.703 0.482
Globulin (g/L) 32.5 (27.7, 37.4) 33.2 (27.7, 38.7) −1.244 0.213
Alanine aminotransferase (U/L) 576 (136, 1319) 632 (167, 1304) −0.677 0.499
Glutamic oxaloacetic transaminase (U/L) 396 (140, 873) 449 (146, 934) −0.790 0.429
Glutamyl transferase (U/L) 111 (72, 147) 113 (77, 157) −0.889 0.374
Alpha fetoprotein (ng/mL) 74.5 (13.9, 123.4) 88.5 (17.6, 123.4) −0.574 0.566
Hyaluronidase (ng/mL) 990 (543.1,1000) 971.1 (288.8,1000) −1.483 0.138
Collage Type IV (ng/mL) 439.2 (247.2,633.6) 414.7 (206.5,527.3) −1.490 0.136
N-Terminal Procollagen III Propeptide (ng/mL) 25.5 (18.6,31.1) 27.7 (18,32.8) −0.326 0.745
Laminin (ng/mL) 135.4 (97,196.1) 124.4 (93.7,196.1) −0.274 0.784
Log10DNA (IU/mL) 5.7 (4.2, 7.3) 6.2 (4.5, 7.5) −0.920 0.358
White blood cell (109/L) 7.2 (5.5, 9.5) 7.2 (5.5, 9.2) −0.406 0.685
Neutrophil-to-lymphocyte ratio 3.8 (2.4, 5.5) 3.6 (2.3, 5.2) −0.879 0.380
Neutrophil_ratio 0.7 (0.6, 0.8) 0.7 (0.6, 0.7) −1.156 0.248
Hemoglobin (g/L) 126.9 (111, 140) 128 (111, 140.2) −0.507 0.612
Platelet distribution width (%) 16.4 (15.6, 16.6) 16.3 (15.5, 16.5) −0.804 0.421
Mean platelet volume (fl) 11.1 (10, 11.7) 11.1 (10, 11.7) −0.627 0.531
Platelets (109/L) 123.2 (83, 161) 129 (85.5, 164) −1.515 0.130
Red cell distribution width (fl) 42.4 (22.2, 48.2) 42.6 (36.9, 47.7) −0.846 0.397
Prothrombin time (sec) 22.4 (18.4, 28.8) 21.4 (18.2, 28) −0.655 0.512
Serum sodium level (mmol/L) 137 (135, 140.1) 138 (134.8, 140.5) −0.522 0.602
International normalization index 1.9 (1.5, 2.4) 1.8 (1.5, 2.2) −0.493 0.622
Fibrinogen (g/L) 1.4 (1, 1.7) 1.5 (1.1, 1.8) −1.037 0.30
Total cholesterol (mmol/L) 2.9 (2.5, 3.2) 2.9 (2.8, 3.2) −1.478 0.139
Triglyceride (mmol/L) 1.4 (1.1, 1.5) 1.4 (1.3, 1.5) −1.613 0.107
ABIC 7.7 (6.3, 8.8) 7.6 (6.3, 8.7) −0.429 0.668
Model for end-stage liver disease 21.8 (17.7, 25.7) 22.4 (19.2, 24.9) −0.802 0.423
Integrated MELD 35.461 (30.542, 40.243) 33.8 (30.05,38.275) −0.312 0.755
Death 117 (42.4) 111 (40.2) 0.187 0.666
Gender 236 (85.5) 247 (89.5) 1.656 0.198
Acute kidney injury 43 (15.6) 42 (15.2) 0.01 0.906
Pulmonary infection 50 (18.1) 53 (19.2) 0.048 0.827
Hepatic encephalopathy 72 (26.1) 65 (23.6) 0.35 0.554
Hepatorenal syndrome 46 (16.7) 35 (12.7) 1.447 0.229
Gastrointestinal bleeding 23 (8.3) 19 (6.9) 0.232 0.630
Continuous variables are expressed as the mean ± standard deviation or median (first quartile, third quartile) or n(%) as appropriate. Kruskal-Wallis H test. χ2 values. t values. ABIC: International normalized ratio and creatinine score; AKI: Acute kidney injury; HE: Hepatic encephalopathy; HRS: Hepatorenal syndrome; INR: International normalization index; MELD: Model for end-stage liver disease; PT: Prothrombin time; PI: Pulmonary infection; RSF: Random survival forest; RDW: Red cell distribution width.

Importance evaluation of variables

An RSF algorithm was carried out to present the importance evaluation chart of the study variables. The importance evaluation chart [Figure 1] indicated the importance of the top 30 variables. AKI, HRS, HE, age, RDW, prothrombin time (PT), triglyceride, gastrointestinal bleeding, INR, and platelet count were identified as prognostic factors by the RSF algorithm.

Figure 1
Figure 1:
Importance evaluation chart of variables by the RSF model. AKI: Acute kidney injury; ALT: Alanine aminotransferase; INR: International normalized ratio; NLR: Neutrophil-to-lymphocyte ratio; PDW: Platelet distribution width; PT: prothrombin time; RDW: Red cell distribution width; RSF: Random survival forest; WBC: White blood cell.

Cox proportional hazards regression model

To construct a simplified RSF model for clinical application, multivariate Cox regression (step forward method) was used to explore the most valuable variables for predicting the prognosis of ACLF patients. As shown in Table 2, HE, age, serum sodium level, AKI, RDW, and INR were identified as independent risk factors for ACLF patients. Figure 2 presents a prognostic nomogram for ACLF patients based on the Cox proportional hazards regression model.

Table 2 - Results of univariate Cox regression analysis and multivariate Cox regression analysis of the included variables.
Univariate analysis Multivariate analysis
Variable HR 95% CI P value Coefficient HR 95% CI P value
HE 4.318 2.990–6.237 <0.001 0.879 2.408 1.624–3.571 <0.001
Age 1.050 1.036–1.064 <0.001 0.035 1.035 1.021–1.050 <0.001
Serum sodium level 0.963 0.948–0.978 <0.001 −0.022 0.978 0.959–0.998 0.027
AKI 5.866 3.940–8.732 <0.001 1.191 3.289 2.170–4.985 <0.001
RDW 0.969 0.958–0.981 <0.001 −0.026 0.974 0.963–0.986 <0.001
INR 2.068 1.691–2.528 <0.001 0.640 1.897 1.530–2.350 <0.001
AKI: Acute renal injury; CI: Confidence interval; HE: Hepatic encephalopathy; INR: International normalized ratio; HR: Hazard ratio; RDW: Red cell distribution width.

Figure 2
Figure 2:
Mortality risk nomogram predictive chart. AKI: Acute kidney injury; INR: International normalization index; RDW: Red cell distribution width.

Simplified RSF model

A simplified RSF model was developed based on HE, age, serum sodium level, AKI, RDW, and INR. The diagnostic performance of the RSF model was validated through the out-of-band (OOB) method [Figure 3]. As shown in Figure 3A, the green line represents the Nelson-Aalen estimator survival curve, and the red line represents the overall ensemble survival curve. The overall ensemble survival curve was highly consistent with the Nelson-Aalen estimation survival curve, indicating that the estimated survival curve (green line) by the RSF model was in good agreement with the real survival curve (red line).

Figure 3
Figure 3:
Performance of the RSF model. OOB: Out-of-band; CRRS: Continuous ranked probability score; RSF: Random survival forest.

An online individual mortality risk predictive tool

The current study further developed an online individual mortality risk predictive tool based on the RSF algorithm for ACLF patients. As shown in Figure 4A, our online individual mortality risk predictive tool could predict individual mortality risk percentages at different time points based on the RSF algorithm. As the reference survival curve, the current online individual mortality risk predictive tool also provided the individual mortality risk predicted curves generated by the Cox regression algorithm [Figure 4B]. In addition, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at different time points [Figure 4C]. This online individual mortality risk predictive tool is available at https://zhangzhiqiao13.shinyapps.io/Individual_mortality_risk_predictive_tool_for_liver_failure/.

Figure 4
Figure 4:
Individual mortality risk predictive tool based on the RSF model: (A) Individual mortality risk predictive tool based on the RSF algorithm; (B) Individual mortality risk predictive tool based on the Cox regression algorithm; (C) Predicted mortality percentage and 95% confidence interval. AKI: Acute kidney injury; INR: International normalization index; RDW: Red cell distribution width; RSF: Random survival forest.

Performance of the RSF model in the model cohort

The AUROCs for predicting 3-, 6-, and 12-month mortality were 0.916, 0.916, and 0.905, respectively, for the RSF model in the model cohort [Figure 5A]. The mortality of patients with high RSF scores was significantly poorer than that of patients with low RSF scores [Figure 5B]. Calibration curves demonstrated that the predicted mortality was highly consistent with the actual mortality in the model cohort [Supplementary Digital Content, Figure 1, http://links.lww.com/CM9/A572].

Figure 5
Figure 5:
Performance of the prognostic model in the model group: (A) Time-dependent receiver operating characteristic curve chart; (B) Survival curve chart.

Internal validation of RSF model

In the validation cohort, the AUROCs for predicting 3-, 6-, and 12-month mortality were 0.912, 0.910, and 0.880, respectively, for the RSF model [Figure 6A]. Figure 6B indicates that the mortality of patients with high RSF scores was significantly poorer than that of patients with low RSF scores. Calibration curves demonstrated that the predicted mortality was consistent with the actual mortality in the validation cohort [Supplementary Digital Content, Figure 2, http://links.lww.com/CM9/A573].

Figure 6
Figure 6:
Performance of the prognostic model in the validation group: (A) Time-dependent receiver operating characteristic curve chart; (B) Survival curve chart.

Comparison of diagnostic accuracy

The AUROCs of the RSF model were superior to those of the Cox model for predicting 3-, 6-, and 12-month mortality [Table 3 and Supplementary Digital Content, Figure 3, http://links.lww.com/CM9/A574]. The nonparametric comparison suggested that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis at different time points.

Table 3 - Comparison of diagnostic accuracy in the model group.
AUROC Brier score
Models 3 months 6 months 12 months 3 months 6 months 12 months
RSF model 0.916 0.916 0.905 0.119 0.119 0.128
COX model 0.872 0.866 0.848 0.138 0.146 0.156
MELD 0.683 0.657 0.660 0.206 0.218 0.221
iMELD 0.782 0.768 0.763 0.183 0.191 0.196
ABIC 0.771 0.762 0.763 0.186 0.193 0.197
ABIC: International normalized ratio and creatinine score; AUROC: area under the time-dependent ROC curves; iMELD: Integrated MELD; MELD: Model for end-stage liver disease; MELD-Na: MELD-sodium; MESO: MELD and serum sodium ratio; RSF: Random survival forest.

Comparison of the Brier score and decision tree analysis

For predicting 3-, 6-, and 12-month mortality, the Brier scores were 0.119, 0.119, and 0.128 for the RSF model and 0.138, 0.146, and 0.156 for the Cox model [Table 3]. The Brier score comparison suggested that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis at different time points. Decision tree analysis further indicated that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis [Supplementary Digital Content, Figure 4, http://links.lww.com/CM9/A575].

Discussion

The current study developed a novel online individual mortality risk predictive tool based on an RSF algorithm for ACLF patients. This online individual mortality risk predictive tool could predict individual mortality risk predictive curves at the individual level. In addition, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at user-defined time points. Time-dependent ROC curve, decision tree, and Brier score analyses indicated that the RSF model was superior to the Cox model for predicting the prognosis of ACLF patients.

HE, age, serum sodium level, AKI, RDW, and INR were identified as independent risk factors for ACLF patients by multivariate Cox regression in the current study. The variable importance assessment through the RSF algorithm further proved HE, age, serum sodium level, AKI, RDW, and INR as risk factors for ACLF patients. Previous studies have provided strong clinical evidence for the following variables as risk factors for ACLF patients: HE,[35,36] age,[35,37,38] INR,[4,36,38,39] RDW,[40,41] AKI,[42–44] and serum sodium level.[45,46]

The RSF algorithm could identify the variables that had a nonlinear effect on prognosis.[18] Miao et al[18] reported that the diagnostic accuracy of the RSF model was superior to that of the Cox model for predicting 1-year mortality in patients with cardiac arrhythmias. Similar to a previous study, the current study indicated that the RSF model was superior to the Cox model for predicting the prognosis of ACLF patients.

The present study features several advantages. First, the current study performed a long-term follow-up for ACLF patients until September 2018, providing valuable detailed survival information for the evaluation of the long-term application value of prognosis models. Second, the current study developed an online individual mortality risk predictive tool that could predict individual mortality risk predictive curves for individual patients. Third, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at user-defined time points. To the best of our knowledge, our online individual mortality risk predictive tool was a rare online web tool that could provide individual mortality risk prediction for ACLF patients.

This study also had several shortcomings. First, the current research was that there was no independent external cohort to verify the diagnostic accuracy and clinical application value of the current prognostic model. Second, the algorithm and predictive process of the random living forest model could not be expressed by a conventional formula as a nonparametric model, affecting the generalization and application of research conclusions to a certain extent. Third, several interesting potential risk factors, such as thyroxine and the liver-to-abdominal area ratio, were not enrolled in the survival analysis due to incomplete data.[47,48] Fourth, the sample size of the current study was relatively small, which might affect the reliability of the research conclusions to a certain extent. Prospective cohort studies with more variables and larger sample sizes would help to improve the diagnostic accuracy and clinical application value of prognostic models.

In conclusion, the current study developed a novel online individual mortality risk predictive tool that could predict individual mortality risk predictive curves for an individual patient. Additionally, the current online individual mortality risk predictive tool could further provide the predicted mortality percentages and 95% confidence intervals at user-defined time points, which was valuable for improving individual treatment decisions.

Acknowledgements

The authors thank Mrs. Qingmei Liu for her help and support in the development of the online individual mortality risk predictive tool.

Funding

This study was funded by the Guangdong Medical Science and Technology Foundation (No. B2018237 and No. A2016450) and the Jiangmen Science and Technology Bureau (No. 2019A098).

Conflicts of interest

None.

References

1. Lee WM. Hepatitis B virus infection. N Engl J Med 1997; 337:1733–1745. doi: 10.1056/nejm199712113372406.
2. Bernal W, Jalan R, Quaglia A, Simpson K, Wendon J, Burroughs A. Acute-on-chronic liver failure. Lancet 2015; 386:1576–1587. doi: 10.1016/s0140-6736(15)00309-8.
3. Arroyo V, Moreau R, Jalan R, Gines P. Acute-on-chronic liver failure: a new syndrome that will re-classify cirrhosis. J Hepatol 2015; 62: (1 Suppl): S131–S143. doi: 10.1016/j.jhep.2014.11.045.
4. Malinchoc M, Kamath PS, Gordon FD, Peine CJ, Rank J, ter Borg PC. A model to predict poor survival in patients undergoing transjugular intrahepatic portosystemic shunts. Hepatology 2000; 31:864–871. doi: 10.1053/he.2000.5852.
5. Luca A, Angermayr B, Bertolini G, Koenig F, Vizzini G, Ploner M, et al. An integrated MELD model including serum sodium and age improves the prediction of early mortality in patients with cirrhosis. Liver Transpl 2007; 13:1174–1180. doi: 10.1002/lt.21197.
6. Dominguez M, Rincon D, Abraldes JG, Miquel R, Colmenero J, Bellot P, et al. A new scoring system for prognostic stratification of patients with alcoholic hepatitis. Am J Gastroenterol 2008; 103:2747–2756. doi: 10.1111/j.1572-0241.2008.02104.x.
7. Tong JJ, Zhao W, Mu XY, Xu X, Su HB, Liu XY, et al. Predictive value of the Chinese group on the study of severe hepatitis B-acute-on-chronic liver failure score in the short-term prognosis of patients with hepatitis B virus-related acute-on-chronic liver failure. Chin Med J 2019; 132:1541–1549. doi: 10.1097/cm9.0000000000000298.
8. He WP, Hu JH, Zhao J, Tong JJ, Ding JB, Lin F, et al. Comparison of four prognostic models and a new Logistic regression model to predict short-term prognosis of acute-on-chronic hepatitis B liver failure. Chin Med J 2012; 125:2272–2278.
9. Xu H, Gu X, Tadesse MG, Balasubramanian R. A modified random survival forests algorithm for high dimensional predictors and self-reported outcomes. J Comput Graph Stat 2018; 27:763–772. doi: 10.1080/10618600.2018.1474115.
10. Nasejje JB, Mwambi H. Application of random survival forests in understanding the determinants of under-five child mortality in Uganda in the presence of covariates that satisfy the proportional and non-proportional hazards assumption. BMC Res Notes 2017; 10:459doi: 10.1186/s13104-017-2775-6.
11. Wang H, Li G. A selective review on random survival forests for high dimensional data. Quant Biosci 2017; 36:85–96. doi: 10.22283/qbs.2017.36.2.85.
12. Wang W, Liu W. Integration of gene interaction information into a reweighted random survival forest approach for accurate survival prediction and survival biomarker discovery. Sci Rep 2018; 8:13202doi: 10.1038/s41598-018-31497-0.
13. Adham D, Abbasgholizadeh N, Abazari M. Prognostic factors for survival in patients with gastric cancer using a random survival forest. Asian Pac J Cancer Prev 2017; 18:129–134. doi: 10.22034/apjcp.2017.18.1.129.
14. Wang H, Zhou L. Random survival forest with space extensions for censored data. Artif Intell Med 2017; 79:52–61. doi: 10.1016/j.artmed.2017.06.005.
15. Hsich E, Gorodeski EZ, Blackstone EH, Ishwaran H, Lauer MS. Identifying important risk factors for survival in patient with systolic heart failure using random survival forests. Circ Cardiovasc Qual Outcomes 2011; 4:39–45. doi: 10.1161/CIRCOUTCOMES.110.939371.
16. Wang H, Shen L, Geng J, Wu Y, Xiao H, Zhang F, et al. Prognostic value of cancer antigen-125 for lung adenocarcinoma patients with brain metastasis: a random survival forest prognostic model. Sci Rep 2018; 8:5670doi: 10.1038/s41598-018-23946-7.
17. Akai H, Yasaka K, Kunimatsu A, Nojima M, Kokudo T, Kokudo N, et al. Predicting prognosis of resected hepatocellular carcinoma by radiomics analysis with random survival forest. Diagn Interv Imaging 2018; 99:643–651. doi: 10.1016/j.diii.2018.05.008.
18. Miao F, Cai YP, Zhang YX, Li Y, Zhang YT. Risk prediction of one-year mortality in patients with cardiac arrhythmias using random survival forest. Comput Math Methods Med 2015; 2015:303250doi: 10.1155/2015/303250.
19. Cheng C, Wang Q, Zhu M, Liu K, Zhang Z. Integrated analysis reveals potential long non-coding RNA biomarkers and their potential biological functions for disease free survival in gastric cancer patients. Cancer Cell Int 2019; 19:123doi: 10.1186/s12935-019-0846-6.
20. Zhang N, Zhang J, Zhang H, Liu Y, Zhao W, Wang L, et al. Individualized prediction of survival benefit from postmastectomy radiotherapy for patients with breast cancer with one to three positive axillary lymph nodes. Oncologist 2019; 24:e1286–e1293. doi: 10.1634/theoncologist.2019-0124.
21. Zhang Z, Ouyang Y, Huang Y, Wang P, Li J, He T, et al. Comprehensive bioinformatics analysis reveals potential lncRNA biomarkers for overall survival in patients with hepatocellular carcinoma: an on-line individual risk calculator based on TCGA cohort. Cancer Cell Int 2019; 19:174doi: 10.1186/s12935-019-0890-2.
22. Sarin SK, Kedarisetty CK, Abbas Z, Amarapurkar D, Bihari C, Chan AC, et al. Acute-on-chronic liver failure: Consensus recommendations of the Asian Pacific Association for the Study of the Liver (APASL) 2014. Hepatol Int 2014; 8:453–471. doi: 10.1007/s12072-014-9580-2.
23. Angeli P, Gines P, Wong F, Bernardi M, Boyer TD, Gerbes A, et al. Diagnosis and management of acute kidney injury in patients with cirrhosis: revised consensus recommendations of the International Club of Ascites. J Hepatol 2015; 62:968–974. doi: 10.1016/j.jhep.2014.12.029.
24. Runyon BA. Introduction to the revised American Association for the Study of Liver Diseases Practice Guideline management of adult patients with ascites due to cirrhosis 2012. Hepatology 2013; 57:1651–1653. doi: 10.1002/hep.26359.
25. Vilstrup H, Amodio P, Bajaj J, Cordoba J, Ferenci P, Mullen KD, et al. Hepatic encephalopathy in chronic liver disease: 2014 Practice Guideline by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver. Hepatology 2014; 60:715–735. doi: 10.1002/hep.27210.
26. Qu JM, Cao B. Guidelines for the diagnosis and treatment of adult community acquired pneumonia in China (2016 Edition). Chin J Tubere Respir Dis 2016; 39:241–242. doi: 10.3760/cma.j.issn.1001-0939.2016.04.001.
27. Bai Y, Li ZS. Guidelines for the diagnosis and treatment of acute non-variceal upper gastrointestinal bleeding (2015, Nanchang, China). J Dig Dis 2016; 17:79–87. doi: 10.1111/1751-2980.12319.
28. Blackstone EH. Breaking down barriers: Helpful breakthrough statistical methods you need to understand better. J Thorac Cardiovasc Surg 2001; 122:430–439. doi: 10.1067/mtc.2001.117536.
29. Grunkemeier GL, Wu YX. Bootstrap resampling methods: Something for nothing? Ann Thorac Surg 2004; 77:1142–1144. doi: 10.1016/j.athoracsur.2004.01.005.
30. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015; 350:g7594doi: 10.1136/bmj.g7594.
31. Jaeschke R, Guyatt G, Sackett DL. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1994; 271:389–391. doi: 10.1001/jama.271.5.389.
32. Lin B, Pan CQ, Xie D, Xie J, Xie S, Zhang X, et al. Entecavir improves the outcome of acute-on-chronic liver failure due to the acute exacerbation of chronic hepatitis B. Hepatol Int 2013; 7:460–467. doi: 10.1007/s12072-012-9415-y.
33. Saha-Chaudhuri P, Heagerty PJ. Non-parametric estimation of a time-dependent predictive accuracy curve. Biostatistics 2013; 14:42–59. doi: 10.1093/biostatistics/kxs021.
34. Blanche P, Dartigues JF, Jacqmin-Gadda H. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Stat Med 2013; 32:5381–5397. doi: 10.1002/sim.5958.
35. Luo Y, Xu Y, Li M, Xie Y, Gong G. A new multiparameter integrated MELD model for prognosis of HBV-related acute-on-chronic liver failure. Medicine 2016; 95:e4696doi: 10.1097/md.0000000000004696.
36. Gao F, Zhang Q, Liu Y, Gong G, Mao D, Gong Z, et al. Nomogram prediction of individual prognosis of patients with acute-on-chronic hepatitis B liver failure. Dig Liver Dis 2019; 51:425–433. doi: 10.1016/j.dld.2018.08.023.
37. Shi KQ, Cai YJ, Lin Z, Dong JZ, Wu JM, Wang XD, et al. Development and validation of a prognostic nomogram for acute-on-chronic hepatitis B liver failure. J Gastroenterol Hepatol 2017; 32:497–505. doi: 10.1111/jgh.13502.
38. Cao D, Li DJ, Wang Y, Zhang YH, Chen LY, Wang LC. Clinical significance of CTP combined with ABIC score in predicting the short-term prognosis of patients with hepatitis B virus-related acute-on-chronic liver failure (In Chinese). Chin J Hepa 2019; 27:118–122. doi: 10.3760/cma.j.issn.1007-3418.2019.02.009.
39. Chen L, Zheng J, Cai J, Jie Y, Zhang Y, Li H, et al. Predictive value of age-bilirubin-international normalized ratio-creatinine score in short-term survival of acute-on-chronic hepatitis B liver failure. Cell Physiol Biochem 2018; 51:2484–2495. doi: 10.1159/000495904.
40. Qin J, Qiang L, Chen W, Wu G. Red blood cell distribution width is a independent prognostic indicator for mortality in patients with HBV related acute-on-chronic liver failure (In Chinese). J South Med Univ 2018; 38:1354–1359. doi: 10.12122/j.issn.1673-4254.2018.11.13.
41. Jin L, Gao Y, Ye J, Zou G, Li X. Clinical usefulness of measuring red blood cell distribution width in patients with hepatitis B virus-related acute-on-chronic liver failure. Clin Lab 2017; 63:1403–1410. doi: 10.7754/Clin.Lab.2017.170301.
42. Cai JJ, Wang K, Jiang HQ, Han T. Characteristics, risk factors, and adverse outcomes of hyperkalemia in acute-on-chronic liver failure patients. Biomed Res Int 2019; 2019:6025726doi: 10.1155/2019/6025726.
43. Khatua CR, Panigrahi S, Mishra D, Pradhan S, Sahu SK, Barik RK, et al. Acute kidney injury at admission is a better predictor of mortality than its persistence at 48 h in patients with acute-on-chronic liver failure. J Clin Transl Hepatol 2018; 6:396–401. doi: 10.14218/jcth.2018.00035.
44. Chen N, Chen X, Ding X, Teng J. Analysis of the high incidence of acute kidney injury associated with acute-on-chronic liver failure. Hepatol Int 2018; 12:262–268. doi: 10.1007/s12072-018-9866-x.
45. Kim WR, Biggins SW, Kremers WK, Wiesner RH, Kamath PS, Benson JT, et al. Hyponatremia and mortality among patients on the liver-transplant waiting list. N Engl J Med 2008; 359:1018–1026. doi: 10.1056/NEJMoa0801209.
46. Biggins SW, Kim WR, Terrault NA, Saab S, Balan V, Schiano T, et al. Evidence-based incorporation of serum sodium concentration into MELD. Gastroenterology 2006; 130:1652–1660. doi: 10.1053/j.gastro.2006.02.010.
47. Wu D, Sun Z, Liu X, Rao Q, Chen W, Wang J, et al. HINT: A novel prognostic model for patients with hepatitis B virus-related acute-on-chronic liver failure. Aliment Pharmacol Ther 2018; 48:750–760. doi: 10.1111/apt.14927.
48. Lin S, Chen J, Wang M, Han L, Zhang H, Dong J, et al. Prognostic nomogram for acute-on-chronic hepatitis B liver failure. Oncotarget 2017; 8:109772–109782. doi: 10.18632/oncotarget.21012.
Keywords:

Random survival forest; Acute-on-chronic liver failure; Prognosis

Supplemental Digital Content

Copyright © 2021 The Chinese Medical Association, produced by Wolters Kluwer, Inc. under the CC-BY-NC-ND license.