Chronic hepatitis B is one of the most prevalent threats to liver health in the world. Acute-on-chronic liver failure (ACLF) is the acute decompensation of liver function based on chronic liver diseases under the actions of different liver attack events. Due to poor basic liver function and multiple organ failure, 60% to 70% of ACLF patients experience rapid aggravation and die within 3 months.[2,3] There is an urgent requirement for a prognostic model to identify ACLF patients with a high mortality risk, who are in urgent need of liver transplantation in the short term. A few prognostic models could provide mortality risk prediction information for ACLF patients.[4–6] However, these prognostic models could only provide predicted mortality for a special group of patients with similar clinical characteristics at the group level,[7,8] but failed to predict individual mortality risk predictive information for individual patients at the individual level.
The random survival forest (RSF) algorithm is a nonparametric algorithm with great clinical application value that has been recommended for prognostic prediction.[9,10] The RSF algorithm can avoid the influence of multicollinearity and can provide objective evaluations of the interactions between different variables. The RSF algorithm can automatically calculate and order the importance of different variables.[12,13] In addition, the RSF method can address the impact of noise caused by missing or incorrect values. The RSF method has been used to develop prognostic models for different diseases.[15–17] As an effective survival analysis method taking nonlinearity into account, the RSF model was superior to the Cox proportional model for prognostic prediction. Recently, several studies developed online mortality risk predictive tools for different tumors, providing individual mortality risk predictive curves at the individual level.[19–21] For clinicians and patients, individual mortality risk predictive curves at the individual level can provide more valuable reference information for individualized treatment decisions.
Therefore, the present study aimed to develop an online individual mortality risk predictive tool for ACLF patients based on an RSF algorithm, which could predict the individual mortality risk predictive curve at the individual level.
The current study retrospectively enrolled ACLF patients from the Department of Infectious Diseases of The First People's Hospital of Foshan, Shunde Hospital of Southern Medical University, and Jiangmen Central Hospital (n = 391). The last follow-up time of the enrolled patients was September 10, 2018. Inclusion criteria: 1. ACLF was diagnosed according to the guidelines of the Asian Pacific Association for the Study of the Liver; 2. Hepatitis B surface antigen (HBsAg) positivity for >6 months or with a clear history of chronic hepatitis B; and 3. Adequate survival information. Exclusion criteria: 1. Other hepatitis viruses (hepatitis A, hepatitis C, hepatitis E, and hepatitis D); 2. Liver cancer or other malignant tumors; 3. Autoimmune liver disease; 4. Liver failure caused by alcoholic liver disease, drug-induced hepatitis, hyperthyroidism, poisoning, and other reasons; 5. Unstable period of cardio-cerebral infarction; 6. Accompanied with kidney diseases; 7. Pregnancy; 8. Patients with follow-up time <1 month after discharge were not included in the final survival analysis to eliminate the influence of confounding factors; and 9. Patients without critical baseline information (ie, hepatic encephalopathy (HE), age, serum sodium level, acute kidney injury (AKI), red cell distribution width [RDW], and international normalization index [INR]) were not included in the final survival analysis. Two hundred seventy-six ACLF patients were included in the final survival analysis as the model dataset. We performed the present research according to the Declaration of Helsinki. This study was approved by the Ethics Committee of the Shunde Hospital, Southern Medical University (No. 20171108). As a retrospective study and data analysis were performed anonymously, this study was exempt from the informed consent from patients. The current study eliminated all privacy information that could identify the individual information of patients to protect the privacy of the enrolled patients.
Diagnostic criteria and references
The diseases and complications were diagnosed according to the original studies: ACLF, AKI, hepatorenal syndrome (HRS), HE, pulmonary infection (PI), and gastrointestinal hemorrhage (GH).
Three prognostic models were calculated according to the previous formula: model for end-stage liver disease (MELD) = 9.57 × loge (creatinine [mg/dL]) + 3.78 × loge (bilirubin [mg/dL]) + 11.2 × loge (INR) + 6.43 × (etiology: 0 for cholestatic or alcoholic; 1 for otherwise); International normalized ratio and creatinine score (ABIC) = (age × 0.1) + (serum bilirubin (mg/dL) × 0.08) + (serum creatine (mg/dL) × 0.3) + (INR × 0.8); Integrated MELD (iMELD) = MELD + [age (year) × 0.3] − 0.7 × Na (mmol/L)] + 100
Validation cohort based on the bootstrap resampling method
The bootstrap resampling technique is a statistical sampling method with replacement from the original cohort, which is suitable for internal validation of prognostic models.[28,29] Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis proposed the bootstrapping resampling method to be a prerequisite of prognostic model development in case the external original dataset was not available. The current study constructed a validation cohort by drawing patients from the model dataset based on the resampling method (n = 276).
The statistical analyses were carried out by R software (version 3.6.1). Continuous variables are depicted as the mean ± standard deviation or median (first quartile, third quartile). Continuous variables were compared by t test or the Kruskal-Wallis H test. Categorical variables were compared by the chi-squared test or Fisher exact test. The RSF model in the current study was conducted with reference to the model method in several articles performed by other researchers. The RSF model is an ensemble tree-based algorithm for variable selection in high-dimensional datasets. RSF performs well in calculation efficiency and predictive performance with low generalization error. When there are complex interactions between covariate Zs, the RSF model is particularly suitable for variable selection.[9,10] The Brier score was used to evaluate the diagnostic accuracy of prognostic models according to the original studies.[31,32] The predictive accuracy of the model with a smaller Brier score is superior to that of the model with a higher Brier score. The weighted mean rank estimation method was used to compare the differences between the areas under the time-dependent ROC curves (AUROCs).[33,34] A P value < 0.05 was defined as statistically significant.
There were 276 ACLF patients in the model cohort. The validation cohort contained 276 ACLF patients who were drawn from the original model cohort by the bootstrapping resampling method. The comparisons of baseline characteristics of patients in the model and validation cohorts are summarized in Table 1.
Table 1 -
Baseline characteristics of patients in the model and validation groups.
N = 276
N = 276
|Overall survival (months)
||19.3 (0.9, 42)
||21.8 (1.1, 43.2)
||43 (36, 53)
||40 (35, 50)
||71 (60, 83)
||72 (61, 83)
|Uric acid (μmol/L)
||201.4 (201.4, 201.4)
||201.4 (197, 201.4)
|Fasting plasma glucose (mmol/L)
||5.8 (4.2, 6.8)
||5.6 (4.2, 6.6)
|Direct bilirubin (μmol/L)
||188.5 (103.2, 280.4)
||190.8 (105.4, 285.5)
||30.8 ± 5.7
||31.1 ± 5.3
||32.5 (27.7, 37.4)
||33.2 (27.7, 38.7)
|Alanine aminotransferase (U/L)
||576 (136, 1319)
||632 (167, 1304)
|Glutamic oxaloacetic transaminase (U/L)
||396 (140, 873)
||449 (146, 934)
|Glutamyl transferase (U/L)
||111 (72, 147)
||113 (77, 157)
|Alpha fetoprotein (ng/mL)
||74.5 (13.9, 123.4)
||88.5 (17.6, 123.4)
|Collage Type IV (ng/mL)
|N-Terminal Procollagen III Propeptide (ng/mL)
||5.7 (4.2, 7.3)
||6.2 (4.5, 7.5)
|White blood cell (109/L)
||7.2 (5.5, 9.5)
||7.2 (5.5, 9.2)
||3.8 (2.4, 5.5)
||3.6 (2.3, 5.2)
||0.7 (0.6, 0.8)
||0.7 (0.6, 0.7)
||126.9 (111, 140)
||128 (111, 140.2)
|Platelet distribution width (%)
||16.4 (15.6, 16.6)
||16.3 (15.5, 16.5)
|Mean platelet volume (fl)
||11.1 (10, 11.7)
||11.1 (10, 11.7)
||123.2 (83, 161)
||129 (85.5, 164)
|Red cell distribution width (fl)
||42.4 (22.2, 48.2)
||42.6 (36.9, 47.7)
|Prothrombin time (sec)
||22.4 (18.4, 28.8)
||21.4 (18.2, 28)
|Serum sodium level (mmol/L)
||137 (135, 140.1)
||138 (134.8, 140.5)
|International normalization index
||1.9 (1.5, 2.4)
||1.8 (1.5, 2.2)
||1.4 (1, 1.7)
||1.5 (1.1, 1.8)
|Total cholesterol (mmol/L)
||2.9 (2.5, 3.2)
||2.9 (2.8, 3.2)
||1.4 (1.1, 1.5)
||1.4 (1.3, 1.5)
||7.7 (6.3, 8.8)
||7.6 (6.3, 8.7)
|Model for end-stage liver disease
||21.8 (17.7, 25.7)
||22.4 (19.2, 24.9)
||35.461 (30.542, 40.243)
|Acute kidney injury
Continuous variables are expressed as the mean ± standard deviation or median (first quartile, third quartile) or n(%) as appropriate. ∗Kruskal-Wallis H test. †χ2 values. ‡t values. ABIC: International normalized ratio and creatinine score; AKI: Acute kidney injury; HE: Hepatic encephalopathy; HRS: Hepatorenal syndrome; INR: International normalization index; MELD: Model for end-stage liver disease; PT: Prothrombin time; PI: Pulmonary infection; RSF: Random survival forest; RDW: Red cell distribution width.
Importance evaluation of variables
An RSF algorithm was carried out to present the importance evaluation chart of the study variables. The importance evaluation chart [Figure 1] indicated the importance of the top 30 variables. AKI, HRS, HE, age, RDW, prothrombin time (PT), triglyceride, gastrointestinal bleeding, INR, and platelet count were identified as prognostic factors by the RSF algorithm.
Cox proportional hazards regression model
To construct a simplified RSF model for clinical application, multivariate Cox regression (step forward method) was used to explore the most valuable variables for predicting the prognosis of ACLF patients. As shown in Table 2, HE, age, serum sodium level, AKI, RDW, and INR were identified as independent risk factors for ACLF patients. Figure 2 presents a prognostic nomogram for ACLF patients based on the Cox proportional hazards regression model.
Table 2 -
Results of univariate Cox regression analysis and multivariate Cox regression analysis of the included variables.
|Serum sodium level
AKI: Acute renal injury; CI: Confidence interval; HE: Hepatic encephalopathy; INR: International normalized ratio; HR: Hazard ratio; RDW: Red cell distribution width.
Simplified RSF model
A simplified RSF model was developed based on HE, age, serum sodium level, AKI, RDW, and INR. The diagnostic performance of the RSF model was validated through the out-of-band (OOB) method [Figure 3]. As shown in Figure 3A, the green line represents the Nelson-Aalen estimator survival curve, and the red line represents the overall ensemble survival curve. The overall ensemble survival curve was highly consistent with the Nelson-Aalen estimation survival curve, indicating that the estimated survival curve (green line) by the RSF model was in good agreement with the real survival curve (red line).
An online individual mortality risk predictive tool
The current study further developed an online individual mortality risk predictive tool based on the RSF algorithm for ACLF patients. As shown in Figure 4A, our online individual mortality risk predictive tool could predict individual mortality risk percentages at different time points based on the RSF algorithm. As the reference survival curve, the current online individual mortality risk predictive tool also provided the individual mortality risk predicted curves generated by the Cox regression algorithm [Figure 4B]. In addition, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at different time points [Figure 4C]. This online individual mortality risk predictive tool is available at https://zhangzhiqiao13.shinyapps.io/Individual_mortality_risk_predictive_tool_for_liver_failure/.
Performance of the RSF model in the model cohort
The AUROCs for predicting 3-, 6-, and 12-month mortality were 0.916, 0.916, and 0.905, respectively, for the RSF model in the model cohort [Figure 5A]. The mortality of patients with high RSF scores was significantly poorer than that of patients with low RSF scores [Figure 5B]. Calibration curves demonstrated that the predicted mortality was highly consistent with the actual mortality in the model cohort [Supplementary Digital Content, Figure 1, http://links.lww.com/CM9/A572].
Internal validation of RSF model
In the validation cohort, the AUROCs for predicting 3-, 6-, and 12-month mortality were 0.912, 0.910, and 0.880, respectively, for the RSF model [Figure 6A]. Figure 6B indicates that the mortality of patients with high RSF scores was significantly poorer than that of patients with low RSF scores. Calibration curves demonstrated that the predicted mortality was consistent with the actual mortality in the validation cohort [Supplementary Digital Content, Figure 2, http://links.lww.com/CM9/A573].
Comparison of diagnostic accuracy
The AUROCs of the RSF model were superior to those of the Cox model for predicting 3-, 6-, and 12-month mortality [Table 3 and Supplementary Digital Content, Figure 3, http://links.lww.com/CM9/A574]. The nonparametric comparison suggested that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis at different time points.
Table 3 -
Comparison of diagnostic accuracy in the model group.
ABIC: International normalized ratio and creatinine score; AUROC: area under the time-dependent ROC curves; iMELD: Integrated MELD; MELD: Model for end-stage liver disease; MELD-Na: MELD-sodium; MESO: MELD and serum sodium ratio; RSF: Random survival forest.
Comparison of the Brier score and decision tree analysis
For predicting 3-, 6-, and 12-month mortality, the Brier scores were 0.119, 0.119, and 0.128 for the RSF model and 0.138, 0.146, and 0.156 for the Cox model [Table 3]. The Brier score comparison suggested that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis at different time points. Decision tree analysis further indicated that the RSF model was superior to the Cox, MELD, ABIC, and iMELD models for predicting prognosis [Supplementary Digital Content, Figure 4, http://links.lww.com/CM9/A575].
The current study developed a novel online individual mortality risk predictive tool based on an RSF algorithm for ACLF patients. This online individual mortality risk predictive tool could predict individual mortality risk predictive curves at the individual level. In addition, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at user-defined time points. Time-dependent ROC curve, decision tree, and Brier score analyses indicated that the RSF model was superior to the Cox model for predicting the prognosis of ACLF patients.
HE, age, serum sodium level, AKI, RDW, and INR were identified as independent risk factors for ACLF patients by multivariate Cox regression in the current study. The variable importance assessment through the RSF algorithm further proved HE, age, serum sodium level, AKI, RDW, and INR as risk factors for ACLF patients. Previous studies have provided strong clinical evidence for the following variables as risk factors for ACLF patients: HE,[35,36] age,[35,37,38] INR,[4,36,38,39] RDW,[40,41] AKI,[42–44] and serum sodium level.[45,46]
The RSF algorithm could identify the variables that had a nonlinear effect on prognosis. Miao et al reported that the diagnostic accuracy of the RSF model was superior to that of the Cox model for predicting 1-year mortality in patients with cardiac arrhythmias. Similar to a previous study, the current study indicated that the RSF model was superior to the Cox model for predicting the prognosis of ACLF patients.
The present study features several advantages. First, the current study performed a long-term follow-up for ACLF patients until September 2018, providing valuable detailed survival information for the evaluation of the long-term application value of prognosis models. Second, the current study developed an online individual mortality risk predictive tool that could predict individual mortality risk predictive curves for individual patients. Third, the current online individual mortality risk predictive tool could provide predicted mortality percentages and 95% confidence intervals at user-defined time points. To the best of our knowledge, our online individual mortality risk predictive tool was a rare online web tool that could provide individual mortality risk prediction for ACLF patients.
This study also had several shortcomings. First, the current research was that there was no independent external cohort to verify the diagnostic accuracy and clinical application value of the current prognostic model. Second, the algorithm and predictive process of the random living forest model could not be expressed by a conventional formula as a nonparametric model, affecting the generalization and application of research conclusions to a certain extent. Third, several interesting potential risk factors, such as thyroxine and the liver-to-abdominal area ratio, were not enrolled in the survival analysis due to incomplete data.[47,48] Fourth, the sample size of the current study was relatively small, which might affect the reliability of the research conclusions to a certain extent. Prospective cohort studies with more variables and larger sample sizes would help to improve the diagnostic accuracy and clinical application value of prognostic models.
In conclusion, the current study developed a novel online individual mortality risk predictive tool that could predict individual mortality risk predictive curves for an individual patient. Additionally, the current online individual mortality risk predictive tool could further provide the predicted mortality percentages and 95% confidence intervals at user-defined time points, which was valuable for improving individual treatment decisions.
The authors thank Mrs. Qingmei Liu for her help and support in the development of the online individual mortality risk predictive tool.
This study was funded by the Guangdong Medical Science and Technology Foundation (No. B2018237 and No. A2016450) and the Jiangmen Science and Technology Bureau (No. 2019A098).
Conflicts of interest
1. Lee WM. Hepatitis B virus infection. N Engl J Med
1997; 337:1733–1745. doi: 10.1056/nejm199712113372406.
2. Bernal W, Jalan R, Quaglia A, Simpson K, Wendon J, Burroughs A. Acute-on-chronic liver failure. Lancet
2015; 386:1576–1587. doi: 10.1016/s0140-6736(15)00309-8.
3. Arroyo V, Moreau R, Jalan R, Gines P. Acute-on-chronic liver failure: a new syndrome that will re-classify cirrhosis. J Hepatol
2015; 62: (1 Suppl): S131–S143. doi: 10.1016/j.jhep.2014.11.045.
4. Malinchoc M, Kamath PS, Gordon FD, Peine CJ, Rank J, ter Borg PC. A model to predict poor survival in patients undergoing transjugular intrahepatic portosystemic shunts. Hepatology
2000; 31:864–871. doi: 10.1053/he.2000.5852.
5. Luca A, Angermayr B, Bertolini G, Koenig F, Vizzini G, Ploner M, et al. An integrated MELD model including serum sodium and age improves the prediction of early mortality in patients with cirrhosis. Liver Transpl
2007; 13:1174–1180. doi: 10.1002/lt.21197.
6. Dominguez M, Rincon D, Abraldes JG, Miquel R, Colmenero J, Bellot P, et al. A new scoring system for prognostic stratification of patients with alcoholic hepatitis. Am J Gastroenterol
2008; 103:2747–2756. doi: 10.1111/j.1572-0241.2008.02104.x.
7. Tong JJ, Zhao W, Mu XY, Xu X, Su HB, Liu XY, et al. Predictive value of the Chinese group on the study of severe hepatitis B-acute-on-chronic liver failure score in the short-term prognosis
of patients with hepatitis B virus-related acute-on-chronic liver failure. Chin Med J
2019; 132:1541–1549. doi: 10.1097/cm9.0000000000000298.
8. He WP, Hu JH, Zhao J, Tong JJ, Ding JB, Lin F, et al. Comparison of four prognostic models and a new Logistic regression model to predict short-term prognosis
of acute-on-chronic hepatitis B liver failure. Chin Med J
9. Xu H, Gu X, Tadesse MG, Balasubramanian R. A modified random survival forests algorithm for high dimensional predictors and self-reported outcomes. J Comput Graph Stat
2018; 27:763–772. doi: 10.1080/10618600.2018.1474115.
10. Nasejje JB, Mwambi H. Application of random survival forests in understanding the determinants of under-five child mortality in Uganda in the presence of covariates that satisfy the proportional and non-proportional hazards assumption. BMC Res Notes
2017; 10:459doi: 10.1186/s13104-017-2775-6.
11. Wang H, Li G. A selective review on random survival forests for high dimensional data. Quant Biosci
2017; 36:85–96. doi: 10.22283/qbs.2017.36.2.85.
12. Wang W, Liu W. Integration of gene interaction information into a reweighted random survival forest approach for accurate survival prediction and survival biomarker discovery. Sci Rep
2018; 8:13202doi: 10.1038/s41598-018-31497-0.
13. Adham D, Abbasgholizadeh N, Abazari M. Prognostic factors for survival in patients with gastric cancer using a random survival forest. Asian Pac J Cancer Prev
2017; 18:129–134. doi: 10.22034/apjcp.2017.18.1.129.
14. Wang H, Zhou L. Random survival forest with space extensions for censored data. Artif Intell Med
2017; 79:52–61. doi: 10.1016/j.artmed.2017.06.005.
15. Hsich E, Gorodeski EZ, Blackstone EH, Ishwaran H, Lauer MS. Identifying important risk factors for survival in patient with systolic heart failure using random survival forests. Circ Cardiovasc Qual Outcomes
2011; 4:39–45. doi: 10.1161/CIRCOUTCOMES.110.939371.
16. Wang H, Shen L, Geng J, Wu Y, Xiao H, Zhang F, et al. Prognostic value of cancer antigen-125 for lung adenocarcinoma patients with brain metastasis: a random survival forest prognostic model. Sci Rep
2018; 8:5670doi: 10.1038/s41598-018-23946-7.
17. Akai H, Yasaka K, Kunimatsu A, Nojima M, Kokudo T, Kokudo N, et al. Predicting prognosis
of resected hepatocellular carcinoma by radiomics analysis with random survival forest. Diagn Interv Imaging
2018; 99:643–651. doi: 10.1016/j.diii.2018.05.008.
18. Miao F, Cai YP, Zhang YX, Li Y, Zhang YT. Risk prediction of one-year mortality in patients with cardiac arrhythmias using random survival forest. Comput Math Methods Med
2015; 2015:303250doi: 10.1155/2015/303250.
19. Cheng C, Wang Q, Zhu M, Liu K, Zhang Z. Integrated analysis reveals potential long non-coding RNA biomarkers and their potential biological functions for disease free survival in gastric cancer patients. Cancer Cell Int
2019; 19:123doi: 10.1186/s12935-019-0846-6.
20. Zhang N, Zhang J, Zhang H, Liu Y, Zhao W, Wang L, et al. Individualized prediction of survival benefit from postmastectomy radiotherapy for patients with breast cancer with one to three positive axillary lymph nodes. Oncologist
2019; 24:e1286–e1293. doi: 10.1634/theoncologist.2019-0124.
21. Zhang Z, Ouyang Y, Huang Y, Wang P, Li J, He T, et al. Comprehensive bioinformatics analysis reveals potential lncRNA biomarkers for overall survival in patients with hepatocellular carcinoma: an on-line individual risk calculator based on TCGA cohort. Cancer Cell Int
2019; 19:174doi: 10.1186/s12935-019-0890-2.
22. Sarin SK, Kedarisetty CK, Abbas Z, Amarapurkar D, Bihari C, Chan AC, et al. Acute-on-chronic liver failure: Consensus recommendations of the Asian Pacific Association for the Study of the Liver (APASL) 2014. Hepatol Int
2014; 8:453–471. doi: 10.1007/s12072-014-9580-2.
23. Angeli P, Gines P, Wong F, Bernardi M, Boyer TD, Gerbes A, et al. Diagnosis and management of acute kidney injury in patients with cirrhosis: revised consensus recommendations of the International Club of Ascites. J Hepatol
2015; 62:968–974. doi: 10.1016/j.jhep.2014.12.029.
24. Runyon BA. Introduction to the revised American Association for the Study of Liver Diseases Practice Guideline management of adult patients with ascites due to cirrhosis 2012. Hepatology
2013; 57:1651–1653. doi: 10.1002/hep.26359.
25. Vilstrup H, Amodio P, Bajaj J, Cordoba J, Ferenci P, Mullen KD, et al. Hepatic encephalopathy in chronic liver disease: 2014 Practice Guideline by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver. Hepatology
2014; 60:715–735. doi: 10.1002/hep.27210.
26. Qu JM, Cao B. Guidelines for the diagnosis and treatment of adult community acquired pneumonia in China (2016 Edition). Chin J Tubere Respir Dis
2016; 39:241–242. doi: 10.3760/cma.j.issn.1001-0939.2016.04.001.
27. Bai Y, Li ZS. Guidelines for the diagnosis and treatment of acute non-variceal upper gastrointestinal bleeding (2015, Nanchang, China). J Dig Dis
2016; 17:79–87. doi: 10.1111/1751-2980.12319.
28. Blackstone EH. Breaking down barriers: Helpful breakthrough statistical methods you need to understand better. J Thorac Cardiovasc Surg
2001; 122:430–439. doi: 10.1067/mtc.2001.117536.
29. Grunkemeier GL, Wu YX. Bootstrap resampling methods: Something for nothing? Ann Thorac Surg
2004; 77:1142–1144. doi: 10.1016/j.athoracsur.2004.01.005.
30. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis
or diagnosis (TRIPOD): the TRIPOD statement. BMJ
2015; 350:g7594doi: 10.1136/bmj.g7594.
31. Jaeschke R, Guyatt G, Sackett DL. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA
1994; 271:389–391. doi: 10.1001/jama.271.5.389.
32. Lin B, Pan CQ, Xie D, Xie J, Xie S, Zhang X, et al. Entecavir improves the outcome of acute-on-chronic liver failure due to the acute exacerbation of chronic hepatitis B. Hepatol Int
2013; 7:460–467. doi: 10.1007/s12072-012-9415-y.
33. Saha-Chaudhuri P, Heagerty PJ. Non-parametric estimation of a time-dependent predictive accuracy curve. Biostatistics
2013; 14:42–59. doi: 10.1093/biostatistics/kxs021.
34. Blanche P, Dartigues JF, Jacqmin-Gadda H. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Stat Med
2013; 32:5381–5397. doi: 10.1002/sim.5958.
35. Luo Y, Xu Y, Li M, Xie Y, Gong G. A new multiparameter integrated MELD model for prognosis
of HBV-related acute-on-chronic liver failure. Medicine
2016; 95:e4696doi: 10.1097/md.0000000000004696.
36. Gao F, Zhang Q, Liu Y, Gong G, Mao D, Gong Z, et al. Nomogram prediction of individual prognosis
of patients with acute-on-chronic hepatitis B liver failure. Dig Liver Dis
2019; 51:425–433. doi: 10.1016/j.dld.2018.08.023.
37. Shi KQ, Cai YJ, Lin Z, Dong JZ, Wu JM, Wang XD, et al. Development and validation of a prognostic nomogram for acute-on-chronic hepatitis B liver failure. J Gastroenterol Hepatol
2017; 32:497–505. doi: 10.1111/jgh.13502.
38. Cao D, Li DJ, Wang Y, Zhang YH, Chen LY, Wang LC. Clinical significance of CTP combined with ABIC score in predicting the short-term prognosis
of patients with hepatitis B virus-related acute-on-chronic liver failure (In Chinese). Chin J Hepa
2019; 27:118–122. doi: 10.3760/cma.j.issn.1007-3418.2019.02.009.
39. Chen L, Zheng J, Cai J, Jie Y, Zhang Y, Li H, et al. Predictive value of age-bilirubin-international normalized ratio-creatinine score in short-term survival of acute-on-chronic hepatitis B liver failure. Cell Physiol Biochem
2018; 51:2484–2495. doi: 10.1159/000495904.
40. Qin J, Qiang L, Chen W, Wu G. Red blood cell distribution width is a independent prognostic indicator for mortality in patients with HBV related acute-on-chronic liver failure (In Chinese). J South Med Univ
2018; 38:1354–1359. doi: 10.12122/j.issn.1673-4254.2018.11.13.
41. Jin L, Gao Y, Ye J, Zou G, Li X. Clinical usefulness of measuring red blood cell distribution width in patients with hepatitis B virus-related acute-on-chronic liver failure. Clin Lab
2017; 63:1403–1410. doi: 10.7754/Clin.Lab.2017.170301.
42. Cai JJ, Wang K, Jiang HQ, Han T. Characteristics, risk factors, and adverse outcomes of hyperkalemia in acute-on-chronic liver failure patients. Biomed Res Int
2019; 2019:6025726doi: 10.1155/2019/6025726.
43. Khatua CR, Panigrahi S, Mishra D, Pradhan S, Sahu SK, Barik RK, et al. Acute kidney injury at admission is a better predictor of mortality than its persistence at 48 h in patients with acute-on-chronic liver failure. J Clin Transl Hepatol
2018; 6:396–401. doi: 10.14218/jcth.2018.00035.
44. Chen N, Chen X, Ding X, Teng J. Analysis of the high incidence of acute kidney injury associated with acute-on-chronic liver failure. Hepatol Int
2018; 12:262–268. doi: 10.1007/s12072-018-9866-x.
45. Kim WR, Biggins SW, Kremers WK, Wiesner RH, Kamath PS, Benson JT, et al. Hyponatremia and mortality among patients on the liver-transplant waiting list. N Engl J Med
2008; 359:1018–1026. doi: 10.1056/NEJMoa0801209.
46. Biggins SW, Kim WR, Terrault NA, Saab S, Balan V, Schiano T, et al. Evidence-based incorporation of serum sodium concentration into MELD. Gastroenterology
2006; 130:1652–1660. doi: 10.1053/j.gastro.2006.02.010.
47. Wu D, Sun Z, Liu X, Rao Q, Chen W, Wang J, et al. HINT: A novel prognostic model for patients with hepatitis B virus-related acute-on-chronic liver failure. Aliment Pharmacol Ther
2018; 48:750–760. doi: 10.1111/apt.14927.
48. Lin S, Chen J, Wang M, Han L, Zhang H, Dong J, et al. Prognostic nomogram for acute-on-chronic hepatitis B liver failure. Oncotarget
2017; 8:109772–109782. doi: 10.18632/oncotarget.21012.