Secondary Logo

Institutional members access full text with Ovid®


Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery

Bihorac, Azra, MD, MS*,¶¶; Ozrazgat-Baslanti, Tezcan, PhD*,¶¶; Ebadi, Ashkan, PhD*,¶¶; Motaei, Amir, PhD*,¶¶; Madkour, Mohcine, PhD*,¶¶; Pardalos, Panagote M., PhD†,¶¶; Lipori, Gloria, MBA; Hogan, William R., MD, MS§,¶¶; Efron, Philip A., MD; Moore, Frederick, MD; Moldawer, Lyle L., PhD; Wang, Daisy Zhe, PhD||,¶¶; Hobson, Charles E., MD**,§§; Rashidi, Parisa, PhD††,¶¶; Li, Xiaolin, PhD‡‡,¶¶; Momcilovic, Petar, PhD†,¶¶

doi: 10.1097/SLA.0000000000002706

Objective: To accurately calculate the risk for postoperative complications and death after surgery in the preoperative period using machine-learning modeling of clinical data.

Background: Postoperative complications cause a 2-fold increase in the 30-day mortality and cost, and are associated with long-term consequences. The ability to precisely forecast the risk for major complications before surgery is limited.

Methods: In a single-center cohort of 51,457 surgical patients undergoing major inpatient surgery, we have developed and validated an automated analytics framework for a preoperative risk algorithm (MySurgeryRisk) that uses existing clinical data in electronic health records to forecast patient-level probabilistic risk scores for 8 major postoperative complications (acute kidney injury, sepsis, venous thromboembolism, intensive care unit admission >48 hours, mechanical ventilation >48 hours, wound, neurologic, and cardiovascular complications) and death up to 24 months after surgery. We used the area under the receiver characteristic curve (AUC) and predictiveness curves to evaluate model performance.

Results: MySurgeryRisk calculates probabilistic risk scores for 8 postoperative complications with AUC values ranging between 0.82 and 0.94 [99% confidence intervals (CIs) 0.81–0.94]. The model predicts the risk for death at 1, 3, 6, 12, and 24 months with AUC values ranging between 0.77 and 0.83 (99% CI 0.76–0.85).

Conclusions: We constructed an automated predictive analytics framework for machine-learning algorithm with high discriminatory ability for assessing the risk of surgical complications and death using readily available preoperative electronic health records data. The feasibility of this novel algorithm implemented in real time clinical workflow requires further testing.

*Department of Medicine, College of Medicine, University of Florida, Gainesville, FL

Department of Industrial and Systems Engineering, College of Engineering, UF

University of Florida Health, UF

§Department of Health Outcomes and Policy, UF

Department of Surgery, College of Medicine, UF

||Department of Computer and Information Science and Engineering, College of Engineering, UF

**Department of Health Services Research, Management and Policy, UF

††Department of Biomedical Engineering, College of Engineering, UF

‡‡Department of Electrical and Computer Engineering, College of Engineering, UF

§§Department of Surgery, Malcom Randall VAMC, Gainesville, FL

¶¶Precision and Intelligent Systems in Medicine (PRISMAP), University of Florida, Gainesville, FL.

Reprints: Azra Bihorac, MD, MS, Department of Medicine, Precision and Intelligent Systems in Medicine (PrismaP), Division of Nephrology, Hypertension, and Renal Transplantation, PO Box 100224, Gainesville, FL 32610-0224. E-mail:

TO-B and AE contributed equally to the manuscript.

Author contributions: AB conceived the original idea for the study, and sought and obtained funding. AB, AE, and TO-B had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors were involved in acquisition, analysis, or interpretation of data. AB, AE, TO-B, and PM carried out the analysis. The article was written by AB, AE, and TO-B with input from all co-authors. Critical revision of the manuscript for important intellectual content was done by all authors. AB provided the administrative, technical, or material support. Study supervision was done by AB, PM, and PR. AB is guarantor for this article.

Funding: This work was supported by the National Institute of General Medical Sciences under R01 GM110240. Also supported in part by the NIH/NCATS Clinical and Translational Sciences Award to the University of Florida UL1 TR000064. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Previous presentation: Partial results from this research were presented at the University of Florida Research Day.

Conflicts of interest: AB, AE, TOB, PR, PP, PM, GL, XL, DW, and WH were supported by R01 GM110240 from the National Institute of General Medical Sciences. AB and TOB were supported by Sepsis and Critical Illness Research Center Award P50 GM-111152 from the National Institute of General Medical Sciences. TOB has received grant (97071) from Clinical and Translational Science Institute, University of Florida. AB, AE, and TOB had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. University of Florida and AB, TOB, XL, PP, PM, ZW, and PR have a patent pending on real-time use of clinical data for surgical risk prediction using machine learning models in MySurgeryRisk algorithm.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.