Develop and implement a machine learning algorithm to predict severe sepsis and septic shock and evaluate the impact on clinical practice and patient outcomes.
Retrospective cohort for algorithm derivation and validation, pre-post impact evaluation.
Tertiary teaching hospital system in Philadelphia, PA.
All non-ICU admissions; algorithm derivation July 2011 to June 2014 (n = 162,212); algorithm validation October to December 2015 (n = 10,448); silent versus alert comparison January 2016 to February 2017 (silent n = 22,280; alert n = 32,184).
A random-forest classifier, derived and validated using electronic health record data, was deployed both silently and later with an alert to notify clinical teams of sepsis prediction.
Patients identified for training the algorithm were required to have International Classification of Diseases, 9th Edition codes for severe sepsis or septic shock and a positive blood culture during their hospital encounter with either a lactate greater than 2.2 mmol/L or a systolic blood pressure less than 90 mm Hg. The algorithm demonstrated a sensitivity of 26% and specificity of 98%, with a positive predictive value of 29% and positive likelihood ratio of 13. The alert resulted in a small statistically significant increase in lactate testing and IV fluid administration. There was no significant difference in mortality, discharge disposition, or transfer to ICU, although there was a reduction in time-to-ICU transfer.
Our machine learning algorithm can predict, with low sensitivity but high specificity, the impending occurrence of severe sepsis and septic shock. Algorithm-generated predictive alerts modestly impacted clinical measures. Next steps include describing clinical perception of this tool and optimizing algorithm design and delivery.
1Department of Medicine, Hospital of the University of Pennsylvania, Philadelphia, PA.
2University of Pennsylvania Health System, Philadelphia, PA.
3Division of Pulmonary, Allergy, and Critical Care Medicine, Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA.
4Department of Nursing, Hospital of the University of Pennsylvania, Philadelphia, PA.
5Department of Clinical Informatics, Pennsylvania Hospital, Philadelphia, PA.
6Penn Presbyterian Medical Center, Philadelphia, PA.
7Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA
8Center for Evidence-based Practice, University of Pennsylvania Health System, Philadelphia, PA.
*See also p. 1650.
Drs. Giannini, Ginestra, Chivers, Draugelis, Schweickert, Fuchs, Meadows, Lynch, Donnelly, Pavan, Fishman, Hanson, and Umscheid contributed to conception and design. Drs. Giannini, Ginestra, Chivers, Draugelis, Hanish, Meadows, Lynch and Donnelly contributed to data collection. Drs. Giannini, Ginestra, Chivers, Draugelis, and Umscheid contributed to analysis and interpretation of data. Drs. Giannini, Ginestra, Chivers, and Umscheid contributed to drafting of article. Drs. Giannini, Ginestra, and Umscheid contributed to critical revision of article for important intellectual content.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal).
Supported, in part, by the National Center for Research Resources (grant no. UL1RR024134), which is now at the National Center for Advancing Translational Sciences (grant noUL1TR000003). Dr. Umscheid received support for article research from the National Institutes of Health (NIH). The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Dr. Schweickert received funding from Arjo, Hill Rom, the Society of Critical Care Medicine (consulting), and American College of Physicians. Dr. Umscheid’s institution received funding from National Center for Research Resources (grant no. UL1RR024134), which is now at the National Center for Advancing Translational Sciences (grant no. UL1TR000003); Agency for Healthcare Research and Quality Contracts Evidence-based Practice Center; and the U.S. Food and Drug Administration. He received funding from the Patient-Centered Outcomes Research Institute Advisory Panel and Northwell Health (honoraria for grand rounds). The remaining authors have disclosed that they do not have any potential conflicts of interest.
Address requests for reprints to: Craig A. Umscheid, MD, MS, Office of Clinical Excellence, University of Chicago Medicine, American School Building, 850 E. 58th Street, Suite 123, Office 128, MC 1135, Chicago, IL 60637. E-mail: firstname.lastname@example.org