Institutional members access full text with Ovid®

Share this article on:

Time-dependent prediction and evaluation of variable importance using superlearning in high-dimensional clinical data

Hubbard, Alan PhD; Munoz, Ivan Diaz MS; Decker, Anna MA; Holcomb, John B. MD; Schreiber, Martin A. MD; Bulger, Eileen M. MD; Brasel, Karen J. MD, MPH; Fox, Erin E. PhD; del Junco, Deborah J. PhD; Wade, Charles E. PhD; Rahbar, Mohammad H. PhD; Cotton, Bryan A. MD, MPH; Phelan, Herb A. MD, MSCS; Myers, John G. MD; Alarcon, Louis H. MD; Muskat, Peter MD; Cohen, Mitchell J. MD; on behalf of the PROMMTT Study Group

Journal of Trauma and Acute Care Surgery: July 2013 - Volume 75 - Issue - p S53–S60
doi: 10.1097/TA.0b013e3182914553
Original Articles

BACKGROUND: Prediction of outcome after injury is fraught with uncertainty and statistically beset by misspecified models. Single–time point regression only gives prediction and inference at one time, of dubious value for continuous prediction of ongoing bleeding. New statistical machine learning techniques such as SuperLearner (SL) exist to make superior prediction at iterative time points while evaluating the changing relative importance of each measured variable on an outcome. This then can provide continuously changing prediction of outcome and evaluation of which clinical variables likely drive a particular outcome.

METHODS: PROMMTT data were evaluated using both naive (standard stepwise logistic regression) and SL techniques to develop a time-dependent prediction of future mortality within discrete time intervals. We avoided both underfitting and overfitting using cross validation to select an optimal combination of predictors among candidate predictors/machine learning algorithms. SL was also used to produce interval-specific robust measures of variable importance measures (VIM resulting in an ordered list of variables, by time point) that have the strongest impact on future mortality.

RESULTS: Nine hundred eighty patients had complete clinical and outcome data and were included in the analysis. The prediction of ongoing transfusion with SL was superior to the naive approach for all time intervals (correlations of cross-validated predictions with the outcome were 0.819, 0.789, 0.792 for time intervals 30–90, 90–180, 180–360, >360 minutes). The estimated VIM of mortality also changed significantly at each time point.

CONCLUSION: The SL technique for prediction of outcome from a complex dynamic multivariate data set is superior at each time interval to standard models. In addition, the SL VIM at each time point provides insight into the time-specific drivers of future outcome, patient trajectory, and targets for clinical intervention. Thus, this automated approach mimics clinical practice, changing form and content through time to optimize the accuracy of the prognosis based on the evolving trajectory of the patient.

From the School of Public Health (A.H., I.D.M., A.D.), University of California-Berkeley, Berkeley, California; Department of Surgery (J.B.H., D.J.d.J., C.E.W., B.A.C.), Center for Translational Injury Research, Division of Acute Care Surgery, Medical School, University of Texas Health Science Center at Houston, Houston, Texas; Division of Trauma, Critical Care and Acute Care Surgery (M.A.S.), School of Medicine, Oregon Health & Science University, Portland, Oregon; Department of Surgery (E.M.B.), Division of Trauma and Critical Care, School of Medicine, University of Washington, Seattle, Washington; Department of Surgery (K.J.B.), Division of Trauma and Critical Care, Medical College of Wisconsin, Milwaukee, Wisconsin; Biostatistics/Epidemiology/Research Design Core (E.E.F., M.H.R.), Center for Clinical and Translational Sciences, University of Texas Health Science Center at Houston, Houston, Texas; Division of Epidemiology, Human Genetics and Environmental Sciences (M.H.R.), School of Public Health, University of Texas Health Science Center at Houston, Houston, Texas; Department of Surgery (H.A.P.), Division of Burn/Trauma/Critical Care, Medical School, University of Texas Southwestern Medical Center at Dallas, Dallas, Texas; Department of Surgery (J.G.M.), Division of Trauma, School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, Texas; Department of Surgery (L.H.A.), Division of Trauma and General Surgery, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania; Department of Surgery (P.M.), Division of Trauma/Critical Care, College of Medicine, University of Cincinnati, Cincinnati, Ohio; Department of Surgery (M.J.C.), Division of General Surgery, School of Medicine, University of California-San Francisco, San Francisco, California.

This study was presented at the PROMMTT Symposium held at the 71st Annual Meeting of the American Association for the Surgery of Trauma, September 10–15, 2012, in Kauai, Hawaii.

The sponsors did not have any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit this manuscript for publication.

The views and opinions expressed in this article are those of the authors and do not reflect the official policy or position of the Army Medical Department, Department of the Army, the Department of Defense, or the US Government.

Address for reprints: Alan Hubbard, PhD, School of Public Health, University of California, Berkeley, CA 94720; email: hubbard@berkeley.edu.

© 2013 Lippincott Williams & Wilkins, Inc.