With the development of machine learning techniques to create prediction models using big data, medical researchers have started to experiment with these methods. Many studies have evaluated models predicting complications, readmissions, and patient reported outcomes in spine surgery, however, most have used traditional regression modeling techniques. Dr. Jain and colleagues evaluated two of these machine learning statistical approaches (random forest and elastic net) and compared their results to a traditional logistic regression model predicting discharge to a skilled nursing facility, 90-day readmission, and 90-day major complication rate following lumbar or thoracolumbar fusion involving at least 3 levels. They used the State Inpatient Database from North Carolina, Nebraska, New York, California, and Florida from 2005 to 2010. They identified approximately 38,000 patients undergoing long segment fusion for degenerative conditions. Thirty-five percent were discharged to a skilled nursing facility, 19% were readmitted within 90 days, and 13% experienced a major medical complication. In general, age was a powerful predictor of all adverse outcomes in all of the models. Not surprisingly, medical comorbidities were also strong predictors of all 3 outcomes. While they did not evaluate many surgical factors, an anterior-posterior approach was strongly associated with readmission and complications. Ironically, smoking was associated with lower rates of all 3 adverse outcomes. They used area under the curve analysis to assess the accuracy of the predictive models and found that logistic regression slightly outperformed the machine learning techniques. The models were most accurate in predicting discharge to a nursing facility and least accurate in predicting readmission.
This study did not reveal any new insights regarding adverse outcomes following long segment lumbar fusion surgery, namely that age, comorbidities, and larger magnitude surgery were associated with discharge to a facility, readmissions, and medical complications. However, the authors did evaluate newer machine learning statistical techniques to determine if this improved the accuracy of their prediction models. In this case, it turned out that old-fashioned logistic regression yielded slightly more accurate models. Additionally, logistic regression yields odds ratios that allow for quantitative comparison among risk factors to determine which ones are the most powerful. The machine learning techniques are designed to maximize the predictive accuracy of the model and do not provide as much information on the strength of association between each predictive variable and the outcome. Despite using newer statistical techniques, this paper shared the same limitations of all studies using large administrative databases. For one, the databases frequently do not include important outcomes such as reoperation, number of levels included in the fusion, and patient reported outcomes. In this case, looking at predictors of reoperation and being able to evaluate the effect of longer fusions would have been helpful. Additionally, miscoding can diminish the accuracy of the analysis. This paper raises the question of how these predictive models can and should be used. For more sophisticated patients, being able to accurately predict outcomes and complications based on their individual characteristics might help them in the shared decision making process. The other, more controversial use of these models is to predict the cost of the surgical episode, which a hospital system might use in deciding whether to offer to perform surgery in a bundled payment model. If the predictive model indicated that the cost of the episode exceeded the bundled payment, the hospital might decide against performing the case. Most physicians would consider this unethical. As new payment models that shift risk to hospitals and physicians are developed, we must remain aware of the perverse incentives the new systems can create.
Please read Dr. Jain's paper on this topic in the August 15 issue. Does this change how you view the use of machine learning techniques to create predictive models using big data?
Adam Pearson, MD, MS
Associate Web Editor