Secondary Logo

Journal Logo

The Spine Blog

Friday, April 21, 2017

Are large database studies reliable?

Large administrative and clinical databases have been widely used in the recent spine literature in efforts to answer clinically important questions. The biggest advantage of these databases is in the sheer volume of patients available for analysis, frequently numbering in the tens to hundreds of thousands. The disadvantages of these datasets as compared to those generated prospectively to answer a research question are many, namely that the large databases can include inaccurate claims data, are frequently limited to very short follow-up, and generally include no patient reported outcomes. The large databases are most useful to analyze large-scale epidemiologic trends (i.e. rates of certain surgeries) and geographic variation. They are almost worthless in evaluating long-term patient outcomes. Their utility in studying complications is debatable, though they do have the advantage of including huge numbers of patients that allows for the study of rare complications. In order to better understand the reliability of large datasets to study complications following ACDF, Dr. Cho and colleagues from New York performed identical analyses using the National Surgical Quality Improvement Program (NSQIP) and National Inpatient Sample (NIS) databases. The NSQIP cohort included over 10,000 patients, while the NIS was a 20% stratified sample that represented over 100,000 cases. There were significant baseline differences between the two cohorts, with the NSQIP including more patients over 65 years old, more blacks, fewer Hispanics, and a generally higher rate of comorbidities (other than renal disease). The overall complication rate was slightly higher (7% vs. 6%) in the NIS, though the rates of specific complications were very low (i.e. 1/10,000 mortality, even lower for wound complications). The authors performed a multivariate regression analysis with both datasets in order to identify independent risk factors for mortality, cardiac complications, and sepsis. In general, each cohort had different risk factors for each complication, with the only examples of consistent risk factors being age for all three complications, male gender for mortality, diabetes for cardiac complications, and bleeding disorder for sepsis. A striking difference was how recent weight loss was a very strong predictor of all three complications (odds ratio of 50 for sepsis) in the NIS, while it was not a significant independent predictor of any complication in NSQIP.

This paper does a nice job demonstrating the advantages and limitations of large database analyses. Complications following ACDF are rare, with mortality occurring at a rate of approximately 1/10,000 according to this study. Such rare events cannot be studied with traditional prospective clinical studies, so the databases allow for a glimpse into these events. However, in this case, events such as death (occurring in 11/10,000 NSQIP patients) and wound complications (3/10,000) were so rare that even this study was underpowered to analyze them well. Given the very low numbers of complications and different definitions of risk factors and complications across the two databases, it is not surprising that the multivariate analyses yielded different results. The authors point out the biggest limitation of the NIS, which is that is includes only data for the duration of a single hospital admissions. Given that the average admission for ACDF is under two days, this database most likely misses the vast majority of complications that occur. The NSQIP is somewhat better and likely captures many more complications that present within 30 days of discharge, but it misses complications such as pseudarthrosis, hardware failure, and adjacent segment degeneration. While these databases can identify medical complications and some infections that occur in the short term following surgery, most of the important events occur well beyond 30 days after spine surgery. The inability of these studies to capture long-term reoperation rates and patient reported outcomes markedly limits their utility beyond describing large scale epidemiologic trends.

 

Please read Dr. Cho’s article on this topic in the April 15 issue. Does this change how you view large database studies in the spine literature? Let us know by leaving a comment on The Spine Blog.

Adam Pearson, MD, MS

Associate Web Editor