Any reader of the spine literature has come across an increasing number of studies based on large administrative databases recently. Due to the popularity of these studies that were much less common in the spine literature in the past, Drs. Yoshihara and Yoneoka published a journal club article discussing the strengths and limitations of this study design. Before considering the pros and cons of this research approach, it is worth considering why so many administrative database studies are being performed. For one, computing and statistical methods continue to improve, and “big data” are being used more frequently in many disciplines, medicine included. These large databases capture huge numbers of patients, orders of magnitude more than can be followed in a traditional prospective, clinical trial. This allows for the analysis of relatively rare events and the study of subgroups that could never be evaluated in clinical trials that would be markedly underpowered for such analyses. Additionally, clinical trials are becoming increasingly more difficult to perform due to regulatory issues, demands for more rigorous study designs, and a lack of grant funding for such research, especially for spine-related topics. Finally, researchers have discovered that these studies are relatively easy to perform once a database has been obtained and an analyst has learned their way around it. While querying the databases is relatively easy, formulating questions that can actually be answered with administrative databases and using appropriate analytic methodology can be quite difficult.
Administrative databases are very useful for studying treatment trends over time, evaluating regional variation, and calculating costs to payers. They also allow for the study of rare events—like death following spine surgery—and can have enough patients to evaluate the risk factors for these uncommon complications. They also allow for comparison of the rates of certain well-defined complications, such as readmission, reoperation, and death, among different treatment techniques. While these are all worthy research pursuits, one can argue that the most important outcome following spine surgery is patient reported quality of life. These outcome measures are not included in administrative databases. This is the major limitation of spine surgery database studies, and it precludes comparing efficacy among different treatments. Another limitation of many commonly used databases in the spine literature is that they include only inpatient (i.e. the National Inpatient Sample) data from a single hospital admission or are limited to a 30 day window following the date of surgery (i.e. the National Surgical Quality Improvement Program). Given that many of the most concerning complications such as infection, readmission, and reoperation frequently occur outside these windows, studies analyzing complication rates with these databases likely underestimate the true rate of complications. Researchers need to recognize these limitations while designing their studies, and they should limit their questions to those that can reasonably be answered with the available data. Additionally, they need to take into account that patients selected for different procedures were likely different in ways not captured in billing data (i.e. symptoms and physical exam findings, radiographic characteristics, work status, etc.), and these unrecorded differences cannot be controlled for statistically. Journal reviewers and editors also need to be more stringent in evaluating these articles and should not accept an article for publication simply because it included 100,000 patients. Like most powerful tools, large administrative databases have the ability to be of great benefit, yet, if misused, can cause more harm than good.
Please read Dr. Yoshihara’s article on this topic in the July 15 issue. Does this article change how you view large administrative database studies? Let us know by leaving a comment on The Spine Blog.
Adam Pearson, MD, MS
Associate Web Editor