The rising costs of health care delivery pose significant concerns to system viability; thus, improving outcomes while restricting costs is a primary concern of reform efforts around the world. In spinal care delivery, the increased cost and frequency of spine-related procedures has also led to a shift toward delivery of value-based care.1 Health care decision makers are reluctant to support widespread use of spine care interventions without being assured that these investments will result in valuable clinical and economic returns for their organizations.1 Systematic evaluations help decision makers to make trade-offs in a period of financial constraints and when there is a need for optimal resource allocation.
For the majority of spinal interventions, well-designed prospective, randomized, pragmatic cost-effectiveness studies that address the specific decision-in-need are lacking. Decision analytic modeling allows for the estimation of cost-effectiveness based on what data may be available to date. Given the rising demands for proven value in spine care, the use of decision analytic modeling is rapidly expanding by clinicians and policy makers. To better empower stakeholders in spine care to more critically appraise and appropriately utilize the outputs of economic models of cost-effectiveness, we set out to (1) review the common tenets, strengths, and weaknesses of decision modeling for health economic assessment and (2) illustrate the applications of decision modeling in the spine intervention literature to date.
APPROACHES FOR HEALTH ECONOMIC MODELING
Health economic models help identify, measure, and compare relevant costs and health benefits of health care interventions, providing a tool for evaluating the economic impact of alternate therapies. They typically bring together knowledge from a variety of sources (e.g., clinical trials, databases, unit costs, etc.) when adequate experimental and/or long-term data are not available. “Modeling” occurs when data in their raw form are adjusted, combined, or transformed and represented in a different way. A description of the 10 main components of an economic modeling evaluation based on this examination of economic evaluations is presented in Table 1.
Event probabilities, resource utilization, costs, and patient outcomes must be incorporated into a model to assess the value of the treatment relative to its appropriate comparators. Each decision for population of the model along the way is made by weighing various trade-offs for maximizing the integrity of the analysis. For example, although it is true that it is important to select an appropriate time horizon to ensure that all relevant economic and clinical consequences are adequately captured, available study data are typically collected for a shorter duration of time. In modeling or estimating what will occur beyond the study period's follow-up time frame, several considerations must be made including how missing data will be handled (e.g., the validity of extrapolating results beyond the period of primary data collection).
Economic modeling of longer time horizons is often accomplished using Markov (state-transition) techniques based on cohorts of patients. Markov models assume that a patient is always in one of a finite number of discrete health states, called Markov states. The assumptions that the “states” be discrete and mutually exclusive sometimes lead to cumbersome models due to the need to incorporate prior medical states and comorbidities. As an alternate to Markov models, some long-term economic analyses that involve continuous risk over time are using event-based, individual patient models (microsimulation) that can represent diverse patient characteristics and variation in patient care.
The late health economist Bernie O'Brien described 2 general approaches for collecting and analyzing health economic data: “Frankenstein's Monster vs. Vampire of Trials.”2 “Frankenstein's Monster” pulls together many needed pieces of information from multiple independent sources and then stitches them together into a (hopefully) cohesive whole, enabling the user to track, evaluate, and quantify the effects of various pathways of care.2 Disparate empirical data are essentially integrated and transformed into connected mathematical relationships in decision analysis. Relationships among variables are thereby quantified, providing a powerful analytical framework and tool to optimize decision making, possibly assisting with problems seen in prospective or retrospective primary research such as nonrandomization, confounding, lack of blinding, lack of external validity, poor generalizability, a lack of or inadequate comparators, inadequate duration of data collection, and inadequate measurement of outcomes. The concern with “Frankenstein's Monster,” of course, is that there is often a high degree of uncertainty about data quality, clinical validity of assumptions, and parameter estimates, and the analyst can only hope that the monster will behave in a predictable fashion.2
In the alternate approach, “Vampire of Trials,” the health economist is more akin to Count Dracula—a vampire that feeds heavily off the clinical trial2 (or database). The main source of problems for this approach is that the trial/database has likely been designed to answer primary questions of efficacy/effectiveness/safety and not likely comparative cost-effectiveness. In the case of clinical trials, the high degree of internal validity comes at the price of reduced external validity because study populations, protocols, and circumstances may not be relevant to the “real-world” or diverse populations.2 In the case of database analyses, the data may provide a more “real-world” representation of outcomes, costs, and utilization; however, adequately adjusting for covariates and making fair comparisons among interventions is difficult. Furthermore, both clinical trials and databases often do not contain all of the pertinent outcomes that decision makers should consider when making resource allocation decisions. For these reasons, it has been argued that even if economic questions can be addressed through the collection of primary data, there likely will remain some need for modeling to adjust or project data to address policy-relevant economic questions.2
APPRAISING THE QUALITY OF HEALTH ECONOMIC STUDIES
Given the complexity and potential convolution of health economic studies, there have been many efforts to systematically evaluate and ensure their quality. In the published literature, there are 7 checklists available for appraising economic evaluations.3–9 Five were developed by researchers in countries other than the United States,4–6,8,9 1 designed for pediatric economic evaluations only,6 and 2 for economic data collected alongside clinical trials (“piggyback” economic evaluations).3,8 Four of the checklists have been formally validated both internally by the developers5–8 and externally by other researchers.10–14 The checklists are conceptual in nature and designed for health economic researchers who can interpret the nuances of economic evaluation. To varying extents, they either implicitly or explicitly require ranking of a study's quality relative to other health economic publications. Hence, they may not be as useful to clinicians or researchers who do not regularly conduct health economic evaluations. More importantly, these quality checks often appraise for accepted parameters of modeling and may be poorly suited to gauge the validity of clinical assumptions embedded in the model, sometimes detracting from the core issues that typically constitute a weak economic model. A proper, integrated, clinical and economic critical appraisal is necessary in the evaluation of the strength of evidence provided by a modeling evaluation.
We propose that users of economic evaluations, specifically clinical users of spine intervention economic evaluations, might consider appraising modeling studies in 2 separate stages. In the first stage of evaluating a model, the goal for the user or the reader is to determine whether the model proposed is even plausible or clinically important. Questions to consider are who sponsored the study and what bias does this introduce? Is the mix of patient populations used relevant or even combineable? Why was the analysis conducted? What are the comparators and are they adequate or even clinically relevant? What clinical and economic outcomes are they reporting and do they adequately capture the implications of the intervention? The second important component of the first stage is that users need to determine the source and the strength of the clinical data purporting differences among comparators. They should ask: What is the level of evidence of the clinical data? What clinical assumptions have been made? Is confounding adequately addressed? Do the clinical data match the time horizon for the model? Do you believe the data are strong enough to support this model? In essence, health economic models are built on and are only as strong as the clinical data supporting them. If this foundation and the premise of the model are inadequate, the user need not proceed further. Adding complexity and sophisticated “bells and whistles” will not help the analysis; essentially—garbage in, garbage out. It cannot be emphasized enough that the quality of health economic studies is mostly determined by the underlying clinical data upon which they are built.
If, however, the user does find the modeling study to be a potentially valuable analysis, they can then proceed with the second stage of critical appraisal where they can explore the finer details of the model and more carefully examine the findings, further assessing validity and quality and determining to what extent they find the evaluation of use to them. This stage is where they should consider things such as the appropriateness of the cost inputs, the source of utilities, whether discounting was done, whether any more minor outcomes were missed, whether results were presented in a disaggregated fashion so that absolute differences were transparent, whether ratios were calculated and assessed correctly, whether uncertainty was addressed adequately, whether data were reported accurately, and whether conclusions drawn are reasonable based on the data.
The health economic quality checklists may be helpful at this point to systematically explore the details of an evaluation and to assess its completeness in presenting the implications of alternate decisions. The British Medical Journal5 checklist and the Quality of Health Economic Studies scale7 are suggested because they (1) are mainly suitable for evaluating modeling studies as opposed to other types of health economic evaluations, (2) are applicable to a broad population of patients, and (3) have proven validity and reliability.
DECISION MODELS OF THE COST-EFFECTIVENESS OF SPINE THERAPIES
Illustrative Literature Review
Studies from an array of surgical spine treatments, study designs, modeling methodologies, and patient populations were chosen to help illustrate the range and heterogeneity of economic evaluations in spine care (Table 2). A formal systematic review process was not intended or followed to allow for a broader representation of studies, which is more typical of an illustrative or narrative review. Methodological considerations for the studies including funding source, country of analysis, type of economic evaluation (e.g., cost-effectiveness, cost-utility, etc.), perspective (e.g., payer, societal, etc.), type of model (e.g., Markov, decision analysis, etc.), time horizon (e.g., 1-yr, 10-yr, lifetime, etc.), and clinical and economic outcomes evaluated (e.g., cost-/quality-adjusted life-year [QALY]-gained, cost/death averted, etc.) were abstracted.
As expected on the basis of our intent to survey the spine economic modeling literature, there is substantial heterogeneity across the 20 studies summarized in Table 2 with respect to study design, models used, reporting, and general quality.15–34 Clinical data for many models are derived from retrospective studies. Some studies provide only limited detail regarding model inputs and rationale for model approaches. Many do not perform sensitivity analyses around their assumptions or model inputs to determine the robustness of their models and the impact of various factors on the final economic outcomes. Most studies provide only limited discussion of limitations and biases and their potential influence on results. Consideration of such factors is important for putting results of economic models in context.
Of the 20 decision analysis models on cost-effectiveness of spine interventions, a variety of clinical and economic assumptions were made regarding either the health benefit or the cost of treatment (Table 2).15–34 Although many of these clinical evidence inputs are well-referenced and transparently discussed, some are less obvious and have more questionable clinical validity. By far, the most frequently assumed clinical parameter is the benefit or effectiveness of care or health state utility (QALY-gain). Often, to model and estimate an incremental cost-effectiveness ratio or cost per QALY gained, the average utility for each of the Markov modeled state is assumed or estimated. For example, patients were modeled to improve, worsen, or die. In many cases, no studies existed that measure the average QALY of patients with lumbar stenosis that “worsened” to any extent with a treatment. Hence, the QALY gain assigned in the model to “worsened” was derived from differing patient populations or differing treatment studies measuring QALY for severe low back states or nonspecific low back pain populations. Another common assumption is the use of pre-operative health status as the QALY gain comparator to define the relative benefit of the intervention versus alternative. This assumes that continued treatment with the comparison treatment would provide zero benefit or harm. It also assumes that the cost of the alternative treatment is zero. Hence, using single cohort studies to model cost-effectiveness of competing options assumes many non–evidence-based costs/outcomes. Finally, many modeling studies in search of evidence to make clinical assumption on the odds of a treatment improving or worsening health states will have to rely on reported evidence from different spine diseases. “Low back pain,” while really a symptom, gets lumped together with structural-specific diagnosis-based studies of “lumbar stenosis” or “lumbar disc herniation.” Outcomes of patients with nonspecific back pain cannot be generalized to patients with specific lumbar structural pathologies. Lumbar spondylosis as a search term will generate cost and benefit evidence for a multitude of very different low back diseases. Hence, the benefits reported with fusion for nonspecific back pain should not be used to model likelihood of benefit from fusion for spondylolisthesis.
As is the case with clinical research, all options for collecting health economic or value data are not without their limitations and flaws. Health economic models are sometimes viewed cautiously because of them being complex or not reflective of actual clinical practice. It is our view that there are 2 key factors required for a satisfactory and meaningful health economic model: (1) convincing clinical evidence for differences between or among comparators and (2) transparency of methods so that the analysis can be appraised and ideally duplicated by other investigators. Duplicating an economic model would enable readers to customize the model to their particular environments or to enter other plausible model input values and conduct their own sensitivity analyses.
In clinical research, no single study captures all of the data that are required to make the best care decision for an individual patient. Inferences are based on linking together findings from an array of studies that deliver a preponderance of evidence for a clinical choice (or lack thereof). Similarly, no single health economic study provides all data needed for including economic considerations along with clinical ones. Hence, a fleet of studies, including those based on “Frankenstein” approaches and “vampire” approaches, is needed along with ethical considerations to make appropriate decisions about individual patient care and care for populations of patients. The use of techniques such as microsimulation to integrate the various types of data from various types of studies in the context of health reform is enabling rapid advances that are shaping a field in transition at this time.
In the rapidly evolving era of value-driven health care reform, health economic models make decision analysis explicit and attach costs to the outcomes of care, such that the economic implications of decisions can be simultaneously considered alongside the clinical outcomes. This can be useful to help decision makers understand trade-offs in a period of financial constraints and when there is a need for optimal resource allocation. Because comprehensive randomized, prospective cost-utility trials on the long-term value of spine treatments are lacking for the majority of treatments being critically examined today, health economic models may be helpful to overcome many clinical evidence gaps through modeling and clinical assumptions. The outputs of these health economic models must be critically appraised for both clinical validity and economic quality before altering health care policy, payment strategies, or patient care decisions. Spine care providers, as partners with their health economic colleagues, have unique clinical expertise and perspectives that are critical to interpret the strengths and weaknesses of health economic models and the extent to which they should alter health care reform policies.
- Given the rising demands for proven value in spine care, the use of decision analytic modeling is rapidly increasing by spine clinicians and policy makers.
- Models make decision analysis explicit and attach costs to the outcomes of care such that the economic implications of decisions can be simultaneously considered alongside the clinical outcomes.
- Health economic models are built on, and are only as strong as, the clinical data supporting them. They must be critically appraised for both clinical validity and economic quality before altering health care policy, payment strategies, or patient care decisions.
The authors are indebted to Nancy Holmes and Chi Lam for their administrative assistance and to Katie Moran for providing detailed study abstraction.
1. Rihn JA, Currier BL, Phillips FM, et al. Defining the value
of spine care
. J Am Acad Orthop Surg 2013;21:419–26.
2. O'Brien B. Economic evaluation
of pharmaceuticals. Frankenstein's monster or vampire of trials? Med Care 1996;34:DS99–108.
3. Adams ME, McCall NT, Gray DT, et al. Economic analysis in randomised control trials. Med Care 1992;30:231–8.
4. Sacristán JA, Soto J, Galende I. Evaluation of pharmacoeconomic studies: utilization of a checklist. Ann Pharmacother 1993;27:1126–33.
5. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation
Working Party. BMJ 1996;313:275–83.
6. Ungar WJ, Santos MT. The Pediatric Quality
Appraisal Questionnaire: an instrument for evaluation of the pediatric health economics
7. Chiou CF, Hay JW, Wallace JF, et al. Development and validation of a grading system for the quality
studies. Med Care 2003;41:32–44.
8. Evers S, Goossens M, de Vet H, et al. Criteria list for assessment of methodological quality
of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care 2005;21:240–5.
9. Inotai A, Pékli M, Jóna G, et al. Attempt to increase the transparency of fourth hurdle implementation in Central-Eastern European middle income countries: publication of the critical appraisal
methodology. BMC Health Serv Res 2012;12:332.
10. Ofman JJ, Sullivan SD, Neumann PJ, et al. Examining the value
of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm 2003;9:53–61.
11. Au F, Prahardhi S, Shiell A. Reliability of two instruments for critical assessment of economic evaluations. Value
12. Gerkens S, Crott R, Cleemput I, et al. Comparison of three instruments assessing the quality
of economic evaluations: a practical exercise on economic evaluations of the surgical treatment of obesity. Int J Technol Assess Health Care 2008;24:318–25.
13. Gerard K, Seymour J, Smoker I. A tool to improve quality
of reporting published economic analyses. Int J Technol Assess Health Care 2000;16:100–10.
14. Gonzalez-Perez JG. Developing a scoring system to quality
assess economic evaluations. Eur J Health Econ 2002;3:131–6.
15. Ackerman SJ, Polly DW Jr, Knight T, et al. Comparison of the costs of nonoperative care to minimally invasive surgery for sacroiliac joint disruption and degenerative sacroiliitis in a United States Medicare population: potential economic implications of a new minimally-invasive technology. Clinicoecon Outcomes Res 2013;5:575–87.
16. Angevine PD, Zivin JG, McCormick PC. Cost-effectiveness
of single-level anterior cervical discectomy and fusion for cervical spondylosis. Spine 2005;30:1989–97.
17. Burnett MG, Stein SC, Bartels RH. Cost-effectiveness
of current treatment strategies for lumbar spinal stenosis: nonsurgical care, laminectomy, and X-STOP. J Neurosurg Spine 2010;13:39–46.
18. Carreon LY, Glassman SD, Djurasovic M, et al. RhBMP-2 versus iliac crest bone graft for lumbar spine fusion in patients over 60 years of age: a cost
-utility study. Spine 2009;34:238–43.
19. Carreon LY, Anderson PA, Traynelis VC, et al. Cost-effectiveness
of single-level anterior cervical discectomy and fusion five years after surgery. Spine 2013;38:471–5.
20. Fehlings MG, Jha NK, Hewson SM, et al. Is surgery for cervical spondylotic myelopathy cost
-effective? A cost
-utility analysis based on data from the AOSpine North America prospective CSM study. J Neurosurg Spine 2012;17:89–93.
21. Furlan JC, Chan KK, Sandoval GA, et al. The combined use of surgery and radiotherapy to treat patients with epidural cord compression due to metastatic disease: a cost
-utility analysis. Neuro Oncol 2012;14:631–40.
22. Garrison KR, Donell S, Ryder J, et al. Clinical effectiveness and cost-effectiveness
of bone morphogenetic proteins in the non-healing of fractures and spinal fusion: a systematic review. Health Technol Assess 2007;11:1–150.
23. Kim S, Mortaz Hedjri S, et al. Cost
-utility of lumbar decompression with or without fusion for patients with symptomatic degenerative lumbar spondylolisthesis. Spine J 2012;12:44–54.
24. Koenig L, Dall TM, Gu Q, et al. How does accounting for worker productivity affect the measured cost-effectiveness
of lumbar discectomy? Clin Orthop Relat Res 2014;472:1069–79.
25. Kuntz KM, Snider RK, Weinstein JN, et al. Cost-effectiveness
of fusion with and without instrumentation for patients with degenerative spondylolisthesis and spinal stenosis. Spine 2000;25:1132–9.
26. Malter AD, Larson EB, Urban N, et al. Cost-effectiveness
of lumbar discectomy for the treatment of herniated intervertebral disc. Spine 1996;21:1048–54.
27. O'Neill KR, Wilson RJ, Burns KM, et al. Anterior cervical discectomy and fusion for adjacent-segment disease: clinical outcomes and cost
-utility of surgical intervention [published online ahead of print November 5, 2013]. J Spinal Disord Tech. doi:10.1097/BSD.0b013e31828ffc54.
28. Papatheofanis FJ, Williams E, Chang SD. Cost
-utility analysis of the cyberknife system for metastatic spinal tumors. Neurosurgery 2009;64:A73–83.
29. Qureshi SA, McAnany S, Goz V, et al. Cost-effectiveness
analysis: comparing single-level cervical disc replacement and single-level anterior cervical discectomy and fusion: clinical article. J Neurosurg Spine 2013;19:546–54.
30. Räsänen P, Ohman J, Sintonen H, et al. Cost
-utility analysis of routine neurosurgical spinal surgery. J Neurosurg Spine 2006;5:204–9.
31. Udeh BL, Costandi S, Dalton JE, et al. The 2-year cost-effectiveness
of 3 options to treat lumbar spinal stenosis patients [published online ahead of print January 3, 2014]. Pain Pract. doi: 10.1111/papr.12160.
32. van den Akker ME, Arts MP, van den Hout WB, et al. Tubular diskectomy vs conventional microdiskectomy for the treatment of lumbar disk-related sciatica: cost
utility analysis alongside a double-blind randomized controlled trial. Neurosurgery 2011;69:829–35.
33. Virk S, Sandhu HS, Khan SN. Cost
effectiveness analysis of graft options in spinal fusion surgery using a Markov model. J Spinal Disord Tech 2012;25:E204–10.
34. Washington State Health Care Authority. Cervical spine fusion for degenerative disc disease. Available at: http://www.hca.wa.gov/hta/pages/degenerative_disc_disease.aspx
. Accessed April 18, 2014.
35. Spiegel BM, Targownik LE, Kanwal F, et al. The quality
of published health economic analyses in digestive diseases: a systematic review and quantitative appraisal. Gastroenterology 2004;127:403–11.