Although economic evaluation has become more popular among medical professionals, its use in surgery is still lacking. A review over a 6-year period of the number of studies using utility measurements (used in cost-utility analyses) revealed 649 publications in the medical literature compared with just 57 in the surgical literature.1 Chew et al. feel that “the lack of formal economic evaluation in the surgical literature likely stems from the complexity of the subject and (with few exceptions) the lack of surgeon training in clinical research methodology.”1 Within the plastic surgery literature, the use of economic analysis is even more dismal. A review of the plastic surgery literature over a 16-year period revealed that only 0.6 percent of outcomes studies involved economic analysis.2 Kerrigan et al. note that, “Health-related quality of life is of particular importance in plastic surgery, whose interventions significantly impact quality of life yet may have little impact on length of life.”3
This article, part of a tutorial series on evidence-based medicine, focuses on the fundamental principles of conducting a surgical economic analysis. Although these principles can be applied to any field, we use examples from the surgical literature. We frequently reference recommendations made by the Panel on Cost-Effectiveness in Health and Medicine. The Panel is a nonfederal group of 13 scientists and scholars convened in 1993 by the U.S. Public Health Service to develop recommendations to improve the quality and comparability of economic analysis studies.4
Why should one conduct an economic analysis? It is “a method for evaluating the health outcomes and resource costs of health interventions. Its central function is to show the relative value of alternative interventions for improving health.”4 As in all evidence-based medicine, there should be some uncertainty about the best clinical strategy for a patient in a given health state.5 There are four different types of economic evaluation: cost-minimization analysis, cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis. The definitions, metrics reported, advantages, and disadvantages of these four types of studies are listed in Table 1. Because of the advantages that cost-utility studies have in being able to compare treatments with different outcomes, we will focus on how to conduct a cost-utility analysis.
PERSPECTIVE AND ASSUMPTIONS
Costs and outcomes can be evaluated from a number of different perspectives: patient, hospital, third-party payer, or society. However, it is impossible to assess the economic impact of various studies if the studies use different perspectives. Studies based on different perspectives are not comparable. The Panel recommends a societal perspective. By conducting a study using the societal perspective, “the analyst considers everyone affected by the intervention, and all health effects and costs that flow from it are counted, regardless of who would experience them. Health effects include both benefits and harms, even when these occur in people who are not the intended recipients of the intervention.”4
The reference (or base) case is defined by a standard set of methods and assumptions. It serves as a benchmark whereby others can evaluate the quality of a study and determine whether its results can be compared across studies.4 For example, a study should describe the mean or standard values for probabilities, utilities, costs, and discount rates that will be used for the reference case. As described later, these values can be varied in the sensitivity analysis to obtain best- and worst-case scenarios.
There are several sources for cost data: hospital, Medicare, and private sources. It is important to remember, especially if using hospital data, the difference between costs and charges. Costs are the actual expenditures incurred when providing a service and charges are list prices. As expected, charges are usually inflated and do not reflect the actual cost. In most institutions, obtaining cost data can be difficult because a hospital will not want to divulge the cost structure for fear of losing its competitive edge and permitting insurance reimbursements to decrease if these cost structures are made public. Costs, not charges, should be measured and they should be the costs in dollars of a fixed year. If cost data are from different years, inflation must be taken into consideration by either inflating the data from an earlier year than the fixed year or deflating the data from a later year.6 An inflation calculator is available from the Bureau of Labor Statistics web site.7 If costs are not readily available, as is often the case, some studies use a cost-to-charge ratio to calculate costs. The general formula for a cost-to-charge ratio is: charge for hospitalization × (amount reimbursed by third-party payer/amount charged to third-party payer).8 For example, in a cost-utility analysis of breast reconstruction,9 International Classification of Diseases, Ninth Revision codes for different procedures were determined and these were used to obtain mean hospitalization charges from the Healthcare Cost and Utilization Project data set. The Medicare Provider Analysis and Review database was then used to determine the amount of reimbursement for each diagnosis-related group code. A cost-to-charge ratio was calculated by dividing the amount of the diagnosis-related group reimbursement by the amount of International Classification of Diseases, Ninth Revision charge. This number was then multiplied by the mean charge from the Healthcare Cost and Utilization Project to calculate a cost.
Narrower perspectives, such as third-party payer, have been used in other surgical economic analyses.10,11 However, it must be kept in mind that the third party usually does not pay the full amount billed,12 and the results will not be generalizable to studies with a societal perspective.
According to the Panel, the major categories that should be included as costs are, “costs of health care services; costs of patient time expended for the intervention; costs associated with caregiving (paid or unpaid); other costs associated with illness, such as child care and travel expenses; economic costs borne by employers, other employees, and the rest of society, including so-called friction costs associated with absenteeism and employee turnover; and costs associated with nonhealth impacts of the intervention, such as on the educational system, the criminal justice system, or the environment.”6
There is some controversy as to what should be included in the numerator (as cost) or in the denominator (as effectiveness). It is important not to double-count by including, for example, costs of lost life-years when these are already included in the denominator as lost life-years. However, to have a true societal perspective, it is important to incorporate into quality-adjusted life-years anything that has been excluded as a cost. Thus, if lost wages from time off work will be included as costs, one may wish to include a statement in the utility assessment such as, “In answering this question, please assume that your financial circumstances would not be affected by the illness, as might be the case if you did not have to pay anything for health care and you received disability payments that completely replaced any lost earnings.”13 Similar statements have been used in other economic analyses.14,15 Thus, the costs of time off work are not double-counted in the denominator by the respondent who may consider a loss of income during recovery. Respondents should be instructed to consider the loss of leisure activities. Lost wages should correspond to the age and gender composition of the target population.6 Such data can be obtained from the U.S. Bureau of Labor Statistics.16 For those individuals not in the labor force, average age- and gender-specific wages should still be used.6
How long should costs be included? Again, there is some controversy. However, the Panel definitely recommends including (1) costs of related diseases in the original life span, (2) costs for intervention-related diseases that occur in added years of life, and (3) costs of an ongoing therapy throughout added years of life. Areas of debate abound, including costs of disease unrelated to the intervention and ensuing as a result of added years of life. The Panel concludes that these may be either included or excluded from the reference case. They do not recommend including future non–health care costs (such as food, clothing, and shelter).6
It is essential to know the possible outcomes of the condition under study. Such outcomes may include the probability of surgical complications and what follow-up care, if any, is required from these complications. These data may come from published studies and/or your own clinical trial. If a meta-analysis or a systematic review of the outcomes of interest has not been conducted, you should essentially conduct your own by performing a literature search, critically evaluating the literature, and averaging the results from several quality studies. The studies do not necessarily have to be randomized controlled trials, but they should be of high quality. The goal is to obtain a range of reasonable probability estimates for each outcome. Although the range may come from different studies, it may be based on the 95 percent confidence interval from a single study17 if that is the only high-quality study available. Tips on how to perform a systematic review or meta-analysis are published by Margaliot and Chung18 and Chung et al.19 You must also determine how long you will assess outcomes. For surgical studies, 1 year may be sufficient to assess all possible surgical complications.
Utilities can be viewed as the preferences that individuals or society may have for a particular health state.1 When measuring health states, a continuous scale of 0 to 1 is used in which 0 represents death and 1 represents perfect health. For example, in a study of breast hypertrophy, respondents assigned mild breast hypertrophy a utility of 0.93 and severe breast hypertrophy a utility of 0.70 using one type of utility assessment.20 If an intervention extends a patient's life by 1 year in a 0.70 utility state, and another intervention extends a patient's life by 2 years in a 0.35 utility state, the quality-adjusted life-years gained for the two interventions would be the same at 0.70.1 For permanent health states, quality-adjusted life-years are calculated by multiplying the utility of the health state by the patient's expected number of remaining healthy life years. For temporary health states, when a time frame is specified, quality-adjusted life-years are calculated as shown in Table 2.
The Panel recommends using community preferences. The “community” should be representatives of the general population that contains people with disabilities or chronic illnesses in proportion to the prevalence of the condition under study. Community preferences are different from patient preferences, which are values that people experiencing a particular condition will assign to their own health.4 However, it has been found that the general public underestimates the quality of life of certain conditions compared with patients with those conditions. For example, the general public estimated that the quality of life of chronic dialysis is 0.39, whereas dialysis patients rate the quality of life as 0.56.21 The general public seems to have difficulty assessing the quality of life associated with unfamiliar health conditions.22 One option, although time consuming, is to obtain both community and patient preferences. Any differences between the two groups can then be ascertained. Another option is to use members of the “general public” who are familiar with a particular health condition, such as physicians or patients with the same condition under study who have not received or been evaluated for the treatment under study; for example, rheumatoid arthritis patients waiting for a general check-up if a researcher wants to evaluate a particular type of rheumatoid surgery. The assessment of the utility of colostomy is another example of the importance of choosing the appropriate subjects for measuring preference. It was found that physicians with experience in the treatment of gastrointestinal malignancies provided similar utilities as patients treated for rectal cancer with the formation of a colostomy. In contrast, patients treated for rectal cancer without a colostomy gave the lowest utilities, even lower than a group of healthy volunteers and a group of university science students.23
Several generic utility measurements have been developed, such as the EuroQol,24 the Health Utilities Index,25 and the Quality of Well-Being Scale.26 The choice of a utility tool should reflect the health domains that are important for the particular health problem being investigated.27 Because of the generic nature of these utility assessments, they may not be sensitive enough to capture attributes of a particular disease.1 For example, the EuroQol asks individuals to rate how good or bad various health states are that are related to six domains: mobility, self-care, main activity (e.g., work, study, housework), social relationships, pain, and mood.24 If such health states do not relate to your condition of study, you may have to create your own utility assessment. Surgical studies that have created their own utility assessments include carpal tunnel surgery,28 scaphoid fracture,15 treatment of rheumatoid wrist,29 hand transplantation,30 open tibial fractures,14 and breast hypertrophy.3,20 There are various ways in which one can assess preferences: rating scale, time trade-off, and standard gamble. Descriptions, advantages, and disadvantages of each of these methods are listed in Table 3.31,32
To analyze the results, you should create a decision tree. Software such as TreeAge Pro (TreeAge Software, Inc., Williamstown, Mass.) is available for this purpose. Each node or fork in the tree represents a purposeful or random choice.33 Each branch must have advantages and disadvantages34 and the probabilities of all branches must add up to 1 for that node. Cost, duration, utility, and probability are entered into the decision tree for each outcome. The software then calculates a cost- and quality-adjusted life-year for outcome. In Figure 1, the clinical question is how to best treat a patient with a unilateral hand amputation. The two treatment choices are hand transplantation or no hand transplantation. From each of these treatment choices arises an outcome or a new choice. Utilities are assigned to the different choices (such as prosthesis versus no prosthesis) and probabilities are assigned to the chance events (such as the probability of surgical complication and subsequent graft loss after hand transplantation). Eventually, each branch reaches a terminus, calculated as quality-adjusted life-years in Figure 1.35 We previously illustrated how to calculate quality-adjusted life-years for a single event. At chance events, quality-adjusted life-years are calculated as a weighted average of the quality-adjusted life-years of all possible outcomes resulting from the chance event, and the weighting factors are the probabilities of each element occurring.36 For example, in a cost-utility analysis of hand transplantation,30 there are four possible outcomes of treatment using unilateral hand transplantation: minor immunosuppressive complication, major immunosuppressive complication, graft failure, or death. The quality-adjusted life-years of unilateral hand transplantation are calculated as the total sum of each outcome's quality-adjusted life-years multiplied by the probability of that outcome occurring (Table 4).
The outcomes of an economic analysis are shown in Table 5. Strategies that are less costly and more effective should definitely be adopted. Likewise, strategies that are more costly and less effective should definitely be rejected. All other areas generally fall into “gray zones” that need to be further evaluated or are only weakly accepted or rejected.
As shown in Table 5, there are certain instances in which you will need to calculate an incremental cost-effectiveness ratio. The incremental cost-effectiveness ratio is the cost difference between the two therapies under study divided by the difference in effectiveness between the two therapies ([costA − costB]**[outcomeA − outcomeB]). The use of an incremental cost-effectiveness ratio will be increasingly important, as one of the current health care agendas is to compare the cost-effectiveness of interventions in a head-to-head comparison. For example, the incremental cost-effectiveness ratio for a cost-utility study comparing double hand transplantation versus double prostheses after double hand amputation is $318,961 per quality-adjusted life-year.30 Typically, incremental cost-effectiveness ratios of $50,000 per quality-adjusted life-year are considered an acceptable threshold for adopting a particular intervention and those $100,000 or more are considered unaffordable. Thus, for the hand amputation study, hand transplantation was considered to be more effective but exceeds the cost-effectiveness threshold for adoption as the standard therapy. This cutoff appears to have originated in 1982 U.S. dollars, and $50,000 was the approximate cost of dialysis for a patient in renal failure (renal dialysis is a federal entitlement to all U.S. citizens under Medicare).37 This incremental cost-effectiveness ratio is used in the United States and may not be applicable in other countries that vary in their willingness to pay for health and health care.38 Although some have proposed raising this threshold,37 there is no other “standard” at this time.
Discounting is the valuation of costs and consequences over time. Money spent or saved in the future should not be weighed as heavily as dollars spent or saved today.12 Discounting is unnecessary in studies with durations of less than 1 year.39 However, it is essential when interventions have costs occurring at differing times over the remaining life of the patient. This was shown in a cost-utility analysis14 of amputation versus salvage for open tibial fractures. Amputation involves the long-term expenses of ongoing prosthetics. Long-term costs are influenced more substantially by discounting and long-term expenses have more impact for younger patients. The Panel recommends using both 3 percent and 5 percent as discount rates. Before discounting, all costs should be adjusted for inflation.6 The discounting formula12 shown in Table 6 also gives an example of how to perform discounting. Future quality-adjusted life-years should be discounted in a similar manner, with the same rate used to discount cost.
A sensitivity analysis should also be performed to assess the degree of uncertainty with the result (similar to statistical hypothesis testing).36 Conclusions are not as strong if they change when subjected to the extreme values of the variables.33 One-way sensitivity analysis varies one variable at a time, usually over their entire range. In two- and three-way sensitivity analysis, two or three variables are varied at a time. Although there is no rule as to which variables should be evaluated in multiway analysis, those that appear to be important in one-way sensitivity analysis should be further studied in multiway analysis.36 The Panel recommends varying the discount rate from 0 to 7 percent in the sensitivity analysis.
More sophisticated analyses include Markov models and Monte Carlo simulation. Markov models are useful when there is a problem that involves a risk that is ongoing over time (such as the risk of mortality of a healthy or sick person) and when a given event may occur more than once (such as a hemorrhage after surgery). These scenarios are difficult to represent in a simple tree model.40 Unlike a regular decision tree, future states are independent of past states. A hypothetical cohort of subjects is entered into the model, and subjects encounter a number of different probabilities. The process is repeated hundreds or thousands of times until a distribution of cost or effectiveness values are reached.8 For example, in a study comparing implants versus transverse rectus abdominis musculocutaneous flap breast reconstruction, a Markov model was used because there is the possibility of both immediate and late surgical complications.9 Outcomes were modeled for both the implant and transverse rectus abdominis musculocutaneous flap for the remaining life expectancy of the average 48-year-old woman. A Markov cycle tree may be evaluated as a Monte Carlo simulation. Monte Carlo simulation uses a large number of trials to determine a distribution of survival values, including statistical measures such as variance and standard deviation of the expected utility.40 We recommend further reading40 and the assistance of a health economist to incorporate these sophisticated analyses.
Reference case results should include total costs and effectiveness, incremental costs and effectiveness, and incremental cost-effectiveness ratios. Both undiscounted and discounted incremental cost-effectiveness ratios should be reported. When reporting the results of your economic analysis, it is important to state all assumptions that were used. It is also important to keep in mind the generalizability of your results. Data from economic analyses are not often considered to be generalizable because of differences in the availability of health care resources, clinical practice patterns, and relative prices.41 Recommendations for designing, conducting, and reporting the results of an economic evaluation alongside a randomized trial and a checklist for assessing the generalizability of trial-based studies are available in articles by Ramsey et al.42 and Drummond et al.41 General reporting guidelines are also available in Siegel et al.43
Although economic analyses are not simple to conduct, a well-conducted one provides many rewards. Evidence for the adoption of a more effective treatment is one such reward. The fundamental principles offered in this article provide a basis for conducting a surgery cost-utility study. However, even the best economic analysis is limited by the quality of data available to calculate costs, outcome probabilities, and utilities. To compare outcomes of cost and effectiveness across economic analyses and to interpret the results of a single economic analysis, it is important that all economic analyses use consistent methodology and report the results in a similar manner. One should understand that the conclusion from an economic analysis project gives recommendations based on the societal perspective to design health policy strategies. However, a physician has the ethical duty and the professional right to consider the need of an individual patient when considering the best treatment decision.
This work was supported in part by a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) (to K.C.C.).
1.Chew RT, Sprague S, Thoma A. A systematic review of utility measurements in the surgical literature. J Am Coll Surg.
2.Davis Sears E, Burns PB, Chung KC. The outcomes of outcome studies in plastic surgery: A systematic review of 17 years of plastic surgery research. Plast Reconstr Surg.
3.Kerrigan CL, Collins ED, Kneeland TS, et al. Measuring health state preferences in women with breast hypertrophy. Plast Reconstr Surg.
4.Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. JAMA.
5.Detsky AS, Naglie G, Krahn MD, Naimark D, Redelmeier DA. Primer on medical decision analysis: Part 1. Getting started. Med Decis Making
6.Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on Cost-Effectiveness in Health and Medicine. JAMA.
8.Muennig P. Designing and Conducting Cost-Effectiveness Analyses in Medicine and Health Care.
San Francisco, Calif.: Jossey-Bass; 2002.
9.Preminger BA, Pusic AL, McCarthy CM, Verma N, Worku A, Cordeiro PG. How should quality-of-life data be incorporated into a cost analysis of breast reconstruction? A consideration of implant versus free TRAM flap procedures. Plast Reconstr Surg.
10.Kerrigan CL, Stanwix MG. Using evidence to minimize the cost of trigger finger care. J Hand Surg (Am.)
11.Thoma A, Khuthaila D, Rockwell G, Veltri K. Cost-utility analysis comparing free and pedicled TRAM flap for breast reconstruction. Microsurgery
12.Drummond M, O'Brien B, Stoddart G, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes.
New York: Oxford University Press; 1997.
13.Meltzer D, Johannesson M. Inconsistencies in the “societal perspective” on costs of the Panel on Cost-Effectiveness in Health and Medicine. Med Decis Making
14.Chung KC, Saddawi-Konefka D, Haase SC, Kaul G. A cost-utility analysis of amputation versus salvage for Gustilo type IIIB and IIIC open tibial fractures. Plast Reconstr Surg.
15.Davis EN, Chung KC, Kotsis SV, Lau FH, Vijan S. A cost/utility analysis of open reduction and internal fixation versus cast immobilization for acute nondisplaced mid-waist scaphoid fractures. Plast Reconstr Surg.
16.United States Department of Labor. Bureau of Labor Statistics. Table 2: Median usual weekly earnings of full-time wage and salary workers by age, race, Hispanic or Latino ethnicity, and sex, third quarter 2009 averages, not seasonally adjusted. Available at: http://www.bls.gov/news.release/wkyeng.t02.htm
. Accessed September 10, 2009.
17.Naglie G, Krahn MD, Naimark D, Redelmeier DA, Detsky AS. Primer on medical decision analysis: Part 3. Estimating probabilities and utilities. Med Decis Making
18.Margaliot Z, Chung KC. Systematic reviews: A primer for plastic surgery research. Plast Reconstr Surg.
19.Chung KC, Burns PB, Kim HM. A practical guide to meta-analysis. J Hand Surg (Am.)
20.Chang WT, Collins ED, Kerrigan CL. An Internet-based utility assessment of breast hypertrophy. Plast Reconstr Surg.
21.Sackett DL, Torrance GW. The utility of different health states as perceived by the general public. J Chronic Dis.
22.Ubel PA, Loewenstein G, Hershey J, et al. Do nonpatients underestimate the quality of life associated with chronic health conditions because of a focusing illusion? Med Decis Making
23.Boyd NF, Sutherland HJ, Heasman KZ, Tritchler DL, Cummings BJ. Whose utilities for decision analysis? Med Decis Making
24.The EuroQol Group. EuroQol: A new facility for the measurement of health-related quality of life. Health Policy
25.Feeny D, Furlong W, Torrance GW, et al. Multiattribute and single-attribute utility functions for the health utilities index mark 3 system. Med Care
26.Kaplan R, Bush J. Health-related quality of life measurement for evaluation research and policy analysis. Health Psychol.
27.Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine.
New York: Oxford University Press; 1996.
28.Chung KC, Walters MR, Greenfield ML, Chernew ME. Endoscopic versus open carpal tunnel release: A cost-effectiveness analysis. Plast Reconstr Surg.
29.Cavaliere CM, Chung KC. Total wrist arthroplasty and total wrist arthrodesis in rheumatoid arthritis: A decision analysis from the hand surgeons' perspective. J Hand Surg (Am.)
30.Chung KC, Oda T, Saddawi-Konefka D, Shauver M. An economic analysis of hand transplantation in the United States. Plast Reconstr Surg.
31.Patrick DL, Starks HE, Cain KC, Uhlmann RF, Pearlman RA. Measuring preferences for health states worse than death. Med Decis Making
32.Streiner D, Norman G. Health Measurement Scales: A Practical Guide to their Development and Use.
New York: Oxford University Press; 1989.
33.Mastracci TM, Thoma A, Farrokhyar F, Tandan VR, Cina CS. Users' guide to the surgical literature: How to use a decision analysis. Can J Surg.
34.Detsky AS, Naglie G, Krahn MD, Redelmeier DA, Naimark D. Primer on medical decision analysis: Part 2. Building a tree. Med Decis Making
35.Chen NC, Shauver MJ, Chung KC. A primer on use of decision analysis methodology in hand surgery. J Hand Surg (Am.)
36.Krahn MD, Naglie G, Naimark D, Redelmeier DA, Detsky AS. Primer on medical decision analysis: Part 4. Analyzing the model and interpreting the results. Med Decis Making
37.Ubel PA, Hirth RA, Chernew ME, Fendrick AM. What is the price of life and why doesn't it increase at the rate of inflation? Arch Intern Med.
38.O'Brien BJ, Heyland D, Richardson WS, Levine M, Drummond MF. Users' guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA.
39.Thoma A, Sprague S, Tandan V. Users' guide to the surgical literature: How to use an article on economic analysis. Can J Surg.
40.Sonnenberg FA, Beck JR. Markov models in medical decision making: A practical guide. Med Decis Making
41.Drummond M, Manca A, Sculpher M. Increasing the generalizability of economic evaluations: Recommendations for the design, analysis, and reporting of studies. Int J Technol Assess Health Care
42.Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: The ISPOR RCT-CEA Task Force report. Value Health
©2010American Society of Plastic Surgeons
43.Siegel JE, Weinstein MC, Russell LB, Gold MR. Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine. JAMA.