In the era of evidence-based medicine, decision analysis is an important tool for plastic surgeons to add to their armamentarium in both the research and clinical arena. There is increasing pressure by payers and patients to improve outcomes with limited resources. Some clinical questions may be inappropriate for outcome studies or may have conflicting outcomes that complicate decision making. In these situations, decision analysis can help formulate treatment plans for individuals or can be used to guide policy. Thus, plastic surgeons should become familiar with this research technique and understand its limitations.
The aim of this article is to familiarize plastic surgeons with the decision analysis technique through discussion of its application and essential components. We provide tools for interpretation of decision analysis in published studies and the application of data derived from this technique.
WHAT IS DECISION ANALYSIS?
Decision analysis is the translation of a complex clinical scenario and its component parts into a manageable model. Competing management strategies are quantitatively compared despite the presence of clinical uncertainty. All possible therapeutic options and potential outcomes associated with each strategy are identified and assigned a value. The balance between risk and benefit of treatment strategies is quantified, integrating patient preferences and research outcomes. An optimal decision is identified by choosing the therapeutic strategy with the greatest overall outcome. Economic analysis is a subset of decision analysis in which management strategies and their associated components are evaluated based on cost and outcome. Economic analysis aims to maximize health in the setting of limited resources.
WHAT CLINICAL QUESTIONS ARE SUITED FOR DECISION ANALYSIS?
Before embarking on decision analysis, the first step is to decide whether decision analysis is suitable for the clinical question. Decision analysis is best applied in situations in which there is no clear-cut solution. The question at hand must require the comparison of two or more strategies. If one option has more benefit and less risk, a decision analysis is not needed. Usually, one option has clear advantages, but it also has associated drawbacks that prevent it from being clearly ideal in all respects.
Many clinical scenarios in plastic surgery have uncertainty with regard to the best course of action for patients. For example, in lower-extremity trauma, the reconstructive surgeon may be deciding whether a patient will benefit from reconstruction that will require more recovery time and higher risk of complications or whether the patient should have a below the knee amputation, which will allow earlier ambulation and less risk of complications.1–3 The decision model that is designed must be comprehensive enough to capture important components but simple enough to be understood by the intended audience.
WHAT ESSENTIAL COMPONENTS ARE NEEDED TO PERFORM DECISION ANALYSIS?
Figure 1 illustrates a checklist of essential components of decision analysis that may be used by investigators who are learning the technique. First, the clinical question is explicitly designed, including the specific patient population to which the question is targeted. Second, a model is created to incorporate possible treatment strategies and pertinent associated outcomes. Third, probabilities and outcome values are assigned to the events in the model. Finally, the best strategy is identified, and sensitivity analysis is performed to test the robustness of conclusions drawn from the decision analysis.
Step 1: Define the Question
The first step in performing a decision analysis is to define the clinical question. In doing so, the patient population involved should be clearly identified. The question is evaluated objectively to identify all alternative strategies to address the clinical scenario, keeping in mind the consequences of each treatment and their influences on patient outcome. Let us consider a hypothetical clinical question. Imagine you are consulting a patient for breast reduction surgery, and your patient inquires about the necessity of having suction drains postoperatively. In your practice, you have a partner who rarely uses drains; another partner uses drains, but removes the drain on postoperative day one; and you routinely use drains and leave them in place until the output has reached less than 30 milliliters per day. The patient is reluctant to have drains and would like to discuss the evidence behind their use. This is a good question for decision analysis. Drains may be cumbersome to patients, but it may be worth it if they decrease the incidence of postoperative seroma or hematoma. In helping you and the patient with this clinical question, you must weigh the risks and benefits of drain use. At the same time, you should think about all the options for drain use and the clinical population this question will be applied to. Next, the consequences of each option are considered, such as formation of hematoma, seroma, or infection. Once the patient population, clinical question, and major consequences are identified, we are ready to create a model.
Step 2: Create a Model
A number of computer software programs are available to aid in formation and analysis of the model. Commonly used programs are SMLTREE (Hollenberg JP; Roslyn, N.Y.), TreeAge Pro (TreeAge Software; Williamstown, Mass.), and DecisionMaker (Pratt Medical Group; Boston, Mass.). The two common forms of modeling in decision analysis are the decision tree or Markov model. Typically, decision analysis involves using a decision tree to model the question. Markov modeling is more advanced and is discussed later. We recommend beginners to first start with a simple tree. The tree contains a model of all options, chance events, and the value of those events. A decision tree is made starting with the root problem at the left of the tree. In our example, the index scenario is a patient undergoing breast reduction surgery. Next, the root gives rise to a choice node, which contains all possible treatment choices. In our example, we will have (1) no drain use, (2) brief drain use, and (3) prolonged drain use as our choices. Each of the choices then gives rise to chance nodes. These chance nodes include all important outcomes or consequences that patients may experience after breast reduction surgery in relation to drain use. For example, each choice may give rise to a patient with no complication, seroma, hematoma, or infection. Each chance event eventually reaches an end outcome on the decision tree. A chance event in some situations may be a terminal outcome or may give rise to additional chance events. Figure 2 illustrates a potential decision tree from our example. Lines between nodes of the decision tree signify passage of time as the tree moves from left to right. The time frame of the decision tree will depend on the nature of the clinical question. Our example is modeled over the course of weeks. Other clinical questions, however, may span months, years, or the life of the patient.
In economic analysis, the perspective of the model must be decided. The perspective may be framed with specific consideration of the patient, hospital, third-party insurer, or society. Typically, all outcomes are with respect to the patient. Depending on the designated perspective, however, indirect and direct costs may vary dramatically. Once the perspective is decided upon, we are ready to assign values to the model.
Step 3: Assign Outcome Values and Probabilities
Each terminal branch of the tree must have an assigned outcome. The outcome of interest typically used in medical decision analysis is utility. Utility is the value or relative preference assigned to a health state, on a scale from 0 (death) to 1 (perfect health). Usually, utilities come from patient ratings but may also come from expert opinion or utilities published in the literature. The most commonly accepted methods of primary utility measurement include visual analog or ratings scales, standard gamble, and time trade-off methods. Visual analog or ratings scales ask participants to assign ratings to a health state. There is evidence, however, that utilities obtained from rating scales do not correlate well with more comprehensive methods, such as standard gamble or time trade-off. Standard gamble and time trade-off methods are generally viewed as more reliable.4
In standard gamble and time trade-off tools, the investigator must explain each health state in written or visual terms so that the scenario is easily understood by the participant. This may be difficult, and the investigator may never know the true level of understanding of the clinical scenarios if the patient does not experience them all personally. The standard gamble asks patients a series of questions to assess the risk of death that patients are willing to accept to trade a given health state for perfect health.5 This degree of risk translates into a numerical utility value. If patients are willing to accept minimal risk of death, then the health state has a high utility and is closer to perfect health on the utility scale. If patients are willing to accept a high risk of death to trade a health state for perfect health, the health state has a low utility and is closer to death on the utility scale. For example, if patients are willing to accept 0.005 probability of death to trade having a postoperative hematoma for perfect health, the utility of postoperative hematoma after breast reduction is 0.995. The standard gamble incorporates ratios or percentages; thus, it may be difficult for some participants to conceptualize if they are not familiar with ratios. The time trade-off method translates into a numerical utility score by determining the length of life expectancy that patients are willing to give up to trade a given state for perfect health.6 Patients willing to give up minimal time will assign a health state a utility closer to perfect health. If patients are willing to give up significant time to trade a health state for perfect health, then the utility assigned to that health state will be much lower. The proportion of time patients are willing to give up will be used to calculate the utility for that health state. For example, imagine patient willing to give up one-half a day of two weeks of remaining life to have perfect health instead of a postoperative infection for the same duration. The utility of having a postoperative infection is one minus the proportion of time the patient is willing to give up for perfect health, 0.96 in this case. The time trade-off method may be easier than standard gamble for participants to conceptualize because ratios are not used. Utility can then be translated into quality-adjusted life years. Figure 3 illustrates the calculation used to translate utilities into quality-adjusted life years. These life years are a useful outcome unit because they not only compare the length of remaining life but they compare the quality of remaining life. In addition, quality-adjusted life years are the standard metric for health states and can be used to compare among conditions as disparate as coronary artery disease or spinal cord injury.
Outcomes of all terminal branches coming from a single strategy are averaged based on their probabilities of occurrence, giving an overall utility or value for that choice. This will allow us to compare the choices to aid the decision-making process. To do this, we must assign probabilities for all of the chance events. One requirement of the decision tree is that the sum of all probabilities branching from a given chance node must equal one. Probabilities of events may come from primary data collected from an outcomes study being conducted by the investigator, published studies, or expert opinion. Conclusions of the analysis depend heavily on the quality of data used. For example, data coming from a randomized controlled trial are better than data extracted from case studies. Clinical trials, however, may be impractical or unethical for some clinical scenarios; thus, modeling uses the best available evidence. In some scenarios, the best evidence comes from prospective or retrospective cohort studies. If data are extracted from the literature, a comprehensive literature search akin to that required for a systematic review is performed. Variability of the data is noted in the standard deviations or confidence intervals reported. Average probabilities are extracted from the literature and assigned to respective branches from the chance nodes.
Step 4: Identify the Best Strategy
Once outcome values and probabilities are assigned to the model, the best strategy can be identified. This occurs by “folding back” the decision tree from right to left. Outcomes associated with each strategy contribute to the overall utility for that strategy based on the assigned probability of occurrence. To calculate overall utility for a strategy, the investigator starts at the right of the tree and multiplies the probability for each terminal branch by its utility; these products are added from each branch of the chance node. This is repeated moving from right to left, adding the weighted utilities for each chance node until the strategy is encountered. The sum represents the overall weighted utility for that strategy. The strategy with the greatest utility or quality-adjusted life years is the preferred choice. Some decision analyses, however, cannot find a significant difference between the choices. In this case, it is helpful to know that there is no clear advantage between the choices. The investigator may incorporate cost into the consideration to provide additional information to clarify the dominant strategy. Figure 4 illustrates a sample decision tree from the example comparing reconstruction and amputation for treatment of lower-extremity trauma. The tree contains probabilities, time, and utilities assigned to each branch, which are translated into quality-adjusted life years for each branch and overall quality-adjusted life years for each treatment strategy.
Step 5: Perform Sensitivity Analysis
The decision analysis model is dependent on assumptions made in assigning probabilities and utilities. With variation of utilities given by individual patients and discrepancies of probabilities found in the literature, there is some degree of uncertainty inherent in the decision analysis model. Thus, sensitivity analysis is perhaps the most important component. Sensitivity analysis changes one or more variables at a time to determine whether the overall result of the decision tree changes. Typically, all probabilities and utilities are varied by assuming the best-case and worst-case scenario. If the preferred strategy does not change, then the results are considered robust, making the degree of uncertainty unimportant. If changing one or more variables changes the ideal strategy, then the analysis is considered “sensitive” to that variable.
HOW DO YOU REPORT RESULTS?
Typically, the base case analysis is reported using the best estimates of utilities and probabilities of the health states. With sensitivity analysis, the best-case and worst-case scenarios should be considered, and the decision tree is reanalyzed with utilities and probabilities from these cases. Each of these analyses has an overall utility or quality-adjusted life year for each strategy that is compared and should be reported. In addition, if the sensitivity analysis has identified variables to which the analysis is sensitive, these variables should be reported. Thresholds for these variables should be reported over which strategy becomes preferable over the alternatives. For example, imagine we found that the probability of postoperative seroma with no drain use to be reported between 0.02 and 0.1 in the literature. If sensitivity analysis finds no drain use to have greater quality-adjusted life years as long as the probability of postoperative seroma is less than 0.06, then this is an important threshold that makes drain use more or less of an ideal option.
If an economic analysis is performed, cost values are input at each chance node, similar to the input of utilities for each chance node. The tree is rolled back in the same way, and an overall weighted cost for each strategy is generated. If one strategy has greater utility or quality-adjusted life years and less cost, it is a dominant strategy and clearly preferred. If one strategy has greater utility and more cost, however, an incremental cost-utility ratio should be reported. The incremental cost-utility ratio is the ratio between the added cost compared with the utility gained for the given strategy (Fig. 5). Acceptable thresholds in the literature to adopt a more costly strategy are controversial but are reported to be between $20,000 and $50,000 per quality-adjusted life year gained.7–9
WHAT ARE THE ADVANTAGES AND LIMITATIONS OF DECISION ANALYSIS?
The main advantage of decision analysis is that it gives surgeons a wide perspective and quantifies individual components of a clinical problem. The model aids in understanding the risk-benefit trade-off more clearly so that practitioners can be more cognizant of consequences associated with treatment options. In addition, when utilities are obtained from patient populations, incorporation of patient values into the decision is considered another benefit of the technique.10 The critical assessment of various treatment strategies can benefit society by guiding formulation of health policy and identifying optimal economic directions.
Decision analysis by nature has inherent limitations. Decision trees model clinical scenarios in a format that is easy to interpret by clinicians without expertise in decision analysis.11 It is often impossible, however, to represent all options and chance occurrences in the model. But if the model is an oversimplification of reality, the results of the decision analysis will not be valid. There must be a balance of including the most important events associated with treatment options without making the model too complicated to be understood by the reader. In addition, the decision tree is limited in that patients cannot experience multiple outcomes at the same time.
The use of diverse and imprecise data in a complex clinical scenario is another inherent limitation in decision analysis. It is often difficult to construct decision trees with high-quality outcomes studies. The conclusions of the decision analysis are heavily dependent on the quality of assumptions from which they are made. For this reason, sensitivity analysis is one of the most important components. A well-conducted study uses sensitivity analysis to identify robustness and limitations within the analysis. At the same time, however, these limitations are helpful in identifying areas needing future investigation where evidence is sparse or lacking.
EVALUATION AND APPLICATION OF DECISION ANALYSIS IN THE LITERATURE
Now that readers are familiar with the essential components of decision analysis, they are better able to evaluate decision analysis studies in the published literature. The reader must first determine whether the results are valid. The checklist in Figure 1 not only helps in performing decision analysis but aids the reader in evaluating published decision analysis studies. After validity is assessed, the reader should determine whether his or her patient population has similar characteristics as those modeled in the study. Probability estimates should be similar to the values expected by the patients in question, and utilities should be similar to values that the patients put on health states. For example, if you have significantly higher rates of seroma after breast reduction surgery in your practice than what is reported in the base study or its sensitivity analysis, the results as reported may not be applicable. Practice profile differences and the effect of these differences on overall outcome can be used for reanalysis of the decision model if the author has illustrated each component of the analysis in detail. Next, the reader must decide whether the strength of evidence used in the study is acceptable. This includes review of the quality of the studies used and whether sensitivity analysis was performed to determine the robustness of the results that were reported. Lastly, the patient's values and aversion of risk must be considered. Decision analysis assumes all patients approach risk similarly. If one strategy has overall greater utility but requires an operation with small risk of a severe complication that the patient is unwilling to accept, it may not be the best choice for your patient. A different patient may be willing to accept that risk with hope for a better outcome.
As discussed earlier, the inability to experience multiple outcomes is one limitation of the decision tree design. Markov modeling is a more advanced technique, and it is beyond the scope of this guide to explain the technique in detail. The beginner should be aware of its benefit, however. Markov modeling attempts to address the limitation of decision trees by modeling clinical scenarios that allow patients to come in and out of health states that repeat themselves over time.12 For example, Markov modeling could be applied to decision analysis to compare treatment strategies for keloids, because patients often try multiple treatment options and recurrent health states, such as keloid reformation, are common over time. Time cycle length and the number of cycles that the Markov model passes through are determined by the investigator.
CONCLUSIONS AND ADDITIONAL RESOURCES
Decision analysis is a useful tool in clinical decision making and formulation of health policy. Thus, it is important for plastic surgeons to become familiar with the technique, its uses, and limitations. Decision analysis, however, is not a replacement for clinical judgment. It can help clinicians better understand benefits and consequences associated with treatment strategies. Decision analysis can also help plastic surgeons become more aware of specific aspects of decisions that have the greatest effect on improving outcomes. The reader is directed to a comprehensive decision analysis five-part series published in Medical Decision Making,13–17 which contains information that builds further on this guide.
This work was supported in part by a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (to Dr. Kevin C. Chung).
1. Saddawi-Konefka D, Kim HM, Chung KC. A systematic review of outcomes and complications of reconstruction and amputation for type IIIB and IIIC fractures of the tibia. Plast Reconstr Surg.
2. Chung KC, Saddawi-Konefka D, Haase SC, et al. A cost-utility analysis of amputation versus salvage for Gustilo IIIB and IIIC open tibial fractures. Plast Reconstr Surg.
3. Chung KC, Shauver MJ, Saddawi-Konefka D, et al. A decision analysis of amputation versus reconstruction for severe open tibial fracture from the physician and patient perspectives. Ann Plast Surg.
4. Drummond M, O'Brien B, Stoddart G, et al. Methods for the Evaluation of Health Care Programmes.
New York: Oxford University Press; 1997.
5. Kucey DS. Decision analysis for the surgeon. World J Surg.
6. Redelmeier DA, Detsky AS. A clinician's guide to utility measurement. Prim Care
7. Chen NC, Shauver MJ, Chung KC. A primer on use of decision analysis methodology in hand surgery. J Hand Surg Am.
8. Davis EN, Chung KC, Kotsis SV, et al. A cost/utility analysis of open reduction and internal fixation versus cast immobilization for acute nondisplaced mid-waist scaphoid fractures. Plast Reconstr Surg
2006;117:1223–1235; discussion 1236–1228.
9. Thoma A, McKnight L, Knight C. The use of economic evaluation in hand surgery. Hand Clin.
25:113–123, viii, 2009.
10. Elwyn G, Edwards A, Eccles M, et al. Decision analysis in patient care. Lancet
11. Birkmeyer JD, Liu JY. Decision analysis models: Opening the black box. Surgery
12. Xu P, Lan L, McCabe SJ. Modeling the management of a flexor sheath ganglion. Hand (N Y).
13. Detsky AS, Naglie G, Krahn MD, et al. Primer on medical decision analysis: Part 1. Getting started. Med Decis Making
14. Detsky AS, Naglie G, Krahn MD, et al. Primer on medical decision analysis: Part 2. Building a tree. Med Decis Making
15. Naglie G, Krahn MD, Naimark D, et al. Primer on medical decision analysis: Part 3. Estimating probabilities and utilities. Med Decis Making
16. Krahn MD, Naglie G, Naimark D, et al. Primer on medical decision analysis: Part 4. Analyzing the model and interpreting the results. Med Decis Making
17. Naimark D, Krahn MD, Naglie G, et al. Primer on medical decision analysis: Part 5. Working with Markov processes. Med Decis Making