In the care of the critically ill, intensive care physicians routinely make decisions under conditions of uncertainty, with time pressure, and having incomplete information. Intensive care is cognitively demanding, involves frequent judgments, and puts providers at risk of error and decisional overload. Complicating matters further, imperfect provider knowledge of the costs and benefits of therapies can produce judgments about treatments that are inaccurate. Although the expert intuition of intensive care physicians is frequently correct, there remain times when it is off the mark.
The decisions that are made under this set of conditions—time pressure, incomplete information, imperfect knowledge, and high stakes—are also found outside of medicine and have been the focus of behavioral economists for several decades. Early work in this space studied the performance and behavior of another group of highly trained personnel, firefighters, who operate under similar conditions of extreme time pressure and in emergencies where the consequences of judgments can impact lives (1). Fire ground commanders are emergency providers who deploy personnel and resources at the fire scene, prescribing the tactical approach. Klein et al (1) evaluated the decisions of high-ranking fire ground commanders from seven municipalities in the United States, to understand the relationships among time pressure, cognitive load, decisional approach, and error.
Not surprisingly, most of the decisions made by fire ground commanders occurred quickly and involved significant time pressure. From 156 decision points, 78% were made in less than 1 minute, 10% were made in 1–2 minutes, 6% were made in 2–5 minutes, and the remaining 6% were made in 5 minutes or more. The firefighters rarely described situations where they applied deliberate selection of one alternative from several considered approaches. Rather, a majority of decisions employed an approach called “prototype matching,” where the set of conditions at hand match prior experiences and dictate the course of action. These decisions were almost “nondecisions” for the firefighters because they did not report active deliberation of alternative approaches. When circumstances did not align with prior experiences, a more deliberative process was employed—often with collaboration between other providers. The researchers called this a “recognition-primed decision” model, specifically applied to personnel with high levels of experience in conditions where there is a requirement for rapid and consequential decisions.
Most, if not all, intensive care physicians have had times when they figuratively felt they were extinguishing fires. On a typical day in the hospital, intensive care physicians average more than 100 patient care–related decisions—the majority of these decisions occurring during rounds—for an average of over 28 clinical decisions per hour (2). High-impact decisions related to code status, invasive hemodynamics, family meetings, or procedures occur less frequently, representing approximately 12% of total decision burden, yet are still required nearly every 15 minutes. Intensivists, like fire fighters, apply different decision-making strategies based on the circumstances at hand. Similarly, in both intuitive and analytic approaches, misjudgments of costs and benefits can impact quality. Given the high and growing cost of intensive care delivery, are economic studies a means of improving clinical care?
In this issue of Critical Care Medicine, Wilcox et al (3) present a carefully conducted systematic review of cost-effectiveness studies involving intensive care. Their group examined the last 26 years of the peer-reviewed literature and identified 97 articles that included cost-effectiveness ratios related to the care of the critically ill. These types of studies appeared in the literature only a few times per year, although the frequency increased over the study interval. Additionally, they found an increasing proportion of cost-effectiveness studies to randomized trials, showing a shifting emphasis toward this study type within intensive care research. They conclude that intensive care, despite its high-cost burden, underrepresents cost-effectiveness studies relative to randomized controlled trials in the peer-reviewed literature.
How do the findings by Wilcox et al (3) compare to the rest of the medical literature? To answer this question, we conducted our own PubMed search on the proportion of cost-effectiveness studies to randomized controlled trials for general medicine, over the same 26-year time span. We found a larger proportion of published cost-effectiveness studies to randomized controlled trials between 1993 and 2012 (averaging approximately 0.11), and then an increasing trend of studies between 2012 and 2018 (with 2018 reporting almost 0.20; Fig. 1). In contrast, Wilcox et al (3) showed that the proportion of cost-effectiveness studies to clinical trials, although increasing, still only ranged from 0.012 to 0.04. Is this a feature of there being fewer effective interventions in critical care medicine to study, or are we simply not studying cost-effectiveness? Put another way, have they identified a missed opportunity or a lack of opportunity? The systematic review by Wilcox et al (3) does not give us that answer, but both are possibilities. What is clear is that cost-effectiveness evaluations are both infrequent in the intensive care literature and relatively less frequent compared with the rest of medicine. These are notable gaps and deserve attention in future work.
Wilcox et al (3) goes on to conclude that clinicians, administrators, and policy makers will all require results from economic studies to make better informed decisions regarding resource allocations. Although this may be true to varying extents, it draws attention to an interesting question—who is the consumer of cost-effectiveness research? Is it the clinician, to better inform the complex decisions of intensive care delivery and make better choices? There is no doubt that intensive care is expensive—so perhaps higher value treatment decisions would improve quality and reduce costs? We argue against this assertion, primarily because the perspective of cost-effectiveness research is almost never the bedside provider. Indeed, current recommendations support reporting costs and benefits from the health sector and societal perspectives, and for good reason (4). These referent cases allow findings to be interpreted in a broader context and are more useful for making decisions about health technology or therapy investments within a constrained budget—for hospitals, health systems, and society. The intensive care provider, however, does not act from any of these positions—rather, intensivists make moment-to-moment judgments and decisions for the patient they are actively treating. The expectation that difficult treatment decisions could be further refined to incorporate effects on other “statistical patients” in a way that would measurably improve the quality of all critically ill patients in society or a healthcare system is not realistic. There is no question that there will be cost-effectiveness studies that resonate with some providers, motivating a change in their clinical care, but in a durable, predictable, and large-scale manner, bedside providers are not the most effective target audience for these studies.
Instead, we believe cost-effectiveness studies have a role at hospital, health system, and societal levels for decisions on the availability of therapies and technologies. Cost-effectiveness studies are not able to establish healthcare priorities or the size of healthcare budgets, but they do provide a basis for choosing between therapies to maximize value. One promising way of translating system level judgments to care at the bedside is in the form of “nudges,” strategic ways to present options to providers that encourage specific choices (e.g., listing medication treatments for a particular condition by relative cost-effectiveness, rather than alphabetically) (5). Nudges are ways that choice architecture alters behavior predictably, but that do not remove options or significantly change economic incentives. Choice architecture, informed by cost-effectiveness analysis, is particularly well-suited to intensive care delivery, with its multitude of treatments options, high costs, time pressure, and many decision points (2). There is evidence that nudges can also help in conditions that are prone to decision fatigue (6), such as the ICU. More work should be done to understand how to most effectively deploy tools such as choice architecture and clinical decision in the critical care setting (7).
The systematic review by Wilcox et al (3) is important because it shows us that intensive care significantly lags behind in cost-effectiveness research. We agree that these studies have value in making decisions among therapies, approaches, and technologies—particularly at the hospital, hospital system, and societal levels. A reliance on bedside providers to incorporate economic evaluations into their care is less likely to have a meaningful impact on quality and spending and even may result in unintended negative consequences. However, the results of economic evaluations “can” be translated to the bedside through implementation approaches that coordinate decision-making at different organizational levels. Both the effortful, conscious, and deliberate processes of contemplative decision-making and the automatic decisions associated with putting out clinical fires can benefit from cost-effectiveness studies—it is only a matter of doing the work and finding the right nudges.
1. Klein GA, Calderwood R, Clinton-Cirocco A. Rapid decision making on the fire ground. Proc Hum Factors Ergon Soc Annu Meet 2016; 30:576–580
2. McKenzie MS, Auriemma CL, Olenik J, et al. An observational study of decision making by medical intensivists. Crit Care Med 2015; 43:1660–1668
3. Wilcox ME, Vaughan K, Chong CAKY, et al. Cost-Effectiveness Studies in the ICU: A Systematic Review
. Crit Care Med 2019; 47:1011–1017
4. Sanders GD, Neumann PJ, Basu A, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine. JAMA 2016; 316:1093–1103
5. Thaler RH, Sunstein CR. Nudge: Improving Decisions About Health, Wealth, and Happiness. 2009New York, Penguin Books.
6. Kim RH, Day SC, Small DS, et al. Variations in influenza vaccination by clinic appointment time and an active choice intervention in the electronic health record to increase influenza vaccination. JAMA Netw Open 2018; 1:e181770
7. Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: A systematic review
of trials to identify features critical to success. BMJ 2005; 330:765