Care for the critically ill is complex, dynamic, and resource-intensive; decisions must be made daily whether to initiate, continue, or withhold/withdraw life-sustaining treatments. Accurate prognostication and identification of patients who might benefit or will not benefit from aggressive treatment are essential aspects of patient care in the intensive care unit (ICU). Such clinical understanding is essential for the communication that leads to critical care decision making, which translates into family satisfaction1 and to transitions toward palliative care when patients will no longer benefit from critical care.
Scoring systems have been developed to objectively predict outcomes among seriously ill patients; however, several studies suggest that clinicians are able to as accurately prognosticate mortality for individual patients by a gestalt process compared with prognostic scoring systems.2–4 Poses and colleagues5 showed that critical care clinicians have excellent overall discriminating ability concerning survival (separating patients into those who survive and those who die) and that attending physicians’ predictions were better calibrated (how well predicted probabilities agreed with actual observed risk) than those of house officers. Elsewhere, Poses and colleagues6 point out that accurate prognostic calibration can be an important determinant of quality of care. The critical care physician must be able to predict not only a patient’s survival probability but also whether ICU interventions can influence this probability.
Fellows in critical care have completed at least three years of postgraduate training, usually in internal medicine, and are responsible for supervising residents in the ICU of an academic medical center while they learn the skills of critical care. Fellows learn procedures and the approach to the critically ill patient. During training, fellows likely also develop their prognostic ability. To explore prognostic capability among critical care fellows, we compared fellows’ and attendings’ assessments of futile or probably futile critical care and evaluated factors associated with these assessments. This report builds on our previous work in which attending physicians’ assessments of futile treatment were quantified,7 and explores whether and how assessments from critical care physicians-in-training differ from attendings. We hypothesized that fellows would not discern whether aggressive critical care is futile as accurately as attending physicians.
This study evaluated attendings’ and fellows’ assessments of futile treatment in critical care at the University of California, Los Angeles (UCLA) over a three-month period. The methodology for the development of the survey based on focus group findings, canvassing ICUs to administer the survey to clinicians, collecting patient data, and analyzing data to characterize futile treatment and associated outcomes has been described elsewhere7 and is summarized here. Here, to test the hypotheses stated above, we describe responses to the survey of fellows and compare assessments of futile treatment between attendings and fellows. This study was approved by the UCLA institutional review board (IRB#11-002942).
Assessment of futile critical care
We convened 13 attending clinicians who provide care for critically ill patients for a focus group in October 2011 to discuss whether and to whom they provide futile treatment. During the open-ended discussion, participants were asked to describe patients for whom they provided ICU treatment that they judged to be futile. We transcribed audiotapes and used content analysis to identify descriptions of futile treatment, the reasons that participants characterized treatment as futile, and the words used to describe these reasons. These reasons were collapsed into categories of futile treatment, and the wording used by the participants was employed to describe these categories and to create the questionnaire.
The questionnaire (see Supplemental Digital Appendix 1, https://links.lww.com/ACADMED/A255) asked the critical care physician (attending or fellow) to identify patients who were receiving futile treatment, probably futile treatment, or nonfutile treatment on the day the survey was completed. Clinicians were not given directions on how to determine futile treatment. For patients judged to be receiving futile treatment, the physician was asked to provide the reason(s) that the treatment was futile from among the reasons derived from the focus group: burdens grossly outweigh benefits, patient will never survive outside of an ICU, patient is permanently unconscious, treatment cannot achieve the patient’s goals, or death is imminent. Physicians also could write in a reason. Every day from December 15, 2011, through March 15, 2012, research assistants administered the questionnaire to the attending and fellow critical care specialist providing treatment in five ICUs in the UCLA health system: the medical ICU (MICU), the neurocritical care unit, the cardiac care unit (CCU), the cardiothoracic ICU (CT-ICU), and the academic community hospital mixed-use ICU. Physicians were surveyed independently and asked only to make assessments on patients under their care. In all units except the CT-ICU and the community hospital mixed-use ICU, attendings and fellows cover the same patients in the unit together. There is no fellow in the CT-ICU, and the fellow in the community hospital mixed-use ICU was involved with approximately half of the patients. All physicians provided informed consent to participate in the study and completed a brief questionnaire that asked about gender, race, role in the ICU, and years of practice at UCLA.
Other data collection
We obtained patient demographic characteristics including age, gender, ethnicity and race, insurance, and zip code (used to compute distance from home to hospital); source of admission; and Medicare Severity Diagnosis-Related Group (MS-DRG) weight from hospital administrative data. Sources of admission included emergency room (ER), outpatient setting, skilled nursing facility (SNF) or long-term acute care (LTAC) facility, and transfer from an outside hospital (usually for a higher level of care). Distance from residence to the hospital was dichotomized at 20 miles (32.2 km). MS-DRG weight, determined by the patients’ diagnoses and the resources required during their hospitalization, was used as a measure of severity of illness. We subtracted the date of hospital admission from the date of the physician assessment to compute the hospital day of the futility assessment. Midnight and noon census data were obtained for each ICU to determine daily occupancy. An ICU was considered “full” on days when the average midnight and noon census showed less than two available beds (one bed is always reserved as a “code bed”). We obtained clinician gender and years of practice at the institution from a provider questionnaire. The clinician’s day on service was computed for the first patient-specific assessment by that clinician. We abstracted hospital and six-month mortality from electronic medical records and publicly available death records.
We compared the number of nonfutile, probably futile, and futile treatment assessments by attendings versus fellows using a chi-square test. The frequencies and numbers of reasons listed for a futile assessment by an attending or fellow were compared using a chi-square test and a mixed-effects Poisson regression (suitable for repeated-measures count outcomes), respectively. As a sensitivity check, we identified a subsample of patients who were assessed on the same day by both the attending and fellow to be receiving futile treatment and compared the frequency and number of reasons listed for the futile assessment.
We generated separate multivariate models for attendings and fellows to understand the predictors of an assessment of futile treatment. We used a multilevel ordered probit mixed-effects model that included patient and clinician characteristics, which assumes approximately equal effects of the predictors on moving from nonfutile to probably futile treatment as from probably futile to futile treatment. Because each assessment was cross-classified by patient and physician, we included random intercepts for both patients and physicians. Models were estimated using the MCMCglmm function of R statistical software, version 2.15.2 (copyright Kurt Hornik, Vienna, Austria). We present the average marginal change in a patient’s probability of receiving each type of assessment for a one-unit change in the predictor. The proportionality assumption was checked by setting up identical models except changing the ordinal outcome to be nonfutile versus probably futile and futile in the first model and nonfutile and probably futile versus futile in the second model. We conducted a sensitivity analysis by repeating the models for the limited cohort of patients who were assessed by the attending and fellow on the same day.
To evaluate patient outcomes, we categorized patients separately for attendings and fellows into three groups: patients for whom treatment was never perceived as futile; patients with at least one assessment that treatment was probably futile, but no assessments of futile treatment; and patients with at least one assessment of futile treatment. Patients who were assessed as receiving futile treatment only on the day that they transitioned to comfort care were excluded from analysis. Hospital and six-month mortality was determined for each group of patients.
Thirty-six critical care attendings and 14 fellows participated in the three-month study. After excluding ICU boarders, missing assessments (10.3%), and patients who were perceived to receive futile treatment only on the day transitioned to comfort care, the attendings made 6,897 assessments on 1,125 patients in five ICUs. The fellows made 4,407 assessments on 773 patients in four ICUs. Although fellows assessed fewer patients, they identified more patients as receiving futile treatment compared with attendings: 161 of 773 (20.8%) versus 123 of 1,125 (10.9%, P < .001) patients (Table 1).
Reasons why treatment was considered futile
For both attendings and fellows, the most common reason for treatment to be perceived as futile was that burdens grossly outweighed the benefits (58% of attending assessments and 68% of fellow assessments). Fellows more often listed only a single reason (66 [41%] versus 22 [18%], P < .001), and attendings listed more reasons than fellows (mean 3.0 ± 0.13 versus 2.1 ± 0.11 reasons per patient, P < .001) (Table 2). There were 62 patients for whom assessments of futile treatment were rendered by the fellow and the attending on the same day. Comparison of the number of reasons that the treatment was futile in this matched cohort was similar to the full sample (Supplemental Digital Table 1, https://links.lww.com/ACADMED/A255).
Factors associated with attendings’ and fellows’ assessments of futile treatment
In both attending and fellow multivariate multilevel ordinal probit models, patient age, patient hospital day, and an admission from an SNF or LTAC (in comparison with an admission from the ER) were associated with an increase in the mean probability for patients to be perceived as receiving futile treatment (Tables 3 and 4). In the attending model, patients admitted from the outpatient setting were less likely to be perceived as receiving futile treatment. There was no significant relationship between futile treatment assessment and patient race, Hispanic ethnicity, insurance, MS-DRG, distance from home to hospital, or whether the ICU was full in either model. Neither physician gender nor years at the institution was a significant predictor of the perception of futile treatment. Patients treated in the MICU were significantly more likely to be perceived as receiving futile treatment than patients in the CCU in both the attending and fellow models.
Physician day on clinical service was a significant predictor of assessment of futile treatment and was different between the attending and fellow models. In the fellow model, the probability that a patient would be assessed to be receiving futile treatment significantly increased by the second day on service, whereas the likelihood of an attending assessment of futile treatment did not increase significantly until the fourth day.
There were 3,675 assessments (on 713 patients) made on the same day by attending and fellow used in the sensitivity analysis to construct identical attending and fellow models (Supplemental Digital Tables 2 and 3, https://links.lww.com/ACADMED/A255). These models were similar to models presented above. Examining the proportionality assumption of the ordinal models in two binary probit models, results were similar for both attendings and fellows with the exception that, in the attending model comparing not futile and probably futile versus futile treatment, results were different for service day (Supplemental Digital Tables 4 and 5, https://links.lww.com/ACADMED/A255).
Outcome of patients perceived to receive futile treatment
For both attendings and fellows, hospital and six-month mortality were significantly higher for patients perceived as receiving futile and probably futile treatment compared with patients perceived as receiving nonfutile treatment (Table 5). Patients categorized by attendings as receiving futile treatment had higher ICU and six-month mortality than those categorized by fellows (68% and 84% versus 51% and 62%, respectively, P < .001 for both comparisons). Patients who were assessed by both the attending and the fellow to be receiving futile treatment had a six-month mortality of 87%.
Attending critical care specialists appear to take a more considered and nuanced approach than do critical care fellows in making decisions about whether patients are receiving futile treatment, which results in fewer patients identified as receiving futile treatment. Attending physicians, compared with fellows, tended to provide care for approximately two days longer before classifying patients as receiving futile treatment. In addition, the attending physicians wove together more reasons to support their futile treatment assessments than did the fellows. This slower, more complex assessment of futile treatment by attendings yielded a smaller group of patients that was more likely to die in the hospital and within six months. On the other hand, over 30% of patients that were assessed as receiving futile treatment by fellows were alive at six months. This is consistent with a prior finding that trainees tend toward overconfidence in the “correctness” of their diagnoses.8 These findings suggest that fellows must learn patience and complexity of assessment in order to hone the prognostic skills displayed by critical care attendings. Our findings also suggest that these skills can be assessed, which could provide an additional tool for the evaluation of clinician maturity.
In medicine, judgments of outcomes are necessarily probabilistic; nonetheless, how well physicians make judgments can determine the quality of care they provide.6 We show that fellows do not discriminate as well as attending physicians in their judgments of futile treatment. This is an important topic for critical care fellow training and an aspect of prognostication to be monitored as fellows mature as clinicians. The finding that fellows draw conclusions earlier suggests inadequate maturation in “slowing down when they should,” as pointed out by Moulton and colleagues.9,10
The finding that attending critical care specialists require about four days to reach the conclusion that a patient is receiving futile treatment also has implications for service rotation duration, particularly in academic ICUs. It is notable that this finding was present even after accounting for the number of days that the patient had spent in the hospital. It takes several days for a critical care specialist to get to know a patient well enough to deem treatment nonbeneficial. When a new clinician rotates onto service, the clock restarts, delaying efforts to reorient treatments toward palliation. This finding supports the notion that an intervention that communicates prognostic perceptions between critical care specialists might facilitate decision making for patients unlikely to benefit from ICU care. It also raises the question of whether short-duration rotations compromise outcomes and efficiency. Two small studies in other venues suggest little effect of attending rotation duration on teaching and patient outcomes,11,12 but the data presented suggest that this issue requires more exploration in critical care. Similar questions have been raised about whether frequent transitions lead to fragmented experiences and redundancy for medical trainees.13,14
As might be expected, patients assessed as receiving futile treatment by both fellow and attending had a higher mortality rate than those so designated by only one physician. What is somewhat unexpected is the substantial variation in assessments between attendings and fellows. Although it might be anticipated that greater accountability would lead the attending to render more conservative estimates of futile treatment, one might anticipate that discussion between attendings and fellows would have resulted in less divergent assessments between these groups. This observation raises the question of whether such conversations are indeed occurring and, if not, why not? More group discussion and decision making about nonbeneficial treatment, perhaps explicitly during rounds, might improve patient care as well as trainee education.
This study has several limitations. This is a cross-sectional analysis, meaning that the comparison between fellows and attendings can be used for hypothesis generation, but not causal inference that it was the maturation process that presumably occurs during training that led to more nuanced attending decisions. In addition, we evaluated only one type of prognostication, and probably the type that is most complex; these findings may not apply to fellows learning basic prognostic assessment. Neither attendings nor fellows were given directions on how to determine futile treatment, but were provided with potential reasons on the survey as to why treatment might be considered to be futile that were derived from the focus group (i.e., burdens outweigh benefits). The study was a single-center study and may not be generalizable. It was performed over a three-month period and included only 50 physicians. Lastly, one variable failed in one attending binary probit model testing the proportionality assumption. Because only 6.7% of all attending ratings were for futile treatment, the model comparing not futile and probably futile versus futile may be unstable.
Attending critical care specialists appear to use a more complex approach in making more accurate assessments concerning inappropriate care compared with critical care fellows. Evaluation of prognostic capability may provide a method of evaluation of fellow maturation. Critical care attendings’ nuanced approach to the assessment of nonbeneficial critical care requires time, which may have implications for short-duration attending rotations. Targeted communication interventions might enhance training and patient care.
1. Heyland DK, Rocker GM, Dodek PM, et al. Family satisfaction with care in the intensive care unit: Results of a multiple center study. Crit Care Med. 2002;30:1413–1418
2. Kruse JA, Thill-Baharozian MC, Carlson RW. Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA. 1988;260:1739–1742
3. Scholz N, Bäsler K, Saur P, Burchardi H, Felder S. Outcome prediction in critical care: Physicians’ prognoses vs. scoring systems. Eur J Anaesthesiol. 2004;21:606–611
4. Litton E, Ho KM, Webb SA. Comparison of physician prediction with 2 prognostic scoring systems in predicting 2-year mortality after intensive care admission: A linked-data cohort study. J Crit Care. 2012;27:423.e9–423.e15
5. Poses RM, Bekes C, Copare FJ, Scott WE. The answer to “What are my chances, doctor?” depends on whom is asked: Prognostic disagreement and inaccuracy for critically ill patients. Crit Care Med. 1989;17:827–833
6. Poses RM, Cebul RD, Centor RM. Evaluating physicians’ probabilistic judgments. Med Decis Making. 1988;8:233–240
7. Huynh TN, Kleerup EC, Wiley JF, et al. The frequency and cost of treatment perceived to be futile in critical care. JAMA Intern Med. 2013;173:1887–1894
8. Friedman CP, Gatti GG, Franz TM, et al. Do physicians know when their diagnoses are correct? Implications for decision support and error reduction. J Gen Intern Med. 2005;20:334–339
9. Moulton CA, Regehr G, Mylopoulos M, MacRae HM. Slowing down when you should: A new model of expert judgment. Acad Med. 2007;82(10 suppl):S109–S116
10. Moulton CA, Regehr G, Lingard L, Merritt C, MacRae H. Slowing down to stay out of trouble in the operating room: Remaining attentive in automaticity. Acad Med. 2010;85:1571–1577
11. Elnicki DM, Cooper A. Effects of varying inpatient attending physician rotation length on medical students’ and attending physicians’ perceptions of teaching quality. Teach Learn Med. 2011;23:37–41
12. Jacobs DG, Sarafin JL, Head KE, et al. Trauma attending physician continuity: Does it make a difference? Am Surg. 2010;76:48–54
13. Bernabeo EC, Holtman MC, Ginsburg S, Rosenbaum JR, Holmboe ES. Lost in transition: The experience and impact of frequent changes in the inpatient learning environment. Acad Med. 2011;86:591–598
14. Holmboe E, Ginsburg S, Bernabeo E. The rotational approach to medical education: Time to confront our assumptions? Med Educ. 2011;45:69–80