Secondary Logo

Journal Logo

Observational Study

Adapting a Traumatic Brain Injury Goals-of-Care Decision Aid for Critically Ill Patients to Intracerebral Hemorrhage and Hemispheric Acute Ischemic Stroke

Goostrey, Kelsey J. BA1; Lee, Christopher BA1; Jones, Kelsey BA1; Quinn, Thomas MD2; Moskowitz, Jesse MD3; Pach, Jolanta J. BS4; Knies, Andrea K. Dr rer. Medic5; Shutter, Lori MD, FNCS, FCCM6; Goldberg, Robert PhD7; Mazor, Kathleen M. EdD8,9; Hwang, David Y. MD, FAAN, FCCM, FNCS10,11; Muehlschlegel, Susanne MD, MPH, FCNS, FCCM1,12,13

Author Information
Critical Care Explorations: March 2021 - Volume 3 - Issue 3 - p e0357
doi: 10.1097/CCE.0000000000000357

Abstract

Shared decision-making is a collaborative approach that encourages clinicians, patients, and surrogates to work together to make treatment decisions based on the best available scientific evidence, while also considering the patient’s values, goals, and preferences (1). The Institute of Medicine and major critical care societies have strongly recommended that clinicians incorporate shared decision-making in the ICU (1,2). Although shared decision-making has been successfully implemented for many health decisions (3), empiric research in medical-surgical ICUs has shown that only 2% of ICU clinician-family communications meet all recommended criteria for shared decision-making (4). Our previous research in surrogates and physicians in the neurologic ICU (neuroICU) has identified an urgent but unmet need for goals-of-care decision support and shared decision-making tools (decision aids [DAs]) (5). The sudden and unexpected nature of catastrophic neurologic diseases adds greatly to the surrogates’ burden of making the morally difficult life-or-death goals-of-care decision on behalf of these critically ill and incapacitated patients. This large emotional burden on families, combined with existing deficiencies in clinician-family communication in the ICU, results in poorly informed treatment decisions, high rates of psychologic distress in families, and high utilization of expensive, burdensome treatments which may be misaligned with patients’ values and preferences (4,6,7).

To narrow this gap, we developed a paper-based pilot DA for surrogates of critically ill traumatic brain injury (ciTBI) patients making goals-of-care decisions using a rigorous and systematic approach (8). Importantly, this DA meets International Patient DA Quality Standards (8,9), which is an international guide on the content of high-quality DAs. During the DA’s development process, we received input from stakeholders, both surrogates and physicians, that goals-of-care DAs for other neurologic emergencies are urgently needed.

Intracerebral hemorrhage (ICH) and large hemispheric acute ischemic stroke (AIS) are the fifth leading cause of death in the United States (10,11), and most of the deaths related to both of these diseases are due to withdrawal of life-sustaining treatments in ICUs (12). Currently, no DAs exist for surrogates of critically ill ICH and AIS patients. One other DA for strokes of all severities has recently been developed, which offers a web-based tool for both noncritically ill and critically ill patients making life-sustaining treatment decisions (13). Our DA is different from this tool in several ways: our tool purposefully includes 10 real photographs supplementing text, depicts the estimated prognosis in three levels (rather than 5) using icon arrays (rather than pie charts), all with the intention to lower the cognitive burden on stressed surrogates of critically ill patients (8,13).

The objective of our current study was to adapt our existing goals-of-care DA for surrogates of ciTBI patients to critically ill patients with ICH and large hemispheric AIS, measure usage time, usability and acceptability, and obtain qualitative feedback. These important steps are standard and necessary in the DA development process prior to deploying the DAs in the neuroICU for feasibility.

MATERIALS AND METHODS

This study was conducted at two academic medical centers: the University of Massachusetts Medical School/UMass Memorial Medical Center and Yale New Haven Hospital. The University of Massachusetts Institutional Review Board (IRB protocol number H00015764) and the Yale University Institutional Review Board (IRB protocol number 2000023655) granted approval for this study and allowed participants to be enrolled through verbal consent due to the minimal risk posed by the study.

Goals-of-Care DA for Surrogates of ciTBI Patients

The rigorous, stepwise development process for our goals-of-care DA for surrogates of ciTBI patients, which meets International Patient DA Quality Standards criteria for high-quality DAs (9), has been previously published (8). In brief, this traumatic brain injury (TBI) goals-of-care DA is paper based, written at a sixth grade reading level, printed on 12 double-sided pages in Times New Roman font in letter size 16, and contains 10 photographs and several tables. The ciTBI DA includes the following information: TBI terminology and essential background information; the nature of the goals-of-care decision; treatment options; potential post-ICU discharge dispositions or withdrawal of life-sustaining treatments with comfort-based care; an individualized icon array displaying each patient’s unique estimated prognosis derived from the validated TBI International Mission for Prognosis and Analysis of Clinical Trials model, accompanied by text carefully explaining and emphasizing prognostic uncertainty associated with prediction models; two patient examples; and an one-page worksheet designed to prepare surrogates for the family meeting (including a values clarification exercise, decision-readiness assessment, and a tear-away page to record questions) (Supplemental Fig. 1, Supplemental Digital Content 1, https://links.lww.com/CCX/A517) (8).

Adaptation of ciTBI DA to ICH and AIS

To adapt our existing goals-of-care DA for patients with ciTBI to those with an ICH and hemispheric AIS, we first changed the essential background information and disease-specific terminology about TBI to ICH and AIS, using terminology of “hemorrhagic stroke” and “ischemic stroke.” We anticipated that additional changes to the terminology would be required based on family user feedback, which we were planning to obtain as part of the adaptation process.

Per the International Patient DA Quality Standards, all DAs should include data derived from validated outcome prediction models that carefully visualize potential outcomes and describe uncertainties (9). In accordance with these standards, and based on our prior qualitative work with neuroICU surrogates showing that estimated prognosis was a strongly desired component (14), we chose to include an individualized prognosis estimated by a validated prediction model, similar to the ciTBI DA. Of note, we also chose to provide accompanying text to clearly explain and emphasize prognostic uncertainty (Supplemental Fig. 2, Supplemental Digital Content 2, https://links.lww.com/CCX/A518), due to our awareness of the potential pitfalls of predicting an individual patient’s prognosis using prediction models derived from broad populations (8). In all of our DAs, explicit text informs surrogates that the prediction model does not take into account their loved one’s medical history, injuries to other organs, clinical course, and complications in the ICU and that they should ask the doctor how these factors may change their loved one’s prognosis.

Two board-certified neurointensivists (D.Y.H., S.M.) reviewed the current literature and carefully weighed advantages and disadvantages of existing models, while also considering patients’ and families’ desire for information on longer term outcomes. After considerable discussion, for the ICH model, we selected the previously validated 12-month functional outcomes associated with the original ICH score, given the ICH score’s widespread use and the value of its 12-month follow-up data (15). We contacted the senior author who shared the raw and deidentified 12-month functional outcome data using the modified Rankin Score for each ICH score for the purpose of our DA adaptation. With similar reasoning, after reviewing 23 different AIS prognostication models (16), we selected the most robust, high-level data, which stem from the large hemicraniectomy trials. We selected the 12-month outcome data available from the published hemicraniectomy trials Decompressive Craniectomy In Malignant Middle Cerebral Artery Infarcts, Decompressive Surgery for the Treatment of Malignant Infarction of the Middle Cerebral Artery (DESTINY), Hemicraniectomy After Middle Cerebral Artery Infarction With Life-Threatening Edema Trial (17), and DESTINY II (18).

We created icon arrays that can be individualized to the patient using the patient’s clinical variables. For ICH, we created six different icon arrays based on the ICH score ranging from 0 to 5. Of note, we did not create an icon array for the most severe ICH score of 6, since no patient received this score in the 12-month validation study (15), likely due to the fact that an ICH score of 6 is both rare and severe, such that patients did not survive to 12 months post ICH. It is also highly unlikely that a DA would be used for a patient with an ICH score of 6 due to the severity of the disease and nearly certain ultimate outcome. For hemispheric AIS, we created four different icon arrays, differentiating between patients less than or equal to 60 years or greater than 60 years, as well as those who did or did not undergo hemicraniectomy. For both ICH and hemispheric AIS, we dichotomized favorable and unfavorable modified Rankin Scale outcomes as less than or equal to 4 and greater than 4.

Acceptability and Usability Testing

Participants.

Participants were recruited from both neuroICU waiting rooms using the following inclusion criteria: age greater than or equal to 18 years, family member or friend of a patient admitted to the neuroICU with any disease, willing to read the DA with a study staff member present; English-speaking, and willing to spend approximately 45 minutes with study staff to provide verbal feedback and undergo usability and acceptability testing using validated scales (19,20). We used a purposive sampling strategy to recruit surrogates who were either actively engaged in the process of supporting their family member in the neuroICU environment, including decision-making for their family member, but did not require surrogates to specifically be making decisions for TBI or stroke patients, given that the focus of feedback was on ease of navigation and use.

Trained members of the study team approached and recruited family members or friends of patients admitted to the neuroICU with a neurologic illness using an IRB-approved verbal consent script and printed fact sheet. After verbal consent was obtained, a member of the study team brought the participant to a private room near the neuroICU waiting room. The participant was provided with a hypothetical scenario about a patient with an ICH or hemispheric AIS awaiting a goals-of-care decision. Participants received the DA as a spiral-bound, color-printed booklet for either ICH or hemispheric AIS on an alternating schedule. The only instructions provided by the study team member were when to start reading the DA and to also complete the worksheet at the end of the DA.

Covariates Collected.

The study team member observed and timed the participant while they read through the DA, noting any signs of struggle or confusion, either nonverbal or verbal. We collected data on the age, gender, and study center of the participants.

Assessment Scales.

Following the reading of the DA and worksheet, the participant completed a short, written questionnaire assessing usability and acceptability of the DA using validated scales. We used the industry standard for assessing usability, the System Usability Scale (SUS), which is a 10-item, five-point Likert-scale with scores ranging from 0 to 100 (19). A SUS greater than or equal to 68 is an accepted cut-off for “good usability” (19). Our target SUS for “excellent usability” was greater than 80 (21). The Acceptability Scale is comprised of an nine-item, four-point Likert-scale that assesses DA content and seven additional questions assessing acceptability of length, information amount, balance of information, usefulness of the DA, understanding of prognosis, the personal worksheet, and completeness of information for decision-making. Two open-ended questions ask what participants liked about the DA and worksheet and any suggestions to improve both items (20). As in our previously published DA derivation study, we conservatively targeted greater than 70% for “good acceptability,” as our review of the literature did not reveal a validated or accepted cut-off for what constituted “good” acceptability (8). Both scales are validated and considered the industry’s standard in DA research. Finally, a short verbal feedback session was elicited by a member of the study staff.

Statistical Analysis.

Standard descriptive statistics were applied to calculate median usage times, SUS and Acceptability Scale scores, as well as proportions for response options. For the Acceptability Scale, we calculated the composite proportions for each Likert-scale point (“excellent,” “good,” “fair,” or “poor”) for the nine content questions. For each of the seven additional acceptability questions, we dichotomized the responses as “favorable” or “unfavorable.” We qualitatively analyzed the verbal feedback and grouped responses by positive comments about the DAs and suggestions for improvement. We recruited a purposive sample of 20 participants. This number was based on what was pragmatically feasible and on studies that show that 80% of usability problems can be identified with as few as 10 participants, whereas 95% of usability problems can be identified with 20 participants (22).

RESULTS

Over a 4-month period (July 2018 to October 2018), 20 participants completed acceptability and usability at both centers; 11 participants received the ICH DA, and nine received the hemispheric AIS DA. Of the participants, 15 (75%) were female, and the mean participant age was 55 (± 15) years. Median time to read the DA was 22 minutes (interquartile range [IQR], 16–26 min). Participants rated the usability of both DAs as “excellent” with a median SUS score of 84 of 100 (IQR, 61–93) with a SUS greater than 68 indicating good usability (19) (Fig. 1). Most participants (89%) rated the acceptability of the DAs’ content as good (49.2%) or excellent (40.1%; multicolor bar in Fig. 1). Participants rated all other items on the Acceptability Scale favorably (length [85%], information amount [80%], balance [90%], usefulness [95%], prognosis [85%], worksheet [95%], and enough information to make a decision [85%]) (light blue bars in Fig. 1). The individual responses of the Acceptability Scale are provided in Supplementary Table 1 (Supplemental Digital Content 3, https://links.lww.com/CCX/A519).

F1
Figure 1.:
Acceptability and usability testing. Twenty participants completed usability and acceptability testing at UMass and Yale. Participants were recruited from the neuroICU waiting rooms at each center and completed testing using validated scales (19 , 20). The multicolor bar represents a composite of nine questions related to the acceptability of the content of the decision aids. Eighty-nine percent of participants rated the content as good or excellent (multicolor bar). The light blue bars represent favorable responses related to the acceptability of length (85%), information amount (80%), balance (90%), usefulness (95%), prognosis (85%), worksheet (95%), and enough information to make decision (85%). The individual responses for the Acceptability Scale can be found in Supplementary Table 1 (Supplemental Digital Content 3, https://links.lww.com/CCX/A519). The dark blue bar represents the System Usability Scale (SUS), the industry standard in measuring usability of a tool (19). Participants rated usability as excellent (median SUS = 84/100), where a SUS greater than 68 is considered good (23).

Verbal feedback from participants was largely positive and acknowledged that both DAs would be very helpful for families making goals-of-care decisions on behalf of their loved ones (Table 1). Several suggestions to improve the DAs were made, including using more lay terms for “intracerebral hemorrhage” and “acute ischemic stroke.” Based on this feedback, we changed the disease names in the titles of DAs to “Bleeding Stroke” and “Stroke,” respectively.

TABLE 1. - Qualitative Feedback by Acceptability and Usability Testing Participants on the Intracerebral Hemorrhage and Large Hemispheric Acute Ischemic Stroke Decision Aids
Participants Found the Decision Aid Helpful Participants Made These Suggestions for Improvement
“I think it was very well done. I think people would find it very helpful. I think it is easy to understand. It’s heart wrenching, but it’s helpful.” [I] “wished [there were more] footnotes [explaining] glossary terms, like what AIS means.”
“With my father, I couldn’t imagine how difficult it would be if I had to make a decision. I think this tool will make a big difference. I think it was really well put together. I think it was all well done.” “[I] wish [there was] more information about the consequences to the family member - for example, the financial and emotional costs associated with each option.”
“I think this will be a really great thing for the families to see. It would be helpful to start thinking about what they will do before the meeting. I personally liked the icons. I see these types of images a lot in my work. I think it was clear.” “In a digital version, there could be videos of, for example, someone on a ventilator, and that at certain points in the aid, the digital system could ask the reader ‘Did you understand ___? Yes/ No’ before allowing them to move on.”
“It’s a great survey, easy to read (especially for a layperson experiencing shock), well laid-out and explained, pretty spot-on, and easy to navigate. The aid could really help someone in that position to make a tough decision.”
We show representative quotes of the qualitative feedback by participants grouped by positive comments about the decision aids and suggestions for improvement. These were derived by either the participants providing written feedback as part of the open-ended questions of the Acceptability Scale or research staff eliciting verbal qualitative feedback and taking notes of the responses.

DISCUSSION

We adapted a novel goals-of-care DA for surrogates of ciTBI patients for use in surrogates of critically ill patients with ICH and large hemispheric AIS by incorporating validated, disease-specific, 12-month outcome prediction data to communicate probabilities of long-term outcome, including the uncertainties pertaining to the outcome. The DAs meet international DA standards and were rated with very high acceptability and usability by surrogates.

To our knowledge, these DAs are the first goals-of-care DAs specifically for critically ill patients with ICH and large hemispheric AIS. There is only one other DA that has been developed for patients who have developed a stroke and their surrogates (13). However, our DAs differ from this recently published DA in several ways: first, our DAs specifically target critically ill patients, rather than serving surrogates of stroke patients of all severities; second, our DAs contain 12-month validated outcomes. Surrogate decision-makers highly value long-term validated outcomes, which the authors of the other DA acknowledged as a limitation in their DA (13). The other DA uses an ordinal prognostication model predicting 90-day functional outcome, which the authors self-developed and derived from data from the Virtual International Stroke Trials Archive Plus (13). Third, our DA uses icon arrays to depict prognosis, in contrast to pie chart and horizontal stacked bars to allow portrayal of 5 ordinal levels of the modified Rankin Score, which could be overwhelming in users with lower numeracy skills; we have based our choice of icon arrays on research that has shown that icon arrays are best for risk communication (24,25) and on direct feedback from surrogates (8). For our DA, we conducted acceptability testing in addition to the standard usability testing. Last, in contrast to the other web-based DA, our tool is paper based with many photographs, which may facilitate use among computer-illiterate surrogates (13).

One other medical/surgical ICU DA is available for patients with prolonged mechanical ventilation without a neurologic diagnosis, but a recent large clinical trial of this DA produced neutral results for the primary outcome of surrogate-physician prognostic concordance and psychologic distress in families (26). Our DA is different from this DA, because it is aimed at surrogates of neurocritically ill patients with TBI, ICH, and large hemispheric AIS, who face much more complex decision-making related to “both” the neurologic prognosis and the need for ventilation and artificial nutrition, and it includes a worksheet with values clarification, which surrogates complete. The goal is for clinicians to use this worksheet to guide the goals-of-care discussion via a “facilitated deliberation” (27–30). We based this approach on expert recommendations (31), and results from two clinical trials suggested that a knowledge approach alone will not influence emotionally laden life-support decisions, which are made intuitively using a “gut feeling” (26,27,32). However, our approach has not been tested in a clinical trial.

We understand that there is great risk in applying prediction models to individual patients. A gap analysis exploring prognostication in neurocritical care noted that current prognostic models are useful but limited (16). Some of the major gaps included lack of individualized patient measures, such as failure to account for medical comorbidities, and not including in-hospital events and clinical complications (16). Another study comparing formal prediction models with clinician judgment when predicting 3-month functional outcome (modified Rankin scale) in ICH patients found that clinicians judgment correlated more closely with actual patient outcome at 3 months (33). However, we included icon arrays with data derived from validated prediction models, because per the International Patient DA Quality Standards, high-quality DAs should present probabilities of potential outcomes (9) and because surrogates expressed a strong desire for it in our prior qualitative research and physicians felt it could ground other physicians given the existing variability in prognostication (14,34–36). We mitigated the risk both by using validated models of high quality and by explicitly acknowledging the uncertainty around the projected outcome in each DA and stating that each patient has other, individual factors that influence outcome, which are not contained in the model. We also advised surrogates to consult with their doctor and ask for “best case” and “worst case” scenarios.

Our study has a number of important strengths and limitations. Strengths include the systematic development process with acceptability and usability testing and feedback from family member and surrogate stakeholders from two different medical centers. Additionally, we ensured that our DAs meet international DA standards. Of note, there is considerable variability in DA research with regards to standards of acceptability and usability testing practices (37). Similar to our published ciTBI DA development process, we recruited general neuroICU surrogate decision-makers for our acceptability and usability testing convenience sample using a hypothetical scenario and did not strictly limit the sample to only families of patients with an ICH or large hemispheric AIS in the midst of a real-time goals-of-care decision. Use of hypothetical scenarios has been a successful, practical approach in a number of prior research studies of ICU surrogates tasked with making high-acuity decisions (38–40). The experiential realism of our hypothetical scenario was enhanced by the fact that we did recruit all subjects for acceptability and usability directly from a neuroICU environment. We acknowledge that using population-based risked estimates to gauge an individual patient’s prognosis is a limitation and that we cannot completely eliminate that small risk of self-fulfilling prophecy; however, research has shown that surrogates desire outcome prognostication (14), and no DA is meant to stand alone without a clinician-family discussion. Additional potential limitations include that we did not collect race, ethnicity, or education level in our acceptability and usability testing participants. Furthermore, we excluded non-English speakers since our DA has not yet been translated and validated in other languages.

CONCLUSIONS

We adapted our goals-of-care DA for surrogates of critically ill TBI patients to surrogates of critically ill patients with ICH and large hemispheric AIS. We created two new DAs, which meet international DA standards, include disease-specific icon arrays to visualize individualized patient outcome probabilities, and have been rated highly by surrogate decision-makers for usability and acceptability. We are currently testing feasibility of DA implementation in a multicenter pilot trial.

ACKNOWLEDGMENT

We thank Dr. Claude Hemphill (University of California, San Francisco) for providing us the de-identified raw data of the 12-month validation study of the Intracerebral Hemorrhage score.

REFERENCES

1. Kon AA, Davidson JE, Morrison W, et al. Shared decision making in ICUs: An American college of critical care medicine and American thoracic society policy statement. Crit Care Med. 2016; 44:188–201
2. Institute of MedicineCrossing the Quality Chasm: A New Health System for the 21st Century. 2001, Washington, DC: The National Academies Press
3. Stacey D, Légaré F, Lewis K, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017; 4:CD001431
4. White DB, Braddock CH III, Bereknyei S, et al. Toward shared decision making at the end of life in intensive care units: Opportunities for improvement. Arch Intern Med. 2007; 167:461–467
5. Muehlschlegel S, Shutter L, Col N, et al. Decision aids and shared decision-making in neurocritical care: An unmet need in our neuroICUs. Neurocrit Care. 2015; 23:127–130
6. Azoulay E, Chevret S, Leleu G, et al. Half the families of intensive care unit patients experience inadequate communication with physicians. Crit Care Med. 2000; 28:3044–3049
7. Azoulay E, Pochard F, Kentish-Barnes N, et al. Risk of post-traumatic stress symptoms in family members of intensive care unit patients. Am J Respir Crit Care Med. 2005; 171:987–994
8. Muehlschlegel S, Hwang DY, Flahive J, et al. Goals-of-care decision aid for critically ill patients with TBI: Development and feasibility testing. Neurology. 2020; 95:e179–e193
9. Elwyn G, O’Connor A, Stacey D, et al. International Patient Decision Aids Standards (IPDAS) CollaborationDeveloping a quality criteria framework for patient decision aids: Online international Delphi consensus process. BMJ. 2006; 333:417
10. Yang Q, Tong X, Schieb L, et al. Vital signs: Recent trends in stroke death rates - United States, 2000-2015. MMWR Morb Mortal Wkly Rep. 2017; 66:933–939
11. Benjamin EJ, Virani SS, Callaway CW, et al. American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics SubcommitteeHeart disease and stroke statistics-2018 update: A report from the American Heart Association. Circulation. 2018; 137:e67–e492
12. Creutzfeldt CJ, Holloway RG. Treatment decisions after severe stroke: Uncertainty and biases. Stroke. 2012; 43:3405–3408
13. Chen EP, Arslanian-Engoren C, Newhouse W, et al. Development and usability testing of understandng stroke, a tailored life-sustaining treatment decision support tool for stroke surrogate decision makers. BMC Palliat Care. 2020; 19:110
14. Quinn T, Moskowitz J, Khan MW, et al. What families need and physicians deliver: Contrasting communication preferences between surrogate decision-makers and physicians during outcome prognostication in critically Ill TBI patients. Neurocrit Care. 2017; 27:154–162
15. Hemphill JC III, Farrant M, Neill TA Jr. Prospective validation of the ICH score for 12-month functional outcome. Neurology. 2009; 73:1088–1094
16. Wartenberg KE, Hwang DY, Haeusler KG, et al. Gap analysis regarding prognostication in neurocritical care: A joint statement from the German neurocritical care society and the neurocritical care society. Neurocrit Care. 2019; 31:231–244
17. Vahedi K, Hofmeijer J, Juettler E, et al. DECIMAL, DESTINY, and HAMLET investigatorsEarly decompressive surgery in malignant infarction of the middle cerebral artery: A pooled analysis of three randomised controlled trials. Lancet Neurol. 2007; 6:215–222
18. Juttler E, Unterberg A, Woitzik J, et al. Hemicraniectomy in older patients with extensive middle-cerebral-artery stroke. N Engl J Med. 2014; 370:1091–1100
19. Brooke JA. Jordan PW TB, Weerdmeester BA. Quick and dirty” usability scale. In: Usability in Industry. 1996, London, United Kingdom: Taylor & Francis; 189–94
20. O’Connor A, Cranney A. Acceptability Scale 1996 2002. Available at: https://decisionaid.ohri.ca/eval_accept.html. Accessed March 07, 2020
21. Cox CE, Wysham NG, Walton B, et al. Development and usability testing of a Web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care. 2015; 5:6
22. Faulkner L. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003; 35:379–383
23. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. J Usability Stud. 2009; 4:114–123
    24. Zikmund-Fisher BJ, Fagerlin A, Ubel PA. Improving understanding of adjuvant therapy options by using simpler risk graphics. Cancer. 2008; 113:3382–3390
    25. Price M, Cameron R, Butow P. Communicating risk information: The influence of graphical display format on quantitative information perception-accuracy, comprehension and preferences. Patient Educ Couns. 2007; 69:121–128
    26. Cox CE, White DB, Hough CL, et al. Effects of a personalized web-based decision aid for surrogate decision makers of patients with prolonged mechanical ventilation: A randomized clinical trial. Ann Intern Med. 2019; 170:285–297
    27. Rubin EB, Buehler AE, Cooney E, et al. Intuitive vs deliberative approaches to making decisions about life support: A randomized clinical trial. JAMA Netw Open. 2019; 2:e187851
    28. Usher M, Russo Z, Weyers M, et al. The impact of the mode of thought in complex decisions: Intuitive decisions are better. Front Psychol. 2011; 2:37
    29. Wilson TD, Schooler JW. Thinking too much: Introspection can reduce the quality of preferences and decisions. J Pers Soc Psychol. 1991; 60:181–192
    30. Creutzfeldt CJ, Longstreth WT, Holloway RG. Predicting decline and survival in severe acute brain injury: The fourth trajectory. BMJ. 2015; 351:h3904
    31. Tannenbaum AM, Halpern SD. Knowing is (only) half the battle. Ann Intern Med. 2019; 170:338–339
    32. Boyd EA, Lo B, Evans LR, et al. “It’s not just what the doctor tells me:” Factors that influence surrogate decision-makers’ perceptions of prognosis. Crit Care Med. 2010; 38:1270–1275
    33. Hwang DY, Dell CA, Sparks MJ, et al. Clinician judgment vs formal scales for predicting intracerebral hemorrhage outcomes. Neurology. 2016; 86:126–133
    34. Turgeon AF, Lauzier F, Simard JF, et al. Canadian Critical Care Trials GroupMortality associated with withdrawal of life-sustaining therapy for patients with severe traumatic brain injury: A Canadian multicentre cohort study. CMAJ. 2011; 183:1581–1588
    35. Turgeon AF, Lauzier F, Burns KE, et al. Canadian Critical Care Trials GroupDetermination of neurologic prognosis and clinical decision making in adult patients with severe traumatic brain injury: A survey of Canadian intensivists, neurosurgeons, and neurologists. Crit Care Med. 2013; 41:1086–1093
    36. Holloway RG, Benesch CG, Burgin WS, et al. Prognosis and decision making in severe stroke. JAMA. 2005; 294:725–733
    37. Evans R, Elwyn G, Edwards A, et al. Toward a model for field-testing patient decision-support technologies: A qualitative field-testing study. J Med Internet Res. 2007; 9:e21
    38. Chapman AR, Litton E, Chamberlain J, et al. The effect of prognostic data presentation format on perceived risk among surrogate decision makers of critically ill patients: A randomized comparative trial. J Crit Care. 2015; 30:231–235
    39. Zier LS, Sottile PD, Hong SY, et al. Surrogate decision makers’ interpretation of prognostic information: A mixed-methods study. Ann Intern Med. 2012; 156:360–366
    40. Hwang DY, Knies AK, Mampre D, et al. Concerns of surrogate decision makers for patients with acute brain injury: A US population survey. Neurology. 2020; 94:e2054–e2068
    Keywords:

    critical care; intracerebral hemorrhage; palliative medicine; prognosis; shared decision-making; stroke

    Supplemental Digital Content

    Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine.