Journal Logo

Feature Articles

Partnering With Family Members to Detect Delirium in Critically Ill Patients*

Fiest, Kirsten M. PhD1–3; Krewulak, Karla D. PhD1; Ely, E. Wesley MD, MPH4; Davidson, Judy E. DNP, RN5; Ismail, Zahinoor MD2,3,6; Sept, Bonnie G.1; Stelfox, Henry T. MD, PhD1,2

Author Information
doi: 10.1097/CCM.0000000000004367

Abstract

Delirium is a disturbance of consciousness characterized by acute onset and fluctuating course of inattention, accompanied by either perceptual disturbances or a change in cognition where a patient’s ability to receive, process, store, and recall information is impaired (1). Delirium is present in one-third of critically ill patients admitted to ICUs (2–8) and is associated with negative outcomes in ICU patients, including longer hospital stays (9), increased risk of long-term cognitive impairment, and death (10–12). Despite its high prevalence and detrimental effects, delirium may go undetected in the ICU or assumed to be a normal part of a patient’s clinical course (13). Although clinical delirium detection tools such as the Intensive Care Delirium Screening Checklist (ICDSC [3]) and Confusion Assessment Method for ICU (CAM-ICU [2]) have been implemented worldwide, no screening approach is perfect and there is a high degree of variability in reported prevalence and occurrence rate estimates (14); up to 66% of delirium cases are missed using nonstandardized clinical approaches (15). Family members often know a patient best and may notice subtle changes in patients’ cognition and behavior earlier than a clinical observer unfamiliar with the patient (16).

Many tools to detect delirium in adults have been developed (13), including two family-administered tools identified in a recent systematic review (17): the Family Confusion Assessment Method (FAM-CAM) (18) and Sour Seven (19), which have not been validated in the ICU. By partnering with patients’ family members (e.g., immediate family, relatives, friends) to detect delirium and as proxies to assess acute change in mental status, we may ultimately be able to improve experiences and outcomes of care for both patients and families (10,20,21). The feasibility and acceptability of employing family-administered delirium detection in the ICU were explored by our team in a pilot study (22); following modifications to the recruitment procedure, including the addition of the family member of a former ICU patient to the study team, we determined the study to be both feasible and acceptable to patients and families. The objective of this study was to evaluate the diagnostic accuracy of two family-administered tools (FAM-CAM and Sour Seven) to detect delirium in critically ill patients.

MATERIALS AND METHODS

Study Design

This diagnostic accuracy study is reported according to Standards for the Reporting of Diagnostic Accuracy Studies (STARD) standards (Online Resource) and registered on ClinicalTrials.gov (NCT03379129). Further details on study design, measures, and the reference standard can be found in the Supplemental Materials (Supplemental Digital Content 1, http://links.lww.com/CCM/F459).

Participants

Participants were recruited from the Foothills Medical Centre (FMC) ICU (Calgary, Canada) between December 2017 and March 2019. The FMC ICU is a closed 28-bed medical-surgical ICU staffed by a multidisciplinary team including fellowship-trained intensivists, registered nurses, and diversity of allied health professionals. Study eligibility criteria are detailed in Table 1.

TABLE 1.
TABLE 1.:
Study Eligibility Criteria

Eligibility was assessed daily by a research assistant. Consecutive, eligible patients with at least one family member (defined as family, relatives, or friends with previous knowledge of the patient) present were asked by the on-duty nurse if a team member could approach them to discuss a research opportunity. If the patient (or surrogate) and family agreed, informed consent was sought, and they were enrolled in the study as a dyad by a trained patient researcher (B.G.S.). Informed consent was obtained for all individual participants included in the study.

Procedure

Data were collected as long as the patient remained eligible for the study, for a maximum of 5 days. Family members completed demographic questions (e.g., relationship to patient, age, education, ethnicity) at the time of the first assessment. Patient demographic (e.g., age, sex) and clinical characteristics (e.g., admitting diagnosis, severity of illness) were obtained from both questionnaires and eCritical, a bedside clinical information system that collects in real time physiologic and clinical data and has been validated for research purposes (23).

Measures

Reference Standard.

Under the supervision of a board-certified neuropsychiatrist (Z.I.), a team of experienced ICU nurses applied Diagnostic and Statistical Manual for Mental Disorders, Fifth Edition, criteria to determine the presence of delirium once daily for a maximum of 5 days. A clinical diagnosis (or determination) of delirium (while remaining blind to the results of clinical and family-administered tools of delirium) was performed using a standardized assessment sheet and was comprised of a history (data collected from patient, family member(s), bedside nurse, intensivist, and medical record) and physical examination of symptoms and signs related to attention, thinking, concentration, sleep-wake cycle, and agitation (Appendix 1, Supplemental Digital Content 2, http://links.lww.com/CCM/F460). Delirium, along with the current Richmond Agitation-Sedation Scale (RASS) score (24), was recorded as either positive or negative on the reference standard assessment.

Family-Administered Delirium Assessments.

Family-administered delirium assessments (FAM-CAM and Sour Seven) were completed once daily for a maximum of 5 days and a patient researcher (B.G.S.) was available to provide standardized answers to commonly asked questions. Family members were blind to the results of the reference standard delirium assessments and nurses were blind to the results of the family assessments.

The FAM-CAM is an 11-item delirium detection tool designed to be administered by family members. Sudden changes to a patient‘s attention, concentration, orientation, or perception may indicate delirium (18). The FAM-CAM has high sensitivity (88%), specificity (98%), and reliability (kappa = 0.85) in general hospital settings (18). The Sour Seven is a seven-item tool to detect delirium designed to be administered by caregivers (19). Assessed symptoms include reduced attention, altered level of awareness, and disordered thinking. Scores of greater than or equal to 9 out of 18 are indicative of delirium and have 100% specificity and 63.2% sensitivity in hospitalized seniors (19). Established cutpoints for delirium on both the FAM-CAM (positive or negative for delirium) and Sour Seven (≥ 4/18: probable delirium and ≥ 9/18: delirium) were prespecified and employed in the present study. Neither tool requires training to complete, nor does it ask direct questions of the patient (18,19).

Clinical Delirium Assessments.

The ICDSC is completed by bedside nurses once per shift (bid) to assess delirium as part of the standard of care. The ICDSC is an eight-item, delirium assessment tool for use in the ICU (1 point per item); scores of greater than or equal to 4 out of 8 on the ICDSC are indicative of delirium (sensitivity: 99%; specificity: 64%) (3). The CAM-ICU was also employed in the present study (scored dichotomously) to assess its performance compared to the other tools. The CAM-ICU was assessed by trained research assistants bid on all eligible patients. The CAM-ICU (2) is a four-item dichotomous scale of ICU delirium, with a high sensitivity (93–100%) and specificity (98–100%).

Data Analysis

Descriptive analyses were estimated by means and proportions and compared using t tests and tests of proportions. For the FAM-CAM (scored dichotomously), the first reference standard and FAM-CAM conducted “within the same day” were used to evaluate criterion validity. The most severe Sour Seven score (or the first score if all scores were equally severe) was compared with the first reference standard assessment conducted “within the same day.” To determine the additive value of the family assessments to clinical assessment (i.e., ICDSC or CAM-ICU), we also compared any positive ICDSC/CAM-ICU or FAM-CAM in 1 day, and any positive ICDSC/CAM-ICU or Sour Seven (cutpoint of 4) in 1 day, to the reference standard. Measures of diagnostic accuracy (sensitivity, specificity) were calculated for both tools (with accompanying 95% CIs), using the research nurse assessment as the reference standard (all analyzed assessments had complete data). Area under the receiver operating characteristic curve (AUC) estimates were calculated to identify the best performing tools, providing a measure of maximal sensitivity versus specificity. All analyses were conducted in STATA v14.0 (STATA Corp, College Station, TX).

In order to achieve the minimum sensitivity and specificity (95% and 75% [2], respectively) necessary to assess validity with a 10% margin of error, 80% power, and an estimated prevalence of delirium of 60% (2), a sample size of 147 patient and family member dyads was required (25).

Ethical Approval

The study was approved by the Conjoint Health Research Ethics Board at the University of Calgary (Reference number: REB16-2060).

RESULTS

From December 2017 to March 2019, 910 adult patients were admitted to the FMC ICU (Fig. 1). Of 881 dyads screened for eligibility, 196 were approached for consent; the most common reasons for ineligibility were not being expected to remain in the ICU for 24 hours (n = 231, 37.7%) and no family being present (n = 124, 20.2%). One hundred forty-seven dyads with at least 24 hours of data composed the study sample. Further details on the criterion validity of the CAM-ICU, criterion validity of the FAM-CAM and Sour Seven, and ease of use of the tools can be found in the Supplemental Materials (Supplemental Digital Content 1, http://links.lww.com/CCM/F459).

Figure 1.
Figure 1.:
Participant flow diagram.

Patient and Family Member Characteristics

Characteristics of the 147 patient and family member dyads are presented in Table 2. The mean age of patients was 56.1 years (sd, 16.2 yr) and almost half of patients (n = 67, 45.6%) were admitted to the ICU with a medical diagnosis and a median Acute Physiology and Chronic Health Evaluation II score of 20 (interquartile range [IQR], 14–26). Patients who participated in our study did not differ from those who declined participation during the study period on age, sex, and severity of illness (all p > 0.05). In 147 family members, the majority were female (n = 108, 73.5%) with a mean age of 54.3 years (sd, 14.3 yr). Family were most commonly spouses (n = 71, 48.3%), children (n = 34, 23.1%), or parents (n = 26, 17.7%) of patients.

TABLE 2.
TABLE 2.:
Demographic and Clinical Characteristics of 147 Included Patients and Family Members

Reference Standard Findings

Delirium was present in 37.0% (95% CI, 33.7–42.4%) of 508 reference standard assessments conducted during the study period. On the first 147 individually paired assessments, delirium was present in 47.6% (95% CI, 39.6–55.8%) for a mean duration of 3.0 days (sd, 2.0 d) (median, 2.5 d; IQR, 0.8–4.8 d). Based on a reference standard assessment positive for delirium and the RASS score, 23.8% (95% CI, 17.5–31.5%) had mixed delirium, 21.1% (95% CI, 15.2–28.5%) had hypoactive delirium, and 2.0% (95% CI, 0.7–6.2%) hyperactive delirium. The reference standard assessment took between 10 and 15 minutes to complete. Ten percent of reference standard assessments were conducted in duplicate by two independent clinicians to assess inter-rater reliability (kappa = 0.64). Of the 482 reference standard assessments with information on blinding, the assessor remained blind to the patient’s other delirium assessments in 96.5% of cases (n = 465).

Diagnostic Accuracy of the FAM-CAM and Sour Seven

The first family and reference standard assessments were compared for the following analyses. The sensitivity of the FAM-CAM, compared with the reference standard, was 54.1% (95% CI, 45.3–62.7%), with a specificity of 76.8% (95% CI, 70.9–82.1%) (Table 3). For possible delirium (cutpoint of 4) on the Sour Seven, the sensitivity was 72.9% (95% CI, 64.5–80.3%) and specificity was 68.8% (95% CI, 62.2–74.8%), as compared to the reference standard assessment. On the Sour Seven, the sensitivity for delirium (cutpoint of 9) was 51.1% (95% CI, 42.4–59.8%) and specificity 82.8% (95% CI, 77.3–87.4%). The AUC was comparable between the tools: 65.0% (95% CI, 60.0–70.0%) on the FAM-CAM, and 71.0% (95% CI, 66.0–76.0%) for possible delirium (cutpoint of 4) and 67.0% (95% CI, 62.0–72.0%) for delirium (cutpoint of 9) on the Sour Seven (Fig. 2). The time to complete the FAM-CAM and Sour Seven was observed through observation to range from 4 to 7 minutes.

Figure 2.
Figure 2.:
Receiver operating characteristic (ROC) curves for family-administered delirium detection tools compared to the reference standard. FAM-CAM = Family Confusion Assessment Method.

The utility of the family-administered tools as an adjunct to standard of care was assessed by comparing the diagnostic accuracy of the clinician-administered ICDSC and CAM-ICU and with and without the family tools (Table 3). Compared to the reference standard assessment, the ICDSC alone had an AUC of 77.0% (95% CI, 73.0–81.0%) and the CAM-ICU an AUC of 80.0% (95% CI, 76.0–83.0%). There was no significant difference between the AUC of the ICDSC alone and AUCs with the addition of the family-administered tools (FAM-CAM + ICDSC: AUC 79.0%, p = 0.63; Sour Seven [cutpoint of 4] + ICDSC: AUC 75.0%, p = 0.65; Sour Seven [cutpoint of 9] + ICDSC: AUC 79.0%, p = 0.25). There was no difference in the performance of the FAM-CAM + CAM-ICU and the CAM-ICU alone (FAM-CAM + CAM-ICU: AUC 77.0% [95% CI, 73.0–81.0%], p = 0.05) and the Sour Seven (cutpoint of 4) and the CAM-ICU (AUC, 78.0%; 95% CI, 74.0–83.0%; p = 0.16). The AUC for the CAM-ICU alone was greater than the AUC for the Sour Seven (cutpoint of 9) and the CAM-ICU (AUC, 74.0%; 95% CI, 70.0–78.0%; p = 0.002).

TABLE 3.
TABLE 3.:
Criterion Validity of the Delirium Tools Compared to the Reference Standard Assessment

Subgroup Performance of the FAM-CAM and Sour Seven

We assessed the criterion validity of the FAM-CAM and Sour Seven in five subgroups of ICU patients where delirium assessment may be challenging. Sensitivity and specificity in the subgroups (e.g., age, severity of illness, frailty, mechanical ventilation, RASS) showed no differences compared to the overall estimates, although cell sizes were small in many groups (Supplemental Table 1, Supplemental Digital Content 3, http://links.lww.com/CCM/F461).

DISCUSSION

We conducted a diagnostic accuracy study of two family-administered delirium detection tools for use in critically ill patients in the ICU. Delirium was present in over 30% of patients on the first reference standard assessment, reflecting the high burden of this condition. The operating characteristics of the family-administered assessments were fair and lower than the clinician-administered assessments. The diagnostic accuracy of these tools was similar in subgroups at increased risk for delirium, including those over the age of 65, with a high severity of illness, the presence of clinically significant frailty, use of mechanical ventilation, and level of sedation, although small sample sizes in some groups limited the power to detect a difference. Combining family-administered assessments with the ICDSC was not significantly different from ICDSC assessments alone. Combining the family assessments with CAM-ICU was not significantly different (FAM-CAM or Sour Seven [cutpoint of 4]) from or even worse (Sour Seven [cutpoint of 9]) than the CAM-ICU assessments alone.

A balance of sensitivity and specificity in diagnostic accuracy studies needs to be considered. Family delirium detection improved the sensitivity but reduced the specificity compared to the ICDSC and CAM-ICU alone, resulting in lower AUC estimates. Family members in our study were better able to identify delirium (sensitivity’s 51.1–72.9%) than ICU clinicians from The Netherlands not using a standardized approach to screening (sensitivity’s 28.0–34.8%) (15). The lower sensitivity and specificity of the ICDSC (part of standard of care and not this research project) in this study (64%, 90%), compared to the original validation article (3) (99%, 64%), illustrate the tension between conducting a tightly controlled experiment and the pragmatic work of studying delirium in a real-world environment, where diagnosis can be subjective and challenging. Chanques et al (26) asked ICU patients whether they felt delirious during their stay and compared this to a reference standard assessment of delirium, which resulted in a sensitivity of 38%. It is possible that these patients, and the family members in the current study, have a different understanding of delirium than clinicians trained to detect and manage delirium. Future research should explore whether providing education on delirium prior to delirium assessments improves delirium knowledge and the diagnostic accuracy of family-administered tools.

Compared to the initial validation studies of the CAM-ICU (2) and ICDSC (3), the addition of family-administered delirium detection resulted in equivalent or worse measures of diagnostic accuracy in the present study. A systematic review of the diagnostic accuracy of nine studies of the CAM-ICU and four studies of the ICDSC also reported lower pooled sensitivity and specificity than the original validation studies (27). Compared with the original validations of the FAM-CAM and Sour Seven, the diagnostic accuracy of these tools in the current study was reduced. The FAM-CAM was validated in a sample of 52 community-dwelling elderly patients and their caregivers, resulting in a sensitivity of 88% and specificity of 98% compared to the CAM (18). In 80 hospitalized seniors and their informal caregivers, compared to the CAM, the Sour Seven (cutpoint of 4) resulted in a sensitivity of 89.5% and specificity of 90%, while at a cutpoint of 9, the sensitivity was 63.2% and the specificity 100% (19). An exploration of different cutpoints in the current study did not yield improved measures of diagnostic accuracy. ICU patients represent a group of severely ill group patients receiving therapies that make delirium assessment more difficult than in a general hospital setting, such as mechanical ventilation and sedating medications; the assessment of delirium by family members of critically ill adults may have been complicated by these factors. Given the differences in operating characteristics observed between the current study and the initial validation papers, consideration should be given to modifying the FAM-CAM and Sour Seven for the ICU environment to see if diagnostic accuracy can be improved.

In 2013, the Society for Critical Care Medicine published evidence-based guidelines to reduce the persistent impairments to physical, psychiatric, and cognitive functioning experienced by ICU survivors (28); focusing on Pain, Agitation, and Delirium (PAD; recently updated to include Immobility and Sleep [29]) the guidelines address elements of ICU care that transcend diagnosis (2018 guidelines were released following the planning of this study). The Assess, prevent and manage pain; Both spontaneous awakening and breathing trials; Choice of analgesia and sedation; Delirium assess, prevent and managed; Early mobility and exercise; Family engagement and empowerment (ABCDEF) bundle was developed as an evidence-based strategy to implement the PAD guidelines (30). The individual ABCDEF bundle components (31,32) and the bundle as a whole (33,34) are shown to improve outcomes in ICU patients irrespective of ventilator status, sedation level, or disease process. Given the observed feasibility of having family members participate in delirium detection in our study, we suggest that there may be opportunities to explore engaging and empowering families in components of the ABCDEF bundle.

Although not traditionally involved in care, families of ICU patients are frequently at the bedside and want to be engaged (35). Family knows a patient best, may notice subtle changes in a patient’s behavior and functioning before an unfamiliar member of the care team (16) and are proxies to identify acute change in mental status. In other critically ill populations, family involvement in care has resulted in an improvement in symptoms of psychiatric comorbidity post-ICU (36,37). By capacitating them to participate in ICU care, we may improve family adverse outcomes following a patient’s discharge. Given the fair diagnostic accuracy of the current FAM-CAM and Sour Seven in the ICU, this hypothesis requires further investigation using modified versions of the family tools, with outcomes assessed using experimental study designs.

The strengths of this study include the diverse population of critically ill patients from a large, tertiary care medical center. The study design was preregistered and the protocol codesigned by patients, researchers, and clinicians and tested in a pilot study. The standardized assessment of delirium following a detailed protocol, which was developed with experts and users, is an important strength. There are limitations related to the cross-sectional nature of the data, as we were unable to determine incident cases of delirium. This study was conducted at a single center, limiting generalizability; however, the hospital is the largest tertiary care medical center in the region, serving a population of over 1.7 million. Family was not always present at the bedside, which resulted in challenges completing serial assessments over 5 days. In addition, some patients did not have family present (20.2%) and were thus ineligible to participate. Future research should explore a method to impart the potential benefits of family involvement in care in those critically ill patients without family present. Family members in the current study may have misunderstood some of the questions on the tools or may have a different understanding of delirium, which might have affected the diagnostic accuracy. Although family-administered tools had fair operating characteristics, they were not as accurate as clinician-administered tools and provided no significant incremental benefit when used in conjunction. This suggests that family-administered tools may be a viable option when clinical tools are not feasible (e.g., due to limited nursing resources [38]) and exploration is warranted whether there are other potential benefits to engaging families in care.

CONCLUSIONS

As critical care medicine continues to evolve, patients who were once heavily sedated and restrained are being liberated from sedation and mobilized. Family-administered delirium detection has fair but lower diagnostic accuracy than clinical assessments using the ICDSC and CAM-ICU. Engaging and empowering ICU patients’ families to participate in care is feasible and should be further explored.

ACKNOWLEDGMENTS

The Family Confusion Assessment Method is used with permission. We thank you to those who assisted in study recruitment (Israt Yasmeen, Brianna Rosgen, Gwen Knight), Cherri Zhang for assistance with data analysis, the research nurses, and the patients and families who participated.

REFERENCES

1. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (DSM-5). 2013Fifth Edition. Washington, DC, American Psychiatric Association,
2. Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: Validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA 2001; 286:2703–2710
3. Bergeron N, Dubois MJ, Dumont M, et al. Intensive Care Delirium Screening Checklist: Evaluation of a new screening tool. Intensive Care Med 2001; 27:859–864
4. Girard TD, Kress JP, Fuchs BD, et al. Efficacy and safety of a paired sedation and ventilator weaning protocol for mechanically ventilated patients in intensive care (Awakening and Breathing Controlled trial): A randomised controlled trial. Lancet 2008; 371:126–134
5. Pandharipande PP, Pun BT, Herr DL, et al. Effect of sedation with dexmedetomidine vs lorazepam on acute brain dysfunction in mechanically ventilated patients: The MENDS randomized controlled trial. JAMA 2007; 298:2644–2653
6. Pandharipande P, Cotton BA, Shintani A, et al. Prevalence and risk factors for development of delirium in surgical and trauma intensive care unit patients. J Trauma 2008; 65:34–41
7. Guenther U, Popp J, Koecher L, et al. Validity and reliability of the CAM-ICU flowsheet to diagnose delirium in surgical ICU patients. J Crit Care 2010; 25:144–151
8. Salluh JI, Wang H, Schneider EB, et al. Outcome of delirium in critically ill patients: Systematic review and meta-analysis. BMJ 2015; 350:h2538
9. Thomason JW, Shintani A, Peterson JF, et al. Intensive care unit delirium is an independent predictor of longer hospital stay: A prospective analysis of 261 non-ventilated patients. Crit Care 2005; 9:R375–R381
10. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA 2004; 291:1753–1762
11. Pandharipande PP, Girard TD, Jackson JC, et al.; BRAIN-ICU Study Investigators: Long-term cognitive impairment after critical illness. N Engl J Med 2013; 369:1306–1316
12. Brummel NE, Jackson JC, Pandharipande PP, et al. Delirium in the ICU and subsequent long-term disability among survivors of mechanical ventilation. Crit Care Med 2014; 42:369–377
13. Brummel NE, Vasilevskis EE, Han JH, et al. Implementing delirium screening in the ICU: Secrets to success. Crit Care Med 2013; 41:2196–2208
14. Devlin JW, Fraser GL, Joffe AM, et al.; Can delirium Assessments Be Accurately Labelled Investigators g: The accurate recognition of delirium in the ICU: The emperor’s new clothes? Intensive Care Med 2013; 39:2196–2199
15. Spronk PE, Riekerk B, Hofhuis J, et al. Occurrence of delirium is severely underestimated in the ICU during daily care. Intensive Care Med 2009; 35:1276–1280
16. Bigatello LM, Amirfarzan H, Haghighi AK, et al. Effects of routine monitoring of delirium in a surgical/trauma intensive care unit. J Trauma Acute Care Surg 2013; 74:876–883
17. Rosgen B, Krewulak K, Demiantschuk D, et al. Validation of caregiver-centered delirium detection tools: A systematic review. J Am Geriatr Soc 2018; 66:1218–1225
18. Steis MR, Evans L, Hirschman KB, et al. Screening for delirium using family caregivers: Convergent validity of the Family Confusion Assessment method and interviewer-rated Confusion Assessment Method. J Am Geriatr Soc 2012; 60:2121–2126
19. Shulman RW, Kalra S, Jiang JZ. Validation of the sour seven questionnaire for screening delirium in hospitalized seniors by informal caregivers and untrained nurses. BMC Geriatr 2016; 16:44
20. Stevens RD, Nyquist PA. Types of brain dysfunction in critical illness. Neurol Clin 2008; 26:469–486, ix
21. Tonelli MR, Misak CJ. Compromised autonomy and the seriously ill patient. Chest 2010; 137:926–931
22. Krewulak KD, Sept BG, Stelfox HT, et al. Feasibility and acceptability of family administration of delirium detection tools in the intensive care unit: A patient-oriented pilot study. CMAJ Open 2019; 7:E294–E299
23. Brundin-Mather R, Soo A, Zuege DJ, et al. Secondary EMR data for quality improvement and research: A comparison of manual and electronic data collection from an integrated critical care electronic medical record system. J Crit Care 2018; 47:295–301
24. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: Validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med 2002; 166:1338–1344
25. Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform 2014; 48:193–204
26. Chanques G, Ely EW, Garnier O, et al. The 2014 updated version of the Confusion Assessment Method for the Intensive Care Unit compared to the 5th version of the Diagnostic and Statistical Manual of Mental Disorders and other current methods used by intensivists. Ann Intensive Care 2018; 8:33
27. Gusmao-Flores D, Salluh JI, Chalhub RÁ, et al. The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: A systematic review and meta-analysis of clinical studies. Crit Care 2012; 16:R115
28. Barr J, Fraser GL, Puntillo K, et al.; American College of Critical Care Medicine: Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med 2013; 41:263–306
29. Devlin JW, Skrobik Y, Gélinas C, et al. Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU. Crit Care Med 2018; 46:e825–e873
30. Morandi A, Brummel NE, Ely EW. Sedation, delirium and mechanical ventilation: The ‘ABCDE’ approach. Curr Opin Crit Care 2011; 17:43–49
31. Pileggi C, Mascaro V, Bianco A, et al. Ventilator bundle and its effects on mortality among ICU patients: A meta-analysis. Crit Care Med 2018; 46:1167–1174
32. Nassar Junior AP, Besen BAMP, Robinson CC, et al. Flexible versus restrictive visiting policies in ICUs: A systematic review and meta-analysis. Crit Care Med 2018; 46:1175–1180
33. Pun BT, Balas MC, Barnes-Daly MA, et al. Caring for critically ill patients with the ABCDEF bundle: Results of the ICU liberation collaborative in over 15,000 adults. Crit Care Med 2019; 47:3–14
34. Barnes-Daly MA, Phillips G, Ely EW. Improving hospital survival and reducing brain dysfunction at seven California community hospitals: Implementing PAD guidelines via the ABCDEF bundle in 6,064 patients. Crit Care Med 2017; 45:171–178
35. Burns KEA, Misak C, Herridge M, et al.; Patient and Family Partnership Committee of the Canadian Critical Care Trials Group: Patient and family engagement in the ICU. Untapped opportunities and underrecognized challenges. Am J Respir Crit Care Med 2018; 198:310–319
36. Jabre P, Belpomme V, Azoulay E, et al. Family presence during cardiopulmonary resuscitation. N Engl J Med 2013; 368:1008–1018
37. Lautrette A, Darmon M, Megarbane B, et al. A communication strategy and brochure for relatives of patients dying in the ICU. N Engl J Med 2007; 356:469–478
38. Penoyer DA. Nurse staffing and patient outcomes in critical care: A concise review. Crit Care Med 2010; 38:1521–1528; quiz 1529
Keywords:

critical care; delirium; family; intensive care units; patient-centered care; validation studies

Supplemental Digital Content

Copyright © 2020 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine and Wolters Kluwer Health, Inc.