Delirium, sometimes referred to as “acute brain failure,” is characterized by altered consciousness with a reduced ability to focus, sustain, or shift attention that develops quickly and fluctuates over the course of the day (1). Delirium is highly prevalent among critically ill patients (2–6) and is associated with greater lengths of ICU and hospital stays, mortality, and cost of care (3–5). Delirium severity has also been associated with adverse patient outcomes including higher risks of nursing home placement and mortality (7–9). Delirium severity scores have been used in clinical settings and for conducting research outside the ICU (10), but their use in the ICU have been restricted mostly to research. Measuring delirium severity in the ICU may not only fulfill a prognostic role for patients with delirium but could also serve as a guide for successful therapeutic interventions.
Delirium Rating Scale-Revised-98 (DRS-R-98) is a widely used delirium severity scale advocated in the ICU setting given its strong psychometric properties (11). Unfortunately, DRS-R-98 use in the ICU has been limited by the inherent difficulty in implementing and interpreting it among mechanically ventilated patients due to the structure of its questions. Hence assessments performed through DRS-R-98 results in missing data, thus decreasing its clinical applicability. It also requires significant administration time and expert judgment on several of its items, thereby further restricting its use by untrained busy clinicians. An ICU delirium severity tool that can overcome these limitations would be ideally suited for the complex, chaotic ICU environment.
Indiana University Center for Aging Research conducted a randomized clinical trial the “Pharmacological Management of Delirium” (PMD) (12) in which every enrolled patient underwent daily sedation, delirium and delirium severity assessments. The presence of these assessments allowed us to undertake the current study with the objective to develop a new delirium severity tool and to assess its reliability and validity.
The Research Compliance Administration of Indiana University-Purdue University approved the study (Protocol no. 1010002428). Informed consent was obtained from patients’ legally authorized representatives.
Patients enrolled in the PMD trial and admitted to the ICU services of three Indianapolis hospitals (Wishard Memorial Hospital [WMH] now known as Eskenazi Health, University Hospital, and Methodist Hospital) from March 2009 to January 2015 were included in the study. PMD is a National Institutes of Health funded clinical trial (12) testing the effectiveness of a multicomponent intervention to reduce delirium duration and severity in the ICU. The details of the trial have been published elsewhere (12). WMH is a 457-bed, university-affiliated, public hospital with three ICU units, an eight-bed surgical ICU (SICU), a 14-bed medical ICU (MICU), and a 29-bed progressive (step-down) ICU. University hospital is a 257-bed tertiary care hospital with 36 MICU and SICU beds. Methodist hospital is an 802-bed tertiary care center with a 65-bed MICU/SICU.
Inclusion and Exclusion Criteria
Inclusion criteria: 1) admitted to the ICUs of WMH, University Hospital, and Methodist Hospital; 2) greater than or equal to 18 years old; and 3) had delirium based on Confusion Assessment Method for the ICU (CAM-ICU) (13). Exclusion criteria: 1) not English speaking; 2) hearing impaired; 3) legally blind; 4) admitted with alcohol intoxication; 5) prisoners; 6) having an axis 1 psychiatric disorder; or 7) pregnant/nursing.
Richmond Agitation-Sedation Scale (RASS) (14) and the CAM-ICU (13) were used to assess patients’ sedation and delirium, respectively. RASS has excellent interrater reliability (interclass correlation coefficient = 0.956; k = 0.73; 95% CI = 0.71–0.75) and high validity (14). CAM-ICU has high criterion validity (sensitivity = 97%, specificity = 98%, accuracy = 98.4%) and high interrater reliability (k = 0.96; 95% CI = 0.92–0.99) (13). Trained research assistants performed bid RASS/CAM-ICU assessments. Patients with a RASS score of –4 (no response to voice, but movement or eye opening to physical stimulation) or –5 (no response to voice or physical stimulation) were ineligible for CAM-ICU assessments. Patients were considered delirious if they had a RASS greater than or equal to –3 (any response to verbal stimulation) and a positive CAM-ICU result, achieved by showing signs of acute change in mental status or fluctuating course, displaying features of inattention, and either disorganized thinking or altered level of consciousness (13). Research assistants administered DRS-R-98 bid to assess delirium severity covering 24-hour period using information from family, nurses, doctors, and medical charts (11). DRS-R-98 is a 16-item scale with 13 severity items; each rated from 0 to 3 with a maximum of 39 points with higher scores indicating greater delirium severity. DRS-R-98 assesses symptoms such as impairments in attention, short- and long-term memory, visuospatial ability and orientation, perceptual and sleep-wake cycle disturbances, abnormalities of language, thought process and content, motor agitation/retardation, and mood lability. It has excellent interrater reliability (intraclass correlation = 0.97) and internal consistency (Cronbach’s α = 0.94) (11). All the research assistants had bachelor’s degree and one was an MD. Dr. Paula Trepacz, the developer of DRS-R-98, trained the research assistants on DRS-R-98 administration. The initial training consisted of didactics followed by as-needed consultations. Afterwards, Dr. Malaz Bosutani, an expert dementia and delirium researcher oversaw the training and quality control for DRS-R-98 administration and scoring.
Development of CAM-ICU-7 Delirium Severity Scale
A 7-point rating scale (0–7) was derived from the CAM-ICU and RASS assessments. The CAM-ICU items were further categorized as shown in Table 1. The scoring method was adapted from a prior study validating CAM-S as a delirium severity instrument outside the ICU setting (10). CAM-ICU-7 maintained the same scoring scheme of CAM-S, but the scores were objectively derived based on the CAM-ICU and RASS items (Table 1). For acute onset, we could only create a binary outcome based on the definition. For inattention, disorganized thinking and altered level of consciousness, we conducted regression models with DRS-R-98 as the dependent variable and chose the cut-off points in each domain to maximize the correlation with DRS-R-98. The final CAM-ICU-7 score ranges from 0 to 7 with 7 being most severe. CAM-ICU-7 scores were further categorized as 0–2: no delirium, 3–5: mild to moderate delirium, and 6–7: severe delirium.
Other Data and Clinical Outcomes
Baseline demographics such as age, gender, and race were collected. Patients’ chronic comorbidities were assessed using the Charlson comorbidity index (15). The severity of acute illness was assessed using the Acute Physiology and Chronic Health Evaluation II scale (16). Length of ICU and hospital stay, and in-hospital mortality data were collected from electronic medical records.
Internal consistency reliability was assessed using Cronbach’s alpha. Pearson correlation coefficient was used to assess correlations between the CAM-ICU-7 and DRS-R-98 in the overall sample as well as in specific subgroups. We used Wilcoxon signed rank tests to compare CAM-ICU-7 severity across known subgroups based on their mechanical ventilation status and age. Logistic regression analysis was used to assess the relationship of in-hospital mortality and discharge status with the CAM-ICU-7 (median, maximum) and DRS-R-98 (median) summary scores as well as delirium duration after adjusting for age, race, gender, Charlson comorbidity index, and severity of illness. For assessments with missing items on the DRS-R-98, we imputed the total DRS-R-98 score if at least 50% of the 13 scale items were completed. We calculated the total score by taking the mean of the completed items and multiplying by the total number of items on the DRS-R-98. Due to the skewed nature of the ICU length of stay (LOS) outcome, we used linear regression with the log (ICU LOS + 1) for associations with CAM-ICU-7 severity measures adjusting for age, race, gender, Charlson comorbidity index, and severity of illness. All analyses were conducted using SAS 9.4 software (SAS Institute, Cary, NC).
We included 518 delirious patients in the study. The mean age of the patients was 60.2 years (SD, 16.1), 55% were females, 45% were African-Americans, and 59% were mechanically ventilated (Table 2).
Internal Consistency Reliability
We found high internal consistency reliability of the CAM-ICU-7 scales (Cronbach’s α = 0.85). We performed sensitivity analyses by examining possible effects of race, gender, age, and mechanical ventilation. Cronbach’s alpha was consistently high in various subgroups: 0.85 in African-Americans, 0.85 in Caucasians; 0.86 in females, 0.85 in males; 0.86 in patients less than 65 years old, 0.83 in patients greater than or equal to 65 years old, and 0.83 among mechanically ventilated compared with 0.82 among nonventilated.
Correlation of CAM-ICU-7 With DRS-R-98 (Construct Validity)
We completed 8,056 RASS and CAM-ICU assessments on 518 patients. Of 8,056 RASS and CAM-ICU assessments, there were 5,120 assessments where DRS-R-98 scores were available (3,709 completed assessments, 1,411 assessments with at least seven completed items where the DRS-R-98 scores were imputed). CAM-ICU-7 scores correlated well with the 5,120 DRS-R-98 scores with a correlation coefficient of 0.64, hence demonstrating construct validity. The correlation coefficient was 0.67 for assessments with all completed DRS-R-98 items and 0.56 for assessments with imputed DRS-R-98 scores. The scores also correlated among mechanically ventilated (r = 0.40) and nonventilated assessments (r = 0.66) although ventilated patients had higher DRS-R-98 missing values. The correlation coefficient was 0.66 for patients less than 65 years old and 0.57 for those greater than or equal to 65 years old. Supplemental Figure 1 (Supplemental Digital Content 1, http://links.lww.com/CCM/C446; legend, Supplemental Digital Content 5, http://links.lww.com/CCM/C450) shows the average CAM-ICU-7 scores for number of items completed on DRS-R-98, demonstrating an inverse relationship between missing DRS-R-98 values and CAM-ICU-7 scores.
CAM-ICU-7 Scores by Clinical Subgroups (Known-Groups Validity)
CAM-ICU-7 scores were higher in assessments among mechanically ventilated patients (median = 5 [interquartile range (IQR) = 2–7]), compared to nonventilated assessments (median = 0 [IQR = 0–3]) (p < 0.001). CAM-ICU-7 scores also increased with increasing age, median: (< 50 yr: 0 [0–3], 50–64 yr: 1 [0–4], ≥ 65 yr: 2 [0–5]) (p < 0.001).
Association of CAM-ICU-7 With Clinical Outcomes (Predictive Validity)
The median CAM-ICU-7 score from each patient during hospitalization was associated with an odds ratio (OR) of 1.47 (95% CI = 1.30–1.66) (area under the curve [AUC] = 0.785) of in-hospital mortality after adjusting for age, race, gender, severity of illness, and chronic comorbidities. Similar results were obtained using the highest CAM-ICU-7 scores (OR = 1.32 [1.11–1.57]) (AUC = 0.731). In contrast, the logistic models using median DRS-R-98 scores or delirium duration provided lower AUCs (DRS-R-98 = 0.727 [p = 0.06]; delirium duration = 0.685 [p = 0.003]) for in-hospital mortality compared with using median CAM-ICU-7. For patients who did not die during the hospitalization (n = 461), higher median CAM-ICU-7 scores during hospitalization were associated with lower odds (OR = 0.8 [95% CI = 0.72–0.9]) (AUC = 0.747) of being discharged home after adjusting for age, race, gender, severity of illness, and chronic comorbidities. Similarly highest CAM-ICU-7 scores were associated with lower odds of discharge to home (OR = 0.78 [0.71–0.86]) (AUC = 0.764). Table 3 shows the odds of in-hospital mortality and discharge to home associated with delirium severity (measured by CAM-ICU-7 and DRS-R-98) and delirium duration. Supplemental Table 1 (Supplemental Digital Content 2, http://links.lww.com/CCM/C447) shows the logistic regression models for median CAM-ICU-7 scores associated with mortality and discharge to home compared with the median DRS-R-98 scores and delirium duration. Supplemental Figure 2 (Supplemental Digital Content 3, http://links.lww.com/CCM/C448; legend, Supplemental Digital Content 5, http://links.lww.com/CCM/C450) shows the receiver operating characteristics for the median CAM-ICU-7 and delirium duration for mortality (2a) and discharge to home (2b), respectively. The median CAM-ICU-7 scores (p = 0.001; partial r = 0.145) and highest CAM-ICU-7 scores (p < 0.001; partial r = 0.327) were also associated with longer length of ICU stay.
Subcategorization of CAM-ICU-7 Scores
We categorized the CAM-ICU-7 scores as 0–2: no delirium, 3–5: mild to moderate delirium, and 6–7: severe delirium. After adjusting for age, race, gender, severity of illness, and chronic comorbidities, patients with severe delirium had significantly higher odds of death (OR = 2.92; 95% CI = 1.17–7.26; p = 0.02) than those with mild to moderate delirium.
Our results suggest that CAM-ICU-7 delirium severity scale is a valid, reliable, and practical delirium severity measure that correlates with the currently available, validated delirium severity scale, the DRS-R-98. Furthermore, delirium severity as measured by the CAM-ICU-7 scores significantly predicts the clinical outcomes of in-hospital mortality, discharge destination, and length of ICU stay. Derived from the widely used RASS and CAM-ICU clinical tools, the CAM-ICU-7 delirium severity scale showed good test characteristics with a higher predictive validity for in-hospital mortality over delirium severity measured through the DRS-R-98 and over delirium duration.
In addition to its association with relevant clinical outcomes, the structure of the CAM-ICU-7 offers certain practical elements that may allow easy incorporation into busy clinical practice. First and foremost is the absence of additional data collection. The data to calculate CAM-ICU-7 are already generated through the RASS and CAM-ICU assessments. The other advantage includes an objective ordinal score that could be followed over time to assess the efficacy of therapeutic measures in controlling delirium symptoms. Our project did not address the questions of implementation of the CAM-ICU-7 into the ICU and the efficacy of interventions to reduce delirium severity. Although with an increase in research to reduce delirium burden in the ICU, incorporation of a valid and practical delirium severity measure such as the CAM-ICU-7 will help in answering the aforementioned observations. Also use of short, practical tools in research studies will produce results that could be quickly and efficiently translatable to the clinical setting.
Currently, measurement of delirium severity in the ICU has been limited to research and is not a standard clinical practice. As mentioned above, this is largely due to the lack of brief, practical delirium severity scales along with absence of efficacious therapeutic agents for delirium symptoms. DRS-R-98 is a valid and reliable instrument for measurement of delirium severity (11) that has been extensively used for research. Although it has strong psychometric properties (11) and covers the breadth of delirium symptoms, its use in the ICU has been limited. This is due to the amount of time required for administration, extensive training requirements, and the ICU specific clinical factors including severity of illness and mechanical ventilation that renders it difficult to complete DRS-R-98 assessments. This was evident in our patient population where mechanically ventilated patients had a large number of missing DRS-R-98 assessments. We found higher CAM-ICU-7 scores among patients with missing DRS-R-98 data, raising the question of underestimating severe delirium when symptoms cannot be assessed due to the inability to complete DRS-R-98 (Supplemental Fig. 1, Supplemental Digital Content 1, http://links.lww.com/CCM/C446; legend, Supplemental Digital Content 5, http://links.lww.com/CCM/C450). We also assessed whether deep sedation may artificially inflate delirium severity as measured through the CAM-ICU-7 but found similar distributions of higher delirium severity among both sedated and agitated assessments (Supplemental Fig. 3, Supplemental Digital Content 4, http://links.lww.com/CCM/C449; legend, Supplemental Digital Content 5, http://links.lww.com/CCM/C450).
Besides DRS-R-98, Delirium Detection Score (DDS) (17), Nursing Delirium Screening Scale (Nu-DESC) (18, 19), and Intensive Care Delirium Screening Checklist (ICDSC) (20, 21) have been used in critical care settings to assess delirium severity. These scales consist of items depicting various symptoms of delirium, which together form an overall score with higher scores representing higher severity (17–21). DDS and Nu-DESC do not capture inattention, one of the cardinal features of delirium, whereas DDS has poor sensitivity, making it less desirable as a delirium screening tool (19). Heavy workflow in the ICU with limited time for evaluation and documentation makes it impractical to use separate scales for assessment of delirium and its severity. The ICDSC captures inattention making it a suitable scale for both delirium identification and severity (20, 21). ICDSC also evaluates sleep-wake cycle disturbances not evaluated by the CAM-ICU-7. Evaluating additional constructs is an advantage, but the information to generate ICDSC scoring is collected over 24 hours, which could lead to recall bias and overestimation of delirium severity (22). CAM-ICU administration time of less than 1 minute (23) allows for more frequent administrations along with direct interaction with patients. This provides a higher reproducibility especially among the mechanically ventilated, as CAM-ICU-7 does not rely on observation alone. Future work comparing CAM-ICU-7 to ICDSC will help clarify which of the two instruments has the greatest utility to measure and follow delirium.
Both ICDSC and Nu-DESC can identify patients with subsyndromal delirium (18, 21), characterized by presence of one or more symptoms of delirium and associated with adverse clinical outcomes (18, 21). A critique of the CAM-ICU is that its dichotomous approach of detecting delirium and absence of ordinal grading of delirium severity symptoms could miss patients with lower delirium severity that may benefit from early interventions. This could potentially be mitigated by use of the CAM-ICU-7 that provides a graded scale for delirium severity assessment. As seen in our study, clinical outcomes vary between mild to moderate delirium and severe delirium. This is in contrast to ICDSC, which plateaus at the threshold of clinical delirium, and does not provide further predictive discrimination. As our data were limited to delirious patients only, we were not able to identify subsyndromal delirium. Studies with both delirious and nondelirious patients will be able to clarify assessment of subsyndromal delirium using the CAM-ICU-7.
Limitations: 1) Inability of the CAM-ICU-7 to capture the entire symptom spectrum of delirium severity, compromising its construct validity. This should be evaluated in the context of feasibility versus validity. CAM-ICU-7 is easy to implement as it takes the same time as CAM-ICU that has been adopted internationally and is the most widely used delirium assessment scale in the ICU. In addition, we believe that CAM-ICU-7 captures the core cognitive constructs of delirium. 2) We compared the CAM-ICU-7 with the DRS-R-98 and not the gold-standard psychiatrist-based assessment of delirium or with a validated ICU delirium severity scale such as the ICDSC. Although not validated specifically in the ICU, the DRS-R-98 has been used in the critical care setting (24) and is highly reliable and valid (11). 3) CAM-ICU-7 and DRS-R-98 assessments were performed by the same research assistants that could have led to a higher correlation. 4) Research assistants performed the DRS-R-98, an instrument originally designed for psychiatrists. 5) The timeframe between identifying delirious patients and study enrollment lasted up to 48 hours in some cases that could have resulted in missing highest severity assessments.
Our study has several strengths. We have a large and diverse sample with half of the patients being females and African-Americans. Presence of both mechanically ventilated and nonventilated patients belonging to different age groups provides known-groups validity to the CAM-ICU-7 assessments. Highly trained research assistants performed bid CAM-ICU and DRS-R-98 assessments. Patients were recruited from three different hospitals with different case-mixes. We gave equal severity weight to both the hyper and hypoactive delirium based on RASS assessments. This mitigates the concern of mislabeling an intervention efficacious when it converts hyperactive agitated delirium to hypoactive delirium. This aspect will be beneficial both for clinical monitoring and conducting future research interventions.
The CAM-ICU-7 delirium severity scale is a valid, reliable, and practical delirium severity measure among ICU patients that can be easily calculated and is associated with meaningful clinical outcomes. This practical tool could improve the ability to correlate delirium severity with long-term complications, including cognitive impairment and healthcare resource utilization. Additionally, the CAM-ICU-7 may facilitate evaluation of delirium severity as an outcome of clinical trials attempting to reduce the burden of delirium in the ICU.
We thank E. Wesley Ely, MD, MPH, and Patrick Monahan, PhD, for critically reviewing and providing feedback on the article.
1. Inouye SK, van Dyck CH, Alessi CA, et al. Clarifying confusion: The Confusion Assessment Method. A new method for detection of delirium
. Ann Intern Med 1990; 113:941–948
2. Lin SM, Liu CY, Wang CH, et al. The impact of delirium
on the survival of mechanically ventilated patients. Crit Care Med 2004; 32:2254–2259
3. Ely EW, Shintani A, Truman B, et al. Delirium
as a predictor of mortality in mechanically ventilated patients in the intensive care unit
. JAMA 2004; 291:1753–1762
4. Ely EW, Gautam S, Margolin R, et al. The impact of delirium
in the intensive care unit
on hospital length of stay. Intensive Care Med 2001; 27:1892–1900
5. Milbrandt EB, Deppen S, Harrison PL, et al. Costs associated with delirium
in mechanically ventilated patients. Crit Care Med 2004; 32:955–962
6. Khan BA, Guzman O, Campbell NL, et al. Comparison and agreement between the Richmond Agitation-Sedation Scale and the Riker Sedation-Agitation Scale in evaluating patients’ eligibility for delirium
assessment in the ICU. Chest 2012; 142:48–54
7. Marcantonio E, Ta T, Duthie E, et al. Delirium
severity and psychomotor types: Their relationship with outcomes after hip fracture repair. J Am Geriatr Soc 2002; 50:850–857
8. Kelly KG, Zisselman M, Cutillo-Schmitter T, et al. Severity and course of delirium
in medically hospitalized nursing facility residents. Am J Geriatr Psychiatry 2001; 9:72–77
9. McCusker J, Cole M, Abrahamowicz M, et al. Delirium
predicts 12-month mortality. Arch Intern Med 2002; 162:457–463
10. Inouye SK, Kosar CM, Tommet D, et al. The CAM-S: Development and validation of a new scoring system for delirium
severity in 2 cohorts. Ann Intern Med 2014; 160:526–533
11. Trzepacz PT, Mittal D, Torres R, et al. Validation of the Delirium
Rating Scale-Revised-98: Comparison with the Delirium
Rating Scale and the cognitive test for delirium
. J Neuropsychiatry Clin Neurosci 2001; 13:229–242
12. Campbell NL, Khan BA, Farber M, et al. Improving delirium
care in the intensive care unit
: The design of a pragmatic study. Trials 2011; 12:139
13. Ely EW, Inouye SK, Bernard GR, et al. Delirium
in mechanically ventilated patients: Validity and reliability of the Confusion Assessment Method for the intensive care unit
(CAM-ICU). JAMA 2001; 286:2703–2710
14. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: Validity and reliability in adult intensive care unit
patients. Am J Respir Crit Care Med 2002; 166:1338–1344
15. Charlson ME, Sax FL, MacKenzie CR, et al. Assessing illness severity: Does clinical judgment work? J Chronic Dis 1986; 39:439–452
16. Knaus WA, Draper EA, Wagner DP, et al. APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818–829
17. Otter H, Martin J, Bäsell K, et al. Validity and reliability of the DDS for severity of delirium
in the ICU. Neurocrit Care 2005; 2:150–158
18. Gaudreau JD, Gagnon P, Harel F, et al. Fast, systematic, and continuous delirium
assessment in hospitalized patients: The nursing delirium
screening scale. J Pain Symptom Manage 2005; 29:368–375
19. Luetz A, Heymann A, Radtke FM, et al. Different assessment tools for intensive care unit delirium
: Which score to use? Crit Care Med 2010; 38:409–418
20. Bergeron N, Dubois MJ, Dumont M, et al. Intensive care delirium
screening checklist: Evaluation of a new screening tool. Intensive Care Med 2001; 27:859–864
21. Ouimet S, Riker R, Bergeron N, et al. Subsyndromal delirium
in the ICU: Evidence for a disease spectrum. Intensive Care Med 2007; 33:1007–1013
22. Tomasi CD, Grandi C, Salluh J, et al. Comparison of CAM-ICU and ICDSC for the detection of delirium
in critically ill patients focusing on relevant clinical outcomes. J Crit Care 2012; 27:212–217
23. Guenther U, Popp J, Koecher L, et al. Validity and reliability of the CAM-ICU flowsheet to diagnose delirium
in surgical ICU patients. J Crit Care 2010; 25:144–151
24. Sharma A, Malhotra S, Grover S, et al. Incidence, prevalence, risk factor and outcome of delirium
in intensive care unit
: A study from India. Gen Hosp Psychiatry 2012; 34:639–646