Buttes, Paul MSN, RN, CNML; Keal, Ginger BSN, RN, CNML; Cronin, Sherill Nones PhD, RN, BC; Stocks, Laurel MSN, APRN, RN, CCNS; Stout, Cheryl MSN, RN, CENP
Paul Buttes, MSN, RN, CNML, is nurse manager at Baptist Health Louisville, Kentucky.
Ginger Keal, BSN, RN, CNML, is nurse manager at Baptist Health Louisville, Kentucky.
Sherill Nones Cronin, PhD, RN, BC, is professor of nursing at Bellarmine University, Kentucky.
Laurel Stocks, MSN, APRN, RN, CCNS, is nurse practitioner at University of Louisville Healthcare, Kentucky.
Cheryl Stout, MSN, RN, CENP, nursing director at Baptist Health Louisville, Kentucky.
Work was performed at Baptist Health Louisville.
No financial support was received for this study.
The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial companies pertaining to this article.
Address correspondence and reprint requests to: Sherill Nones Cronin, PhD, RN, BC, Bellarmine University, 2001 Newburg Rd, Louisville, KY 40205 (firstname.lastname@example.org).
Background: Effective management of pain begins with accurate assessment of its presence and severity, which is difficult in critically ill patients. The Critical-Care Pain Observation Tool (CPOT) was developed to evaluate behaviors associated with pain and validated primarily with cardiac surgical patients.
Objective: The purpose of this study was to examine reliability and validity of the CPOT in a general population of adult, critically ill patients.
Methods: Using a sample of 75 patients from critical care units of a community hospital, pain was evaluated at 3 times (prerepositioning, during repositioning, and postrepositioning) by 2 evaluators, using 3 different pain scales: CPOT; Faces, Legs, Activity, Cry, and Consolability (FLACC) scale; and Pain Intensity Numeric Rating Scale.
Results: Results indicated that reliability and validity of the CPOT were acceptable. Interrater reliability was supported by strong intraclass correlations (ranging from 0.74 to 0.91). For criterion-related validity, significant associations were found between CPOT scores and both FLACC (0.87-0.92) and Pain Intensity Numeric Rating Scale (0.50-0.69) scores. Discriminant validity was supported by significantly higher scores during repositioning (mean, 1.85) versus at rest (pre mean, 0.60; post mean, 0.65).
Discussion: The CPOT is an acceptable behavioral pain assessment scale for use in the general critical care patient population and is more appropriate for use with adults than the FLACC.
Pain is an important and common symptom in critically ill patients.1 Unrelieved pain in these patients can lead to acute neurohumoral changes, neuronal remodeling, and long-lasting psychological distress.2 In addition, it can place patients at a higher risk of developing a chronic pain syndrome and may impact the patient’s functioning, quality of life, and well-being in the long term.3
The effective management of pain begins with accurate assessment of its presence and severity. However, the accurate assessment of pain is difficult in critically ill patients, as many are unable to self-report the level and character of their pain experience. In these cases, a method of assessment that involves the evaluation of patient behaviors is required.
The Critical-Care Pain Observation Tool (CPOT) is one of the more recently developed behavioral pain assessment tools and has been the subject of a number of validation and feasibility studies.4-6 However, many of these initial studies were performed with cardiac surgical patient populations. Although some recent studies have examined the tool in noncardiac surgical samples,7-9 how the CPOT performs with a more general population, with more diverse clinical diagnoses, remains less clear. Therefore, the purpose of this study was to examine the reliability and validity of the CPOT in a general population of adult, critically ill patients.
This nonrandomized prospective study received approval by the institutional review board. A convenience sample of 75 patients from the critical care units of a community hospital was obtained. Sample size was based on power analysis for statistical testing, using an effect size of 0.4, a power of 0.80, and α of .05. Patients were considered for inclusion in the study if they were 18 years or older; were able to hear, see, and understand English; and displayed no evidence of delirium, as assessed by a negative finding on the Confusion Assessment Method for the Intensive Care Unit. Patients with a history of medical treatment for chronic pain were excluded from the study.
The study procedure mimicked that used by Gelinas et al4 with cardiac surgery patients. Patients who were entered into the study underwent a total of 3 pain assessments (T1-T3) during 1 testing period. T1 was done with the patient at rest. T2 was completed a few minutes later, during a routinely scheduled repositioning of the patient. Positioning has been previously identified as a confirmed nociceptive procedure.10 Finally, T3 was done at recovery, 20 minutes after the positioning procedure. An attempt was made to select subjects at various time intervals from the administration of last pain medication to ensure variability across the range of the scale (ie, to examine its performance at low levels, moderate levels, and high levels of pain).
Patients were independently evaluated by 2 trained raters, using 3 different pain scales. The CPOT, which uses observable physiological and behavioral indicators of pain to make a determination of whether pain is present in the nonverbal patient, contains 4 sections, each with different behavioral categories, including facial expression, body movements, muscle tension, and either compliance with ventilator (for intubated patients) or vocalization (for nonintubated patients). Each section is scored from 0 to 2, for a total score range of 0 to 8.
The Faces, Legs, Activity, Cry, and Consolability (FLACC) scale11 was the method for pain assessment of nonverbal patients in use in the study units at the time of the investigation and was included for comparison purposes. The FLACC is a 5-item pain assessment tool designed to measure postoperative pain in children younger than 7 years. Patients are assessed for the presence and degree of facial expression, leg activity, crying, and consolability. Whereas the tool has been used in practice to assess pain in adult critically ill patients, it has not been adequately tested in this population.
The Pain Intensity Numeric Rating Scale (NRS) is a subjective pain report tool that uses a 0- to 10-point scale, with 0 = no pain present and 10 = worst possible pain. This is an industry standard for pain assessment in cognitively intact individuals. Previous research has demonstrated the measure’s reliability and validity, as well as its sensitivity to change in pain over time.12-14
The order of CPOT and FLACC assessment administration was randomized. The raters’ scores on each were recorded separately and were not revealed until the completion of the study. Upon completion of the CPOT/FLACC assessment, subjects were asked to use the NRS to rate their pain level, either verbally or through manual indication on a printed version.
Characteristics of the sample are displayed in Table 1. Study participants were primarily male (56%) and ranged in age from 23 to 87 years, with a mean of 65.3 years. Most common diagnoses were acute myocardial infarction and gastrointestinal and respiratory disorders.
The CPOT scores ranged from a low of 0 to a high of 4 at T1, from 0 to 7 at T2, and from 0 to 8 at T3. Mean scores were higher during the positioning procedure than during rest or recovery. Mean pain scores on all instruments are shown in the Figure.
Figure. Comparison o...Image Tools
Interrater reliability assessed the degree to which the 2 raters assigned similar CPOT scores to the same patient. It was examined using type C intraclass correlation coefficients. Correlations were statistically significant (P < .001) and moderate to high at all 3 testing times (0.74 [T1], 0.91 [T2], and 0.88 [T3]).
Criterion-related validity assessed the degree to which CPOT scores correlated to other reliable and valid external criteria. This was examined by calculating the relationship between the CPOT and the patients’ self-reports of pain, the criterion standard measurement. Significant associations (P < .001) were found between the CPOT scores and NRS scores at all 3 assessment periods (0.51, 0.69, and 0.50 at T1-T3, respectively). The CPOT scores were also highly correlated with FLACC scores, the assessment method in use in the study units at the time of the investigation (Spearman correlations of 0.92, 0.87, and 0.91 at T1-T3, respectively).
Discriminant validity assessed the degree to which the CPOT was able to differentiate between 2 different pain conditions. This was examined by performing paired t tests between assessments taken at rest (expected low pain levels) and during positioning, expected to elicit higher levels of pain. Mean scores on the CPOT at T1 (preturn) were compared with those at T2 (during turn). Similarly, T2 scores were compared with those at T3 (recovery). The tool’s discriminant validity was supported by significantly higher scores recorded during repositioning (T2; mean, 1.85) versus at rest (preturn mean, 0.60; postturn mean, 0.65) (Table 2).
The results of this study provide further support for the CPOT as an acceptable behavioral pain assessment scale for use in the general critical care patient population. The tool was found to demonstrate strong interrater reliability when used independently by 2 trained observers. Evidence was also found for the tool’s criterion-related and discriminant validity. Furthermore, it was deemed by the study data collectors to be more appropriate for use with adults than the FLACC.
The NRS scores were higher than those obtained with either the CPOT or FLACC at all 3 testing periods. This is consistent with the findings of Gelinas et al4 and suggests that self-reports of pain are only moderately related to pain behaviors. As Stites15 notes, the consistent lack of strong correlation between observational pain tools and patients’ self-reports suggests that behavioral assessment tools may not be useful for establishing the severity of pain.
The CPOT scores did, however, mirror the changes seen in NRS scores from rest period to turning period and back. The mainstay or “criterion standard” of pain assessment remains the patient’s self-report of pain.16 Nevertheless, when a patient is unable to self-report through any communication means, physiologic indicators do give the clinician a baseline by which to compare subsequent evaluations.
Recognized limitations of this study include the lack of random sampling and the use of a limited number of pain observers. Use of the CPOT in actual clinical practice, by a full nursing staff, may affect the tool’s reliability, and this should be tested in future trials.
To date, trials of the CPOT have been reported primarily with critical care populations. However, the tool may also be appropriate for use with other less critically ill nonverbal patients who are unable to communicate an assessment of their pain through more traditional means (eg, pointing to a scale, gesturing, eye blinks, etc), perhaps with some adaptations. Future development of the tool for these populations should be pursued.
Because pain was designated as the fifth vital sign by the American Pain Society, there has been increasing emphasis on the need for reliable and valid pain assessment tools for all patients, both verbal and nonverbal. The CPOT has been carefully developed and validated by a number of researchers, making it one of the most valid and reliable behavioral pain scales available for monitoring pain in adult intensive care unit patients.17 In those critically ill patients who are unable to self-report by any means, and in whom motor function is intact, it is recommended as an appropriate pain assessment method.
The authors thank Rachel Amshoff, BSN, RN; Kristin Ball, BSN, RN; Jeanna Calabrese Varanese, BSN, RN; Deborah Day, BSN, RN; Krista Kirby, BSN, RN; Pennie Lucas, BSN, RN; Christi McKenney, ADN, RN; and Dara Nutt, BSN, RN, CCRN, who assisted with data collection and with teaching the rest of our staff the value of consistent pain assessment.
1. Puntillo K. Pain experiences of intensive care unit patients. Heart Lung. 1990; 19: 526–533.
2. Dunwoody CJ, Krenzischek DA, Pasero C, Rathmell JP, Polomano RC. Assessment, physiological monitoring, and consequences of inadequately treated acute pain. J Perianesth Nurs. 2008; 23: S15–S27.
3. Jenewein J, Moergeli H, Wittmann I, Buchi S, Kraemer B, Schnyder U. Development of chronic pain following severe accidental injury: results of a 3-year follow-up study. Psychosom Res. 2009; 66: 119–126.
4. Gelinas C, Fillion L, Puntillo K, Viens C, Fortier M. Validation of the Critical-Care Pain Observation Tool in adult patients. Am J Crit Care. 2006; 15: 420–427.
5. Gelinas C. Nurses’ evaluations of the feasibility and the clinical utility of the Critical-Care Pain Observation Tool. Pain Manag Nurs. 2010; 11: 115–125.
6. Marmo L, Fowler S. Pain assessment tool in the critically ill post–open heart surgery patient population. Pain Manag Nurs. 2010; 11: 134–140.
7. Vazquez M, Pardavila MI, Lucia M, Aguado Y, Margall MA, Asiain MC. Pain assessment in turning procedures for patients with invasive mechanical ventilation. Nurs Crit Care. 2011; 16: 178–185.
8. Tousignant-Laflamme Y, Bourgault P, Gelinas C, Marchand S. Assessing pain behaviors in healthy subjects using the Critical-Care Pain Observation Tool: a pilot study. J Pain. 2010; 11: 983–987.
9. Paulson-Conger M, Leske J, Maidl C, Hanson A, Dziadulewicz L. Comparison of two pain assessment tools in nonverbal critical care patients. Pain Manag Nurs. 2011; 12: 218–224.
10. Puntillo KA, White C, Morris AB, et al. Patients’ perceptions and responses to procedural pain: results from Thunder Project II. Am J Crit Care. 2001; 10: 238–251.
11. Merkel SI, Voepel-Lewis T, Shayevitz JR, Malviva S. The FLACC: a behavioral scale for scoring postoperative pain in young children. Pediatr Nurs. 1997; 23: 293–297.
12. Scott J, Huskisson E. Graphic representation of pain. Pain. 1976; 2: 175–184.
13. King KM, Tsuyuki R, Faris P, Currie G, Maitland A, Collins-Nakai RL. A randomized controlled trial of women’s early use of a novel undergarment following sternotomy: the Women’s Recovery from Sternotomy Trial (WREST). Am Heart J. 2006; 152: 1187–1193.
14. Lorig K, Ritter PL, Plant K. A disease-specific self-help program compared with a generalized chronic disease self-help program for arthritis patients. Arthritis Rheum. 2005; 53: 950–957.
15. Stites M. Observational pain scales in critically ill adults. Crit Care Nurse. 2013; 33: 68–79.
16. McCaffery M, Pasero C. Teaching patients to use a numerical pain-rating scale. Am J Nurs. 1999; 99: 22.
17. Barr J, Fraser G, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Am J Health Syst Pharm. 2013; 70 (1): 53–58.