The Behavior Pain Assessment Tool for critically ill adults: a validation study in 28 countries

Gélinas, Célinea,*; Puntillo, Kathleen A.b; Levin, Pavelc; Azoulay, Elied

doi: 10.1097/j.pain.0000000000000834
Research Paper
Editor's Choice
Video

Abstract: Many critically ill adults are unable to communicate their pain through self-report. The study purpose was to validate the use of the 8-item Behavior Pain Assessment Tool (BPAT) in patients hospitalized in 192 intensive care units from 28 countries. A total of 4812 procedures in 3851 patients were included in data analysis. Patients were assessed with the BPAT before and during procedures by 2 different raters (mostly nurses and physicians). Those who were able to self-report were asked to rate their pain intensity and pain distress on 0 to 10 numeric rating scales. Interrater reliability of behavioral observations was supported by moderate (0.43-0.60) to excellent (>0.60) kappa coefficients. Mixed effects multilevel logistic regression models showed that most behaviors were more likely to be present during the procedure than before and in less sedated patients, demonstrating discriminant validation of the tool use. Regarding criterion validation, moderate positive correlations were found during procedures between the mean BPAT scores and the mean pain intensity (r = 0.54) and pain distress (r = 0.49) scores (P < 0.001). Regression models showed that all behaviors were significant predictors of pain intensity and pain distress, accounting for 35% and 29% of their total variance, respectively. A BPAT cut-point score >3.5 could classify patients with or without severe levels (≥8) of pain intensity and distress with sensitivity and specificity findings ranging from 61.8% to 75.1%. The BPAT was found to be reliable and valid. Its feasibility for use in practice and the effect of its clinical implementation on patient pain and intensive care unit outcomes need further research.

The new Behavioral Pain Assessment Tool use has shown good reliability and validity for the assessment of pain in ICU patients in 28 countries.

aIngram School of Nursing, Faculty of Medicine, McGill University, Centre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montreal, QC, Canada

bSchool of Nursing, Department of Physiological Nursing, University of California San Francisco (UCSF), San Francisco, CA, USA

cCentre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montreal, QC, Canada

dMedical Intensive Care Unit, Assistance Publique - Hôpitaux de Paris (AP-HP), Hôpital Saint-Louis, Université Paris-Diderot, Paris, France

Corresponding author. Address: Ingram School of Nursing, Faculty of Medicine, McGill University, Centre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montreal, QC, Canada. Tel.: 514-398-6157 or 514-340-8222 ext 4645; fax: 514-340-7592. E-mail address: celine.gelinas@mcgill.ca (C. Gélinas).

Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (www.painjournalonline.com).

Received April 29, 2016

Received in revised form October 06, 2016

Accepted October 27, 2016

Article Outline
Back to Top | Article Outline

1. Introduction

Many patients admitted to the intensive care unit (ICU) are unable to communicate their pain through self-report due to their clinical conditions, altered levels of consciousness (disease-induced or medically induced), and/or being mechanically ventilated and sedated. Because none of these conditions precludes the perception of pain, it is essential that clinicians have valid and reliable pain assessment methods. The use of heart rate and blood pressure changes has been demonstrated to be neither valid nor reliable,3,20,35,37 and guidelines note that vital signs should not be used alone to evaluate pain in ICU patients.4,29

Several behavioral pain assessment tools have been tested in ICU patient populations, with variable degrees of psychometric strength.4 Two of them, the Behavioral Pain Scale (BPS)43 and the Critical-Care Pain Observation Tool (CPOT),22 have demonstrated the strongest psychometric properties,4,26 with recommendations that either be used to assess the presence of pain in adult ICU patients when self-report is not possible. Each has been found to be successfully implemented in ICUs after clinicians are trained,21,49,51 and their use has been related to better pain management and improved clinical outcomes.4,42 However, the BPS and CPOT require interpretation of scores that is more complex than identifying when behaviors are present or absent. They also require more extensive training for clinicians to use them in a reliable fashion.33

Considering these limitations, we created the brief Behavioral Pain Assessment Tool (BPAT), based on the presence/absence of key behaviors, for use in a multinational study of procedural pain in ICU patients.46 The simultaneous use of the BPAT by 2 clinicians was accompanied by the assessments of pain intensity and pain distress in those patients who could self-report. This provided us an opportunity to assess the psychometric properties (ie, reliability and validity) of the new, brief BPAT in relation to pain intensity. We also had the unique opportunity to examine the relationship between the new BPAT and pain distress, the degree of unpleasantness that accompanies the sensation of pain44; a relationship that has rarely been tested.6 Finally, we were able to evaluate the BPAT according to the ICU patient's level of sedation, a new addition to the validation process of behavioral pain assessment tools.

Validation strategies were carefully selected as per the tool's purpose of use, ie, a behavior observation tool to be completed by ICU clinicians for the detection of pain in critically ill adult patients.25,26 The purpose of this study of the behavior items of the BPAT was to examine: (1) interrater reliability of observed behaviors between 2 different clinician raters; (2) discriminant validation of behaviors at rest and during procedures, and according to the patient's sedation level; (3) comparison of behaviors between patients able or not able to self-report; (4) criterion validation of behaviors with the patient's self-report of pain intensity; and (5) convergent validation of behaviors with the patient's self-report of pain distress.

Back to Top | Article Outline

2. Methods

2.1. Study design

This study was a psychometric analysis of the BPAT used in a larger research project named Europain®.46 Physicians and nurses working in the study ICUs in each of the countries volunteered to be coordinators and data collectors during 12 procedures commonly performed in the ICU.

Back to Top | Article Outline

2.2. Participants

Patients were eligible if they were 18 years of age or older, able to speak English or the primary language of the country where they were admitted, and were to undergo at least one of the study procedures as part of their standard care. Exclusion criteria were marked clinical instability, treatment with neuromuscular blocking agents, any condition associated with altered pain perception (eg, Guillain-Barré disease), any condition likely to interfere with behavioral assessments of pain (eg, decerebrate posturing, according to clinical assessment), and/or a definitive or probable diagnosis of delirium as assessed by the ICU clinician.

Back to Top | Article Outline

2.3. Measures

2.3.1. Behavior Pain Assessment Tool

The 8-item BPAT, created for this study, is a shorter version of a previous behavioral checklist developed by the second author (K.A.P.). On the BPAT, there are names and descriptions of 4 facial expressions, 2 verbal responses, and 2 body muscle responses (Fig. 1). Each of the 4 facial expressions was accompanied by a picture drawn digitally by a professional illustrator, using a computerized program. The facial expressions originated from previous research,45 inspired from the Facial Action Coding System.18 The Facial Action Coding System evaluates separate facial muscle contractions which are termed “action units” (AUs). Wince uses AUs 6 and 7 which are also part of grimace. However, grimace includes more AUs, showing a more general contraction of the whole face. The verbal responses include moaning and verbal complaints of pain. The body muscle responses include rigidity of muscles of the extremities and torso and clenched fists. The BPAT was limited to 8 dichotomized behavior items with “present” or “absent” answer choices to provide ease of use for multiple clinicians in the 292 ICUs from 28 countries who participated in this study.

Back to Top | Article Outline

2.3.2. Numeric rating scales of pain intensity and pain distress

Pain intensity and pain distress were measured using 0 to 10 numeric rating scales (NRSs) in patients able to self-report, with higher numbers indicating higher intensity or distress. We included these 2 measures which are known to refer to the sensory (intensity) and affective (distress) components of pain.39 Although they have also been found to be interrelated,34 experimental pain studies showed that pain intensity and pain distress are processed in different areas of the brain.9,30 Construct validation14,32 of NRS scales was supported. Pain intensity and pain distress scales are also recommended as core measures in pain clinical trials.15

Back to Top | Article Outline

2.3.3. Measurement of sedation

The Richmond Agitation Sedation Scale (RASS)53 was used to measure the patient's level of sedation/agitation. The RASS is a 10-point validated sedation scale with 4 levels for agitation, 5 levels for sedation, and 1 level for calm, awake patients. The scale's anchor is centered at 0 (alert and calm). Positive ratings +1 (restless, anxious) to +4 (over combative, violent) indicate agitation. Negative ratings −1 (drowsy, but has sustained awakening) to −5 (unarousable, no response to voice or physical stimulation) indicate sedation. The RASS has high interrater reliability (r = 0.944-0.973) (κ = 0.65-0.80)17,53 and moderate to strong convergent validation with cerebral indicators from the Bispectral index.13,17,40,54

Back to Top | Article Outline

2.4. Data collection packet

A data collection packet was developed in English. The packet was sent to national coordinators in Israel, Dutch-speaking Belgium, and Greece for feedback on clarification of packet contents. The packet was then translated into 12 different languages by bilingual professionals in various countries and back translated by 12 other bilingual translators according to the Brislin model for instrument translation for cross-cultural research.7 English language was used in the remaining countries. A blog that included a video of the BPAT and facial photos was created for training and communication purposes (http://europain.chu-stlouis.fr).

Back to Top | Article Outline

2.5. Ethics procedures

Ethics committee approval for the study was obtained at the study coordinating center in Paris and at the home institution of the principal investigator (K.A.P.). Institutional review board approval which met local legislation criteria, including patient consent requirements, was mandatory for study participation in all ICUs. Failure to obtain this approval (n = 9 countries) or withdrawal after institutional review board approval (n = 2 countries) left 28 of an original 39 countries participating in the study.

Back to Top | Article Outline

2.6. Study procedures

A full description of study procedures such as turning, chest tube removal, and arterial line insertion can be found in Table 1. In short, coordinators selected the procedure(s) to be studied in their ICUs from this predefined list. Patients could be enrolled in 1 or 2 procedures performed on the same day or 2 consecutive days, but not at the same time. Two data collectors (ie, ICU clinicians) were present for each procedure. The person performing the procedure could not be a data collector.

Before the procedure, clinicians were directed to use the BPAT first in the following manner: “immediately before procedure, look at patient for 1 minute. Pay special attention to the patient's face. Mark ALL behaviors that are ‘present’ or ‘absent.’” During the procedure, clinicians were directed to use the BPAT in the following manner: “during the procedure, look at patient for 1 minute when you expect the patient to have the most pain. Pay special attention to the patient's face. Mark ALL behaviors that are ‘present’ or ‘absent.’”

Pain intensity was measured after the BPAT using a 0 to 10 NRS. Before the procedure, patients were asked the following question: “How intense is your pain right now, on this scale, where 0 means no pain and 10 the worst possible pain?” Immediately after the procedure, patients were asked, “how intense was your pain during the procedure, on this scale where 0 = no pain and 10 = worst possible pain?”

Pain distress was measured after the BPAT and pain intensity using a 0 to 10 NRS. Before the procedure, patients were asked the following question: “How distressful (or bothersome) is your pain right now, on this scale, where 0 means no distress and 10 means very distressing?” Immediately after the procedure, patients were asked, “how distressful (or bothersome) was your pain during the procedure, on this scale where 0 = no distress and 10 = very distressing?”

Back to Top | Article Outline

2.7. Data analysis

Data analysis was performed using R statistical software package.48 Descriptive statistics were used to describe the sample, behaviors, and self-reports of pain. Kappa coefficients36 were performed to establish agreement (ie, interrater reliability) between clinicians' observations of the presence or absence of behaviors before and during procedures. Discriminant validation of individual behaviors was examined with multilevel logistic regression models.5,27 Such a model estimates the probability of observing a given behavior as a function of the assessment time (ie, before and during procedures) and sedation level (ie, RASS score). Assessment time and sedation level as well as their interaction were considered fixed effects. The multilevel logistic regression model accounts for correlated residuals due to repeated measurements within the same patient as well as different procedure types (patient and procedure types being random effects). To assess frequencies of behaviors between patients who were able vs unable to self-report pain, we computed 95% confidence intervals for the proportions of observed behaviors before as well as during the procedures. The relationship between the presence of behaviors and patient's self-reports of pain intensity (criterion validation) and pain distress (convergent validation) at each assessment time was examined with ordinary least squares multiple linear regression models.27 In addition, Pearson correlation coefficients were obtained between self-reports of pain intensity and pain distress and the mean number of behaviors at each assessment.

Given that significant moderate correlations were found (>0.50) during the procedure, receiver operating characteristic curve analysis was performed to evaluate the best BPAT cut-point score that would predict mild (ie, ≥1-4), moderate (ie, ≥5-7), and severe (ie, ≥8-10) levels of pain as defined by Paul et al.41 Area under the curve (AUC), a measure of the ability of a scale to adequately classify patients with pain19 was computed, and AUC >0.70, which indicates good discrimination, was expected.31 BPAT thresholds that optimized sensitivity and specificity at different levels of pain reported by patients on the 0 to 10 NRS were also obtained.55 Sensitivity is the true positive rate, which refers to the proportion of patients with a BPAT score above the cut-point and who self-reported experiencing pain at specific NRS thresholds (ie, mild, moderate, or severe). Specificity is the true-negative rate, which refers to the proportion of patients with a BPAT score below the cut-point and who self-reported not experiencing pain at specific NRS thresholds.

Back to Top | Article Outline

3. Results

3.1. Sample description

A total of 192 ICUs in 28 countries of 5 continents (Table 2) participated in the study.46 Of the 5107 procedures in 4080 ICU patients observed for the study, 4812 procedures (94%) in 3851 patients were evaluable. The most common procedure was turning and the least common was wound drain removal (Table 1). Most patients were male (60.8%), and the median (interquartile range) age was 62 (50; 73) years. Among the 3851 patients, more than 37% received mechanical ventilation or had a tracheotomy, but the majority of patients (65.1%) could communicate verbally or by using signs. Median (interquartile range) Sequential Organ Failure Assessment score on the day of the procedure was 3 (2; 6) indicating a low degree of organ dysfunction. The median (interquartile range) RASS score on the procedure day was 0 (−1; 0), indicating an alert and calm state, with a range of +4 to −5. The ICU mortality rate was 10%.

Back to Top | Article Outline

3.2. Description of behaviors and interrater reliability

In each of the study sites, 2 different clinicians independently observed the participants' behaviors using the BPAT at all procedures. The majority of clinicians were females and nurses (Table 3). Close to one-third of clinicians were physicians, and other members of the ICU team such as respiratory therapists and physiotherapists were also involved. Both clinicians mainly observed neutral facial expression on patients at rest before the procedure (Table 4). During the procedure and except for neutral facial expression, all other behaviors increased in frequency. Grimace was the most frequent behavior reported by clinicians at procedure time. Closed eyes were similarly reported at both assessments. Moaning and verbal complaints of pain could only be identified in patients able to vocalize, thus excluding those who were mechanically ventilated or had a tracheotomy. Discordance between clinicians for all behaviors was below 11% before the procedure and below 18% during the procedure.

Moderate (0.43-0.60) to excellent (>0.60) kappa coefficients were obtained (Table 5).36 Except for closed eyes, lower coefficients were reported at rest before the procedure. The lowest kappa levels were found for muscle rigidity, both before and during the procedure. Yet, considering that kappa coefficients were acceptable for all behaviors,36 they were considered present in further analyses as long as one of the clinicians reported them.

Back to Top | Article Outline

3.3. Discriminant validation of behaviors between rest and procedures

Mixed effects multilevel logistic regression models were developed for each of the behaviors. Probability curves for each behavior according to the RASS score both before and during the procedure are shown in Figure 2. Most behaviors were less likely to be present at rest before the procedure vs during the procedure, except for neutral face expression. Closed eyes were more likely to be present at rest in ICU patients with lower vs higher RASS scores, the former indicating increased levels of sedation. Most behaviors including grimace, wince, moaning, verbal complaints, muscle rigidity, and clenched fists were more likely to be present during the procedure regardless of the RASS score.

Table 6 presents estimated coefficients for the time of assessment (ie, before and during the procedure), RASS scores, and their interaction (time × RASS scores). Significant coefficients were found for all behaviors according to the time of assessment showing that, given a constant RASS score of zero, the frequency of behaviors observed was different before vs during the procedure. A negative time coefficient was found for neutral face expression meaning that this behavior was more likely to be present before (ie, at rest) than during the procedure. All other behaviors had positive time coefficients, showing that they were more likely to be present during vs before the procedure.

RASS scores were found to influence the likelihood of observing the presence of all behaviors (Table 6). Except for closed eyes which was more likely to be present in more sedated patients (minus RASS scores), all other behaviors were more likely to be present (ie, positive coefficients) as RASS scores increased. Significant coefficients for interaction effects between time of assessment and RASS scores (ie, Time × RASS) were found for neutral expression, closed eyes, and muscle rigidity. In other words, for these 3 behaviors, the likelihood of observing the behavior as a function of the sedation level changed from before the procedure to during the procedure. Indeed, it can be seen in Figure 2 that the likelihood of neutral expression increases as a function of the RASS score before the procedure, whereas it decreases during the procedure. Similarly, the likelihood of eyes being closed, as a function of the RASS score, decreases much faster while the patients are at rest than during the procedure. Meanwhile, the likelihood of being rigid increases faster during the procedure with each additional RASS unit than it does before the procedure. Interaction coefficients were not found to be significant for grimace, wince, moaning, verbal complaints, and clenched fists. This means that the likelihood of seeing these behaviors was not dependent on the interaction between time and the sedation level. Indeed, and as previously described, these behaviors were consistently more frequent during the procedure and in patients with higher RASS scores, ie, less sedated.

Back to Top | Article Outline

3.4. Discriminant validation of behaviors between patients able or not to self-report

We also compared the proportion of behaviors present in patients able or not to self-report, both before and during the procedure (Table 7). For most of the behaviors, proportions were similar in both groups and at both assessments. At rest before the procedure, proportions were low (ie, ≤0.17) for most behaviors except for neutral face expression and closed eyes. Neutral face expression was more likely to be present in both groups, and closed eyes was more often present in patients unable to self-report.

During the procedure and in both groups (ie, those able vs unable to self-report), proportions of most behaviors increased, except for neutral face expression which was found to decrease. The most frequent behavior observed in both groups was grimace. Other behaviors including wince, closed eyes, and muscle rigidity were also frequently present in both groups. Clenched fists were the least frequently observed behavior and were found equally in both groups. Moaning and verbal complaints were more frequently observed in patients who were able to self-report considering that they were not intubated and mechanically ventilated, allowing them to make verbal sounds. On the other hand, 63% of the patients who were unable to self-report were intubated and mechanically ventilated, preventing them from making any sound. They received scores of 0 (ie, absent) for verbal responses.

Back to Top | Article Outline

3.5. Criterion and convergent validation of behaviors with patient's self-report of pain intensity and pain distress

Multiple linear regression models were used to examine how each of the 8 behaviors predicted the 0 to 10 self-report of pain intensity and pain distress before and during the procedure (Table 8 and Table 9, respectively). At rest before the procedure, the absence of neutral face expression and presence of grimace, wince, closed eyes, moaning, and verbal complaints were significant predictors of both pain intensity and pain distress. Behaviors related to body muscle responses (ie, muscle rigidity and clenched fists) were not found to be significant predictors of pain intensity. However, muscle rigidity was a significant predictor (P = 0.03) of pain distress before the procedure. At rest before the procedure, significant behaviors accounted for 14% and 11% of the total variance of pain intensity and pain distress, respectively. During the procedure, all behaviors, including the absence of neutral face expression, were found to be highly significant predictors of pain intensity and distress, accounting for 35% and 29% of the total variance of pain intensity and distress, respectively.

The total number of behaviors observed (with neutral expression being recoded to “present = zero”) was summed to calculate the total BPAT scores, with the highest possible score being 8. Significant positive correlations were found between the mean number of behaviors and the mean pain intensity and pain distress NRS scores. At rest before the procedure, low positive Pearson correlations were obtained (0.28 and 0.24) for pain intensity and pain distress, respectively (P < 0.001). Pain intensity and pain distress self-reports were highly correlated (r = 0.71, P < 0.001). During the procedure, moderate positive Pearson correlations were found between mean behavior scores and mean pain intensity and pain distress scores (0.54 and 0.49, respectively [P < 0.001]). Again, pain intensity and pain distress self-reports were highly correlated (r = 0.79, P < 0.001).

Considering that moderate positive correlations were found between total BPAT scores and pain intensity and pain distress scores during the procedure, receiver operating characteristic curve analyses were performed to establish the ability of the BPAT to discriminate between patients who reported experiencing pain at specific NRS thresholds. Mild (≥1), moderate (≥5), and severe (≥8) levels of pain intensity and pain distress were used as criteria.41 As can be seen in Table 10, low values of AUC (≤0.60) were found at mild and at moderate levels of pain intensity and pain distress (ie, NRS score ≥1, and ≥5, respectively), indicating poor discrimination of the BPAT, ie, not better than chance.31 Yet, acceptable values of AUC (0.73-0.75) indicating good discrimination of the scale31 were found at severe levels of pain intensity and pain distress (ie, NRS score ≥8). Therefore, the BPAT showed a moderate ability to discriminate patients who reported a severe level of pain (ie, ≥8) from those who reported a pain level <8.

Although a BPAT score >2.5 was found to be the cut-point score that maximized both sensitivity (ie, the true-positive rate) and specificity (ie, the true-negative rate) at mild levels of pain intensity and pain distress (ie, NRS score ≥1), sensitivity and specificity findings were very low (<42%) (Table 10). At moderate and severe levels of pain intensity and pain distress (ie, NRS score ≥5), a BPAT cut-point score >2.5 led to higher sensitivity findings (ie, >71%), but specificity findings remained low (ie, <46%). The best BPAT cut-point score that maximized both sensitivity and specificity was found to be >3.5 at moderate levels of pain intensity and pain distress with sensitivity findings ≥52%, and specificity findings ≥61%. At severe pain levels, a BPAT cut-point score >2.5 was associated with higher sensitivity findings (ie, >90%) but with lower specificity findings (ie, <46%). On the other hand, using a BPAT cut-point score >3.5, sensitivity and specificity findings were higher (61.8%-75.1%) at severe levels of pain intensity and pain distress (ie, NRS score ≥8). In summary and according to AUC sensitivity and specificity findings altogether, the BPAT showed better results in classifying patients with or without severe levels of pain intensity and pain distress and the best BPAT cut-point score was found to be >3.5.

Back to Top | Article Outline

4. Discussion

We conducted a detailed psychometric analysis of a new behavioral pain assessment tool used when adult ICU patients were at rest and during procedures. The BPAT was found to be reliable and valid in this population and context. Its validation in heterogeneous patients in several ICUs from several countries provides a high degree of generalizability to our findings.

Back to Top | Article Outline

4.1. Interrater reliability of behavior observations using the Behavior Pain Assessment Tool

Concordance was high between the 2 clinician raters, ie, at least 89%-82% before and during the procedure, respectively. Thus, considering that it is not feasible to have 2 nurses available for pain behavior observations in the clinical context, BPAT pain assessments can be done by one rater after appropriate training. The highest kappas were for eyes closed which represent a neutral item easily identified by raters. During the procedure, high kappas were obtained for facial expression items (ie, neutral, grimace, wince) and 2 verbal sounds (ie, moaning and verbal complaints). The lowest kappas, at both times, were for muscle rigidity. In this study, muscle rigidity was identified by observation. Although the BPS also directs clinicians to observe upper extremity rigidity,43 with the CPOT,22 muscle rigidity is tested by the clinician while moving the patient's arm. Intensive care unit clinicians need to be trained on testing muscle rigidity because it appears more difficult to assess visually.

Back to Top | Article Outline

4.2. Discriminant validation of Behavior Pain Assessment Tool: influence of time and sedation level

Observation of behaviors was influenced by whether the assessment was preprocedure or procedure, and on the patient's sedation level. At rest, neutral expression was by far the most frequently observed behavior, and almost one-third of patients had their eyes closed during this time. Conversely, most behaviors (except for neutral expression and closed eyes) were more frequently seen during the procedure when an acute reflexive pain response could be expected to occur. Behaviors such as grimace, wince, and muscle rigidity have been demonstrated to be pain-related behaviors in previous studies.20,47

Observation of most behaviors (except for closed eyes) was dependent on the patient's level of sedation, with increase in behavior presence being in patients who were less sedated. The finding about closed eyes being more frequently observed when patients were more heavily sedated is reasonable because sedation often leads to closed eyes. In previous research,24 conscious ICU patients had higher CPOT scores during turning than did unconscious patients, and those who received sedatives had lower CPOT scores. Patients who are sedated are less likely to exhibit behaviors, but this may not mean that they do not have pain. When there is a reason to suspect pain, an intervention should be trialed.29

We demonstrated that behavior items of the BPAT did not discriminate between patients able vs unable to self-report because the proportion of behaviors observed both before and during the procedure was similar for both groups. This finding provides support for the validity of the tool use in nonreporting patients, a situation in which a behavioral pain assessment tool is needed the most.29 There were some differences; however, closed eyes were seen more often in patients unable to self-report and who were more likely to be sedated, and moaning and verbal complaints were more frequently observed during the procedure in patients able to self-report. These findings confirm discriminant validation and make clinical sense. However, the BPAT directions for use may need more specificity. For instance, while “closing eyes” or “eye closure” has been related to the facial expression of pain,45 the terminology “eyes closed” was included in the BPAT. Thus, “closing eyes” should be considered in further research with the BPAT.

Back to Top | Article Outline

4.3. Criterion and convergent validation: Behavior Pain Assessment Tool with the self-reports of pain intensity, and pain distress

Most BPAT behaviors significantly predicted the degree of the patient's self-report of pre-procedural pain intensity. Furthermore, all behaviors significantly predicted the degree of the patient's self-report of procedural pain intensity as well as pain distress. In addition, we found moderate positive Pearson correlations between mean BPAT scores and mean pain intensity and distress scores during the procedure. Similar findings were found for the CPOT and BPS.1,8,16,22,24 Correlations between the behavioral and self-report scale scores were moderate but not high. In actuality, these scales focus on different, but interrelated, pain dimensions: pain intensity (ie, sensory dimension), distress (ie, affective dimension), and behaviors (ie, behavioral dimension; in this case, motor responses to a noxious event).38,39 Although previous research established the relationship between pain intensity and pain behaviors,20,35,47 only one recent study has established the relationship between pain distress (unpleasantness) and behaviors (per CPOT scores), with a positive mild Pearson correlation of 0.31 (P < 0.01) in postoperative cardiac surgery during mediastinal tube removal.6 Our study confirms a moderate correlation between pain distress and behavioral scores during common procedures performed in ICU patients. This moderate correlation demonstrates that affective and behavioral dimensions of pain are interrelated.

Significant positive correlations between the BPAT and NRS mean that their scores move in a linear fashion. However, they are on different scales, so their numbers do not have the same meaning; for example, a score of 7 on the BPAT is not the same as a score of 7 on the NRS. Thus, in clinical practice, the BPAT, like the BPS43 and CPOT,22 must be interpreted differently than the NRS.29 More specifically, NRS scores can be used to identify mild, moderate, and severe levels of pain. With behavioral pain assessment tools, only cut-point scores have been established for the presence of pain (yes/no), In addition, the CPOT was found to better classify the presence of pain when it was reported as moderate to severe by ICU patients.23

Despite positive correlations between the BPAT and pain intensity and distress scores, the total explained variance of the BPAT was no higher than 35%. There may be other behaviors that could be indicative of pain intensity and distress such as moving the hand toward the pain site, touching the pain site, and removing the painful stimulus.20,33 Future research is warranted that would explore other relevant behaviors as possible additions to the BPAT. However, pain is a multidimensional experience,38,39 and observing behaviors does not fully capture all dimensions of pain, although they remain the best option for the assessment of pain in patients unable to self-report.4,29 As also highlighted by Hadjistavropoulos and Craig,28 automatic expressive behaviors included in observational measures are subject to less purposeful distortion than are behaviors dependent upon higher mental processes necessary to self-report pain. Consequently, observational measures of behaviors have clinical utility as indices of pain when the patient's self-report is not available.

Back to Top | Article Outline

4.4. Criterion validation: Behavior Pain Assessment Tool sensitivity and specificity

Using a sum score for the BPAT, with the possible range of 0 to 8, we found that the highest degree of sensitivity and specificity was associated with a BPAT score of 3.5; at this cut-point score, a patient could be having severe pain. That the BPAT appears to better discriminate patients with severe levels of pain intensity and pain distress (AUC > 0.70) is consistent with findings for the CPOT23,24 and BPS.11 Sensitivity was higher than specificity, meaning that the BPAT seems to have a better true-positive rate than true-negative rate. Thus, the BPAT has the ability to adequately classify patients with severe levels of pain, but the BPAT's ability to adequately classify patients not reaching severe levels of pain is not much better than chance (60%). Clinically, when a BPAT score >3.5 is achieved, it is highly likely that the patient is experiencing severe pain. Administration of an opioid, with the possible addition of nonopioid and nonpharmacological strategies, could benefit the patient. Patients with BPAT scores <3.5 may still experience some level of pain, and the use of nonopioid or nonpharmacological strategies could be trialed.

Back to Top | Article Outline

4.5. Limitations

Unlike the BPS and CPOT, the BPAT does not have an item that relates to patient compliance with their ventilator. However, compliance with the ventilator may mean other things than pain such as the presence of secretions and the need for suctioning. Additional items related to body movements, added to the BPAT and tested, may increase the tool's sensitivity. Also, “closed eyes” might have been confused with the action of closing eyes by the observers, and “closing eyes” could be used in further validation of the BPAT.

Another limitation of our study is that we were unable to differentiate patients according to their specific injury and, thus, do not know if mechanism of injury affects validity of the BPAT. Further testing of the BPAT will be required in different ICU patient groups. Finally, this validation was conducted for research purposes, and further validation in daily ICU practice needs to be performed.

Back to Top | Article Outline

4.6. Conclusion

The BPAT was found to be reliable and valid for use in critically ill patients able or not to self-report. Valid behavioral scales are necessary to ensure appropriate assessment of pain and to guide decisions for pain management in this vulnerable population. The use of such scales is recommended in practice guidelines,4,52 and their clinical implementation in the ICU setting has led to improved pain management practices and better patient outcomes.10,12,21,49,51 Implementation studies with the BPAT are needed to determine the effects of its use on ICU pain management practices and patient outcomes. The differences in behavioral expression of pain in an international population are also a relevant topic for further exploration.

Back to Top | Article Outline

Conflict of interest statement

The authors have no conflicts of interest to declare.

Back to Top | Article Outline

Acknowledgements

The study, named Europain, was funded by the 2009 Established Investigator Award from the European Society of Intensive Care Medicine/European Critical Care Research Network and by an academic grant from Hôpital Saint-Louis, Paris, France. The authors gratefully acknowledge all of the national coordinators for the Europain study. They are listed in appendix A as supplemental digital content.

The authors also thank all ICU coordinators who were responsible for the data collection in their units, and all clinicians who were involved in data collection. Finally, the authors thank the ICU patients who agreed to participate in this study.

Back to Top | Article Outline

Supplemental Digital Content

A list of the national coordinators for the Europain study can be found online at http://links.lww.com/PAIN/A394.

Back to Top | Article Outline

Supplemental media

Video content associated with this article can be found at http://links.lww.com/PAIN/A393.

Back to Top | Article Outline

References

[1]. Ahlers SJGM, van der Veen AM, van Dijk M, Tibboel D, Knibbe CAJ. The use of the behavioral pain scale to assess pain in conscious sedated patients. Anest Analg 2010;110:127–33.
[2]. Arbour C, Choinière M, Topolovec-Vranic J, Loiselle C, Puntillo K, Gélinas C. Detecting pain in traumatic brain injured patients exposed to common procedures in the ICU: typical or atypical behaviors? Clin J Pain 2014;30:960–9.
[3]. Arroyo-Novoa CM, Figueroa-Ramos MI, Puntillo KA, Stanik-Hutt J, Thompson CL, White C, Wild LR. Pain related to tracheal suctioning in awake acutely and critically ill adults: a descriptive study. Intensive Crit Care Nurs 2008;24:20–7.
[4]. Barr J, Fraser GL, Puntillo KA, Ely W, Gélinas C, Dasta JF, Davidson JE, Devlin JW, Kress JP, Joffe AM, Coursin DB, Herr DL, Tung A, Robinson BRH, Fontaine DK, Ramsay MA, Riker RR, Sessler CN, Pun B, Skrobik Y, Jaeschke R. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med 2013;41:263–306.
[5]. Bates D, Maechler M, Bolker BM, Walker SC. Fitting linear mixed-effects models using lme4. J Statistical Software 2015;67:1–48.
[6]. Boitor M, Fiola Lachance J, Gélinas C. Validation of the critical-care pain observation tool and vital signs in relation to the sensory and affective components of pain during mediastinal tube removal in postoperative cardiac surgery intensive care unit adults. J Cardiovas Nurs 2016;31:425–32.
[7]. Brislin RW. Back-translation for cross-cultural research. J Cross Cult Psychol 1970;1:187–216.
[8]. Buttes P, Keal G, Cronin SN, Stocks L, Stout C. Validation of the critical-care pain observation tool in adult critically ill patients. Dimens Crit Care Nurs 2014;33:78–81.
[9]. Carlsson K, Andersson J, Petrovic P, Petersson KM, Ohman A, Ingvar O. Predictability modulates the affective and sensory-discriminative neural processing of pain. Neuroimage 2006;32:1804–14.
[10]. Chanques G, Jaber S, Barbotte E, Violet S, Sebbane M, Perrigault PF, Mann C, Lefrant JF, Eledjam JJ. Impact of systematic evaluation of pain and agitation in an intensive care unit. Crit Care Med 2006;34:1691–9.
[11]. Chen YY, Lai YH, Shun SC, Chi NH, Tsai PS, Liao YM. The chinese behavior pain scale for critically ill patients: translation and psychometric testing. Int J Nurs Stud 2011;48:438–48.
[12]. De Jong A, Molinari N, De Lattre S, Gniadek C, Carr J, Conseil M, Susbielles MP, Jung B, Jaber S, Chanques G. Decreasing severe pain and serious adverse events while moving intensive care unit patients: a prospective interventional study (the NURSE-DO project). Crit Care 2013;17:R74.
[13]. Deogaonkar A, Gupta R, DeGeorgia M, Sabharwal V, Gopakumaran B, Schubert A, Provencio JJ. Bispectral Index monitoring correlates with sedation scales in brain-injured patients. Crit Care Med 2004;32:2403–6.
[14]. Downie WW, Leatham PA, Rhind PM, Wright V, Branco JA, Anderson JA. Studies with pain rating scales. Ann Rheum Dis 1978;37:378–81.
[15]. Dworkin RH, Turk DC, Farrar JT, Haythornthwaite JA, Jensen MP, Katz NP, Kerns RD, Stucki G, Allen RR, Bellamy N, Carr DB, Chandlen J, Cowan P, Dionne R, Galer BS, Hertz S, Jadad AR, Kramer LD, Manning DC, Martin S, McCormick CG, McDermott MP, McGrath P, Quessy S, Rappaport BA, Robbins W, Robinson JP, Rothman M, Royal MA, Simon L, Stauffer JW, Stein W, Tollet J, Wernicke J, Witter J. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. PAIN 2005;113:9–19.
[16]. Echegaray-Benites C, Kapoustina O, Gélinas C. Validation of the critical-care pain observation tool in brain surgery patients during common procedures in the intensive care unit. Intensive Crit Care Nurs 2014;30:257–65.
[17]. Ely EW, Truman B, Shintani A, Thomason JWW, Wheeler AP, Gordon S, Francis J, Speroff T, Gantam S, Margolin R, Sessler CN, Dittus RS, Bernard GR. Monitoring sedation status over time in ICU patients: reliability and validity of the richmond agitation-sedation scale (RASS). JAMA 2003;289:2983–91.
[18]. Ekman P, Friesen EP. Manual for the facial action coding system. Palo Alto, CA, USA: Consulting Psychologists Press, 1978.
[19]. Fischer JE, Bachmann LM, Jaeschke R. A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med 2003;29:1043–51.
[20]. Gélinas C, Arbour C. Behavioral and physiological indicators during a nociceptive procedure in conscious and unconscious mechanically ventilated adults: similar or different? J Crit Care 2009;24:628. e7–628. e17.
[21]. Gélinas C, Arbour C, Michaud C, Vaillant F, Desjardins S. Implementation of the critical-care pain observation tool on pain assessment/management nursing practices in an intensive care unit with nonverbal critically ill adults: a before and after study. Int J Nurs Stud 2011;48:1495–504.
[22]. Gélinas C, Fillion L, Puntillo K, Viens C, Fortier M. Validation of the critical-care pain observation tool in adult patients. Am J Crit Care 2006;15:420–7.
[23]. Gélinas C, Harel F, Fillion L, Puntillo KA, Johnston C. Sensitivity and specificity of the critical-care pain observation tool for the detection of pain in intubated adults after cardiac surgery. J Pain Symptom Manage 2009;37:58–67.
[24]. Gélinas C, Johnston C. Pain assessment in the critically ill ventilated adult: validation of the critical-care pain observation tool and physiologic indicators. Clin J Pain 2007;23:497–505.
[25]. Gélinas C, Loiselle CG, Le May S, Ranger M, Bouchard E, McCormack D. Theoretical, psychometric, and pragmatic issues in pain measurement. Pain Manag Nurs 2008;9:120–30.
[26]. Gélinas C, Puntillo KA, Joffe A, Barr JA. A validated approach to evaluating psychometric properties of pain assessment tools for use in nonverbal critically ill adults. Semin Respir Crit Care Med 2013;34:153–68.
[27]. Gelman A, Hill J. Data analysis using regression and multilevel/hierarchical models. Cambridge, United Kingdom: Cambridge University Press, 2006.
[28]. Hadjistavropoulos T, Craig KD. A theoretical framework for understanding self-report and observational measures of pain: a communication model. Behav Res Ther 2002;40:551–70.
[29]. Herr K, Coyne PJ, McCaffery M, Manworren R, Merkel S. Pain assessment in the patient unable to self-report: position statement with clinical practice recommendations. Pain Manag Nurs 2011;12:230–50.
[30]. Hofbauer RK, Rainville P, Duncan GH, Bushnell C. Cortical representation of the sensory dimension of pain. J Neurophysiol 2001;86:402–11.
[31]. Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York: John Wiley and Sons, 2000.
[32]. Jensen MP, Karoly P, Braver S. The measurement of clinical pain intensity: a comparison of six methods. PAIN 1986;27:117–26.
[33]. Joffe A, McNulty B, Boitor M, Marsh R, Gélinas C. Validation of the critical-care pain observation tool in brain-injured critically ill adults. J Cri Care 2016;36:76–80.
[34]. Johnson JE, Rice VH. Sensory and distress components of pain: implications for the study of clinical pain. Nurs Res 1974;23:203–9.
[35]. Kapoustina O, Echegaray-Benites C, Gélinas C. Fluctuations in vital signs and behavioral responses of brain surgery patients in the intensive care unit: are they valid indicators of pain? J Adv Nurs 2014;70:2562–76.
[36]. Landis RJ, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.
[37]. Li D, Miaskowski C, Burkhardt D, Puntillo K. Evaluations of physiologic reactivity and reflexive behaviors during noxious procedures in sedated critically ill patients. J Crit Care 2009;24:472. e9–472. e13.
[38]. McGuire DB. Comprehensive and multidimensional assessment and measurement of pain. J Pain Symptom Manage 1992;7:312–19.
[39]. Melzack R, Casey KL. Sensory, motivational, and central control determinants of pain: a new conceptual model. In: Kenshalo D, editor. The skin senses: Springfield: Chas. C. Thomas, 1968. p. 423–443.
[40]. Ogilvie MP, Pereira BMT, Ryan ML, Gomez-Rodriguez JC, Pierre EJ, Livingstone AS, Proctor KG. Bispectral index to monitor propofol sedation in trauma patients. J Trauma Inj Infect Crit Care 2011;71:1415–21.
[41]. Paul SM, Zelman DC, Smith M, Miaskowski C. Categorizing the severity of cancer pain: further exploration of the establishment of cutpoints. PAIN 2005;113:37–44.
[42]. Payen JF, Bosson JL, Chanques G, Mantz J, Labarere J. Pain assessment is associated with decreased duration of mechanical ventilation in the intensive care unit: a post Hoc analysis of the DOLOREA study. Anesthesiology 2009;111:1308–16.
[43]. Payen JF, Bru O, Bosson JL, Lagrasta A, Novel E, Deschaux I, Lavagne P, Jacquet C. Assessing pain in critically ill sedated patients by using a behavioral pain scale. Crit Care Med 2001;29:2258–63.
[44]. Price DD, Harkins SW, Baker C. Sensory-affective relationships among different types of clinical and experimental pain. PAIN 1987;28:297–307.
[45]. Prkachin KM. The consistency of facial expressions of pain: a comparison across modalities. PAIN 1992;51:297–306.
[46]. Puntillo KA, Max A, Timsit JF, Vignoud L, Chanques G, Robleda G, Roche-Campo F, Mancebo J, Divatia JV, Soares M, Ionescu DC, Grintescu IM, Vasiliu IL, Maggiore SM, Rusinova K, Owczuk R, Egerod I, Papathanassoglou EDE, Kyranou M, Joynt GM, Burghi G, Freebairn RC, Ho KM, Kaarlola A, Gerritsen RT, Kesecioglu J, Sulaj MMS, Norrenberg M, Benoit DD, Seha MSG, Hennein A, Periera FJ, Benbenishty JS, Abroug F, Aquilina A, Monte JRC, An Y, Azoulay E. Determinants of procedural pain intensity in the intensive care unit. The Europain(R) study. Am J Respir Crit Care Med 2014;189:39–47.
[47]. Puntillo KA, Morris AB, Thompson CL, Stanik-Hutt J, White CA, Wild LR. Pain behaviors observed during six common procedures: results from thunder project II. Crit Care Med 2004;32:421–27.
[48]. R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing, 2012.
[49]. Radtke FM, Heymann A, Franck M, Maechler F, Drews T, Luetz A, Nachtigall I, Wernecke KD, Spies CD. How to implement monitoring tools for sedation, pain and delirium in the intensive care unit: an experimental cohort study. Intensive Care Med 2012;38:1974–81.
[50]. Reading AE. A comparison of pain rating scales. J Psychosom Res 1980;24:119–24.
[51]. Rose L, Haslam L, Dale C, Knechtel L, McGillion M. Behavioral pain assessment tool for critically ill adults unable to self-report pain. Am J Crit Care 2013;22:246–55.
[52]. Sauder P, Andreoletti M, Cambonie G, Capellier G, Feissel M, Gall O, Goldran-Toledano D, Kierzek G, Mateo J, Mentec H, Mion H, Rigaud JP, Seguin P. Conférence de consensus commune (SFAR-SRLF) en reanimation: Sédation-analgésie en reanimation (nouveau-né exclu). Réanimation 2008;17:600–12.
[53]. Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O'Neal PV, Keane KA, Tesoro EP, Elswick RK. The richmond agitation-sedation scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med 2002;166:1338–44.
[54]. Yaman F, Ozcan N, Ozcan A, Kaymak C, Basar H. Assesment of correlation between bispectral index and four common sedation scales used in mechanically ventilated patients in ICU. Eur Rev Med Pharmacol Sci 2012;16:660–6.
[55]. Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 1993;39:561–77.
Keywords:

Pain measurement; Behavior; Procedural pain; Validation; Intensive care unit

Supplemental Digital Content

Back to Top | Article Outline
© 2017 International Association for the Study of Pain