Validation of a Behavioral Pain Scale in Critically Ill, Sedated, and Mechanically Ventilated Patients

Abstract

Assessing pain in critically ill patients, particularly in nonverbal patients, is a great challenge. In this study, we validated a behavioral pain scale (BPS) in critically ill, sedated, and mechanically ventilated patients. The BPS score was the sum of 3 subscales that have a range score of 1–4: facial expression, upper limb movements, and compliance with mechanical ventilation. Two assessors observed and scored pain simultaneously with the BPS at rest and during painful procedures. The psychometric properties of the BPS that were studied were reliability, validity, and responsiveness. We achieved 360 observations in 30 patients. The BPS was internally reliable (Cronbach α = 0.72). The intraclass correlation coefficient to evaluate inter-rater reliability was high (0.95). Validity was demonstrated by the change in BPS scores, which were significantly higher during painful procedures, with averages of 3.9 ± 1.1 at rest and 6.8 ± 1.9 during procedures (P < 0.001), and by the principal components factor analysis, which revealed a large first-factor accounting for 65% of the variance in pain expression. The BPS exhibited excellent responsiveness, with an effect size ranging from 2.2 to 3.4. This study demonstrated that the BPS can be valid and reliable for measuring pain in noncommunicative intensive care unit patients.

In Brief

IMPLICATIONS: Despite interest in pain assessment in the intensive care unit (ICU), there is no specific pain scale for critical care patients and, particularly, for uncommunicative patients. This study demonstrated that a behavioral scale can be valid and reliable for measuring pain in ICU patients.

Assessment and management of pain in critically ill patients have recently received increased attention (^1–3). Scientific advances in understanding pain mechanisms, multidimensional methods of pain assessment, and analgesic pharmacology have improved pain management practices. However, pain assessment for critically ill patients, especially for nonverbal patients, continues to present a challenge for clinicians and researchers. Critically ill patients are unable to communicate effectively for several reasons, including tracheal intubation, reduced level of consciousness, restraints, sedation, and administration of paralyzing drugs (^4–6).

Pain experts agree that a patient’s self-report of pain intensity is the most valid measure (⁴). Unfortunately, most of the existing scales are designed for use with patients who can respond verbally to assessment commands. Consequently, pain management in nonverbal patients, such as elderly patients with cognitive impairment, is often guided by less precise and wholly untested methods (⁷). Other methods, such as observational pain tools, must be used in a lieu of patients’ self-reports of pain (⁸). The limited amount of data suggests that certain observable behaviors may be valid indicators of pain (^9,10). Pain behaviors can be markers of the existence, intensity, and causes of pain. Indeed, observing pain behaviors is a common method of assessing pain, especially when patients are unable to verbalize.

Nevertheless, no pain scale comprising behavioral indicators has been validated in the intensive care unit (ICU), except the one developed by Payen et al. (¹¹). The latter consisted of a behavioral pain scale (BPS), which was used to assess pain in patients who had undergone thoracic or abdominal surgery or who had been admitted for management of multiple trauma. However, its psychometric properties were insufficiently studied, and it has never been validated in a medical ICU. In addition, validation of any pain tool requires repeated tests of reliability, validity, and responsiveness across samples, settings, and observers. Therefore, the purpose of this prospective study, which sampled from a population of critically ill patients who were sedated and mechanically ventilated, was to validate Payen et al.’s (¹¹) behavioral scale as a measure of pain using psychometric methods.

Methods

The study was performed over a 6-mo period in a 12-bed ICU of the university teaching hospital Ibn Sina, Rabat, Morocco. The hospital ethical committee approved the study protocol, and because this observational study did not require any deviation from routine medical practice, informed consent was not required.

We included patients who were older than 16 yr, mechanically ventilated, sedated, and unconscious. Inclusion criteria were chosen because they precluded the use of an auto evaluation pain scale. Patients who were quadriplegic, receiving neuromuscular blocking medications, or had a peripheral neuropathy were excluded. Exclusion criteria were primarily selected to not include patients whose diseases or medications might compromise expression of the pain behaviors.

To assess pain intensity, we used the BPS described by Payen et al. (¹¹). The BPS is based on a sum of three subscales: facial expression, upper limb movements, and compliance with mechanical ventilation (Table 1). Each subscale is scored from 1 (no response) to 4 (full response). Therefore, possible BPS scores range from 3 (no pain) to 12 (maximum pain).

Table 1: The Behavioral Pain Scale (11)

In addition to the BPS scores, mean arterial blood pressure and heart rate were also collected, which were measured by multimodal monitors. These two hemodynamic variables were collected because previous studies had shown that increased heart rate and increased arterial blood pressure are the most frequent physiological indicators of pain noted by observing nurses (⁹).

The patients’ sedation levels were assessed using the Ramsay scale (¹²). The Ramsay scale rates sedation level on a scale from 1 to 6, with higher levels indicating greater degrees of sedation. This instrument proved satisfactorily reliable and valid (¹³).

Sample characteristics were also recorded, including age, sex, Acute Physiology and Chronic Health Evaluation (APACHE) II score (¹⁴), and diagnosis categories. APACHE II score was calculated for the first 24 h.

For each patient, the BPS scores and the two physiological variables were collected three times a day by the various teams of nurses (morning team, afternoon team, and night team). Each team comprised four nurses and one nurse’s aide. Assessments were made by two evaluators to measure the inter-rater agreement. The two assessors were the nurse and the physician in charge of the patient. They made their assessments simultaneously but without any communication between them. The assessors were not randomized, for reasons of convenience.

Evaluation of the BPS and the physiological variables was made at rest and during painful procedures to appreciate the BPS responsiveness. The two painful procedures chosen were tracheal suction and peripheral venous cannulation. They were selected because their painful characters had been demonstrated in several previous studies (^15–17) and because they were part of the routine care that was normally planned for the patients. No additional interventions or procedures were performed on the patients for the benefit of the study.

The assessments were done in the first 48 h after admission to the ICU. However, for patients who were not being ventilated at the time of their admission but who were ventilated later during their stay, the assessments were made in the first 48 h after mechanical ventilation began.

Twelve physicians and 16 nurses participated in the study. Before the beginning of the study, a training session was provided to teach assessors how to complete BPS, followed by a probation period (15 days), during which the BPS was tested on some patients (n = 4).

Quantitative variables were expressed as mean ± sd, and significance for all statistical tests was set at P = 0.05.

The sample size required for validation of the BPS was established using the precision of a coefficient, such as Cronbach α or Intraclass Correlation Coefficient (ICC) (¹⁸). Thus, with a precision of Cronbach α of 0.90 ± 0.05 as an objective, and for a scale of 3 subscales, it was required to include 25–30 patients in the study.

The validation of an instrument measuring a subjective variable (like pain) requires a comparison with a “gold standard.” Nevertheless, no pain scale has been validated in critically ill patients who were unable to communicate effectively because of the presence of artificial airways or underlying pathologies. Consequently, we had to validate the BPS with indirect arguments, which consisted of checking the psychometric properties of reliability, validity, and responsiveness.

Reliability refers to the lack of measurement error in a scale and includes internal consistency and inter-rater reliability. Internal consistency is an indication of how the items within a scale are interrelated. Cronbach α is one method of assessing internal consistency (¹⁹). A high Cronbach α value reflects high internal consistency. Generally, a value larger than 0.7 is regarded as satisfactory. Inter-rater reliability (or inter-rater agreement) is the ability of a new instrument to obtain similar measures with different assessors. It was assessed using the intraclass correlation coefficient (ICC) (²⁰). Theoretically, the ICC can range from 0 (no agreement) to 1.0 (perfect agreement). Generally, a value larger than 0.8 is regarded as satisfactory (²⁰). The ICC was calculated for the BPS and for each subscale of the BPS separately. A 95% confidence interval (CI) for the coefficient was derived.

Validity is the degree to which an instrument measures what it claims to measure (²¹). Validity was established in three ways: construct validity, change in BPS scores during pain, and factor structure of the BPS.

Construct validity is the extent to which scores on a scale correlate with scores of other measures in predicted ways (²¹). We hypothesized that a significant correlation would be found between the BPS scores and the two physiological variables that were supposed to measure the same concept (pain). We also tested the correlation between the BPS and the Ramsay scale. Spearman nonparametric coefficients were used.

Change in BPS scores was assessed by comparing the BPS scores at rest and after painful procedures. We hypothesized that if the BPS really measures pain, the BPS scores should be much higher during painful procedures than while the patient is at rest. Wilcoxon paired tests (nonparametric) were used.

Furthermore, the factor structure of the BPS was extracted by performing exploratory principal components factor analysis. This is a statistical procedure that enables the underlying dimensions of a scale to be determined (²¹).

Responsiveness refers to an instrument’s ability to detect important changes over time in the concept being measured, even if those changes are small (²²). The magnitude of this property was assessed by the effect size. This coefficient is calculated by dividing the difference between the mean BPS scores at rest and during painful procedures by the sd of the mean scores at rest. The effect size is considered small if it is less than 0.2, moderate if it is near 0.5, and large if it is more than 0.8 (²²).

Results

The various teams assessed 38 patients. However, the assessments of 8 patients could not be included for 3 major reasons: (a) the patient died before the end of the assessments (n = 2), (b) the presence of exclusion criteria (administration of neuromuscular blockade) (n = 3), and (c) an incomplete or incorrect collection of data (n = 3).

Thirty patients were included. The principal patient characteristics are presented in Table 2. Each patient was assessed three times a day (morning, afternoon, and night), by two observers (a physician and a nurse), and at two different times (at rest and during painful procedures). Thus, the various teams achieved 360 observations (30 patients × 2 observers × 2 different times × 3 times per day). Realization of a complete assessment usually required 3–4 min.

Table 2: Principal Patient Characteristics

All patients were sedated with midazolam in continuous infusion except one patient who received thiopental (status epilepticus). The mean amount of midazolam administered was 5.6 ± 2.5 mg/h. The Ramsay scale had an average value of 3.9 ± 1.6. For analgesia, the drug frequently used was morphine, also in continuous perfusion. The mean amount of morphine administered was 3 ± 0.7 mg/h.

Change in physiological variables is shown in Table 3. There was a significant increase in both hemodynamic variables during painful procedures. The amplitude of this increase was 10.7% for heart rate and 2.6% for mean arterial blood pressure.

Table 3: Physiological Variables at Rest and During Painful Procedures

Cronbach α values indicated that the BPS had good internal consistency (Cronbach α = 0.72). ICC to evaluate the inter-rater agreement were high for all subscales of the BPS. For facial expression, ICC was 0.91 (95% CI, 0.88–0.93). For upper limb movements, ICC was 0.90 (95% CI, 0.87–0.92). For compliance with mechanical ventilation, ICC was 0.89 (95% CI, 0.85–0.92). ICC for the total score of the BPS was 0.95 (95% CI, 0.94–0.97). These values showed excellent inter-rater agreement. We also compared the BPS scores obtained by the three teams of caregivers. There was no significant difference (Table 4).

Table 4: Behavioral Pain Scale Scores as Assessed by Three Nursing Teams

No significant correlation was found between the BPS scores and the physiological variables for variability. The correlation coefficients were r = 0.16 (P = 0.13) for heart rate and r = −0.02 (P = 0.84) for mean arterial blood pressure. When the correlation between the BPS scores and Ramsay scale was investigated, as expected, a significant negative correlation emerged (r = −0.432; P < 0.001). The higher the sedation level, the lower the BPS scores (Fig. 1).

Figure 1.: Correlations between the behavioral pain scale (BPS) and the Ramsay scale.

BPS scores obtained at rest and during painful procedures appear in Table 5. The scores were significantly greater during painful procedures than at rest and did not differ between the two categories of painful procedures (tracheal suction and peripheral venous cannulation). Moreover, all subscale scores were significantly higher during painful procedures.

Table 5: Behavioral Pain Scale (BPS) Total Scores and BPS Subscale Scores at Rest and During Painful Procedures, with the Effect Size

Using exploratory principal components factor analysis, we found a large first factor, which accounted for 65% of the variance in pain expression, with strong correlation of the subscales with this factor, including coefficients of 0.90 for facial expression, 0.85 for upper limb movements, and 0.64 for compliance with mechanical ventilation. Table 6 shows the correlation matrix between the subscales of the BPS. The 3 subscales were significantly correlated (all P < 0.001), with a high correlation between facial expression and upper limb movements (r = 0.70) and moderate correlations between compliance with mechanical ventilation and the 2 other subscales (r = 0.40 with facial expression and r = 0.29 with upper limb movements).

Table 6: Correlation Matrix Among the Items of the Behavioral Pain Scale

The effect size for responsiveness was large for the three subscale scores and for the total BPS scores (Table 5). These results showed an excellent responsiveness and, consequently, the excellent ability of the BPS to quantify change in clinical status and detect painful procedures.

Discussion

This validation study showed that the BPS had good psychometric properties when used with critically ill patients. In particular, the BPS showed a high inter-rater reliability (ICC = 0.95) and a satisfactory internal consistency (Cronbach α = 0.72). Validity of the BPS was demonstrated by a significant increase in BPS scores during painful procedures and by principal components factor analysis that identified a large first factor, which accounted for 65% of the variance in pain expression. Furthermore, the BPS exhibited an excellent responsiveness, suggesting that this is a powerful tool to detect the impact of painful stimulation in ICU patients.

Each of our patients was assessed by three teams of nurses to remove a possible bias caused by assessments being made by the same caregivers. Results showed that there was no significant difference among the evaluations made by the three teams.

At rest, theoretically, the BPS scores should be equal to 3, indicating the absence of pain. However, the mean BPS scores, which were near 4, suggest the possibility of preexisting background pain before any procedure was performed. Indeed, our patients, like all ICU patients, are subjected to a multitude of painful constraints, including various tubes (nasogastric and endotracheal), central and arterial lines, wrist restraints, etc. Another explanation could be that the amount of analgesic infusion was insufficient. This fact highlights the need for an instrument that can be used to titrate and adapt analgesia in critical care.

Pain is a stressor that produces a sympathetic stimulation (tachycardia, change in arterial blood pressure, diaphoresis, and change in pupillary size) (^4,23). These physiological variations can help to detect pain among patients with impaired mental status (^4,8,23,24). Puntillo et al. (⁹), in a study of patients having difficulties with verbal communication (mechanically ventilated or having been tracheally extubated less than four hours), showed that the most frequently noted physiological indicators of pain were increased heart rate and increased arterial blood pressure. In our study, heart rate and arterial blood pressure increased significantly during painful procedures, with the increase for heart rate measuring approximately 10%. These results coincide with the observations of clinicians who generally associate pain with a variation of from 10% to 20% in physiological variables (²⁵). However, it is agreed that these physiological indicators lack specificity in the ICU and can be influenced by many medications (vasopressors, β adrenergic blockers, antiarrhythmics, sedative drugs, etc.) and pathological conditions (sepsis states, shock, hypoxia, and fear) (⁴). Moreover, no significant correlation was found among the BPS scores and the two physiological variables in our study. Unfortunately, the absence of an objective measure of pain in ICU patients limited the testing of construct validity. The study of Payen et al. (¹¹) had the same results, and no published study with a sufficient level of scientific evidence has found a correlation among these physiological variables and pain (⁹).

However, the correlation between the BPS and Ramsay scale was negative and significant. The logical direction of the association is the higher the sedation level, the lower the ability to express painful behaviors.

In the present study, the BPS yielded a Cronbach α of 0.72, thus fulfilling Nunnally and Bernstein’s (²⁶) criterion for satisfactory internal consistency. The inter-rater reliability of the BPS was found to be excellent (ICC = 0.95). This indicates that the BPS produces consistent scores from different assessors. Reliability is an essential property when caregivers are numerous, as in the ICU.

The BPS total and subscale scores were significantly higher during the procedures (Table 5). This change in BPS scores testifies to the instrument’s capacity to detect and discriminate pain and provides the evidence that the BPS is a valid measure of pain. It is also important that all of the subscales changed, indicating that they all have the same ability to discriminate pain.

Principal factor analysis revealed that a large first factor was dominant and that the three subscales were strongly related to this factor, which means each of the BPS subscales contributed to the overall pain assessment rating. The largest contributor was facial expression (r = 0.90), followed by upper limb movements (r = 0.85), and then compliance with mechanical ventilation (r = 0.64). Furthermore, the positive significant correlation found among the three subscales demonstrates that they evaluate the same concept, which, in this case, was pain intensity.

This analysis has shown that behavioral indicators can be a valid and reliable measure of pain. Few studies have evaluated pain behaviors in the ICU (^9,10,25). The most recent one (¹⁰) identified specific procedural pain behaviors such as grimacing, rigidity, wincing, shutting of eyes, verbalization, and clenching of fists. But in that study, the patients were awake and could measure their pain with a numeric rating scale. In fact, facial expression, which contributed most to the pain rating in our study, is a sign found in various works measuring both acute and chronic pain (^25,27,28). Prkachin (²⁷) has suggested that four facial actions carry the bulk of facial information about pain: lowering the brow, tightening and closing of the eyelids, wrinkling of the nose, and raising the upper lip. He has also provided evidence of the existence of a universal facial language of pain. The facial scales, which are especially useful for measuring pain in infants and children, highlight the value of this type of signal (^4,23,29). Pediatric scales also rely on upper limb movements as a measure of pain (^23,29). In our study, upper limb movements contributed as much as facial expression to the pain rating. Compliance with mechanical ventilation, adapted from the Comfort scale (¹¹), had a moderate but effective contribution to pain assessment. The reason could be that this subscale might be affected by some factors unrelated to pain, such as hypoxemia, bronchospasm, and mucous plugging, which can lead to coughing and some fighting of the ventilator.

In addition to these psychometric properties, the BPS showed good feasibility, in as much as the average time of assessment was only four minutes. The short time required will make the BPS suitable for everyday clinical use.

This study has two limitations. First, one aspect of the validation process has not been addressed, namely the criterion validity (validity of the BPS in comparison with another validated pain scale). We could have compared the BPS to subjective rating of the level pain by an independent rater (a nurse) on a visual analog scale (VAS). However, apart from the BPS, no other validated instrument has been developed to measure the level of pain in mechanically ventilated ICU patients, and the VAS has never been validated in such patients. In addition, a number of studies have found that from 35% to 55% of nurses under-rate patient pain when using the VAS (⁴). This precludes any analysis of criterion validity in which the new instrument would be compared to a reference instrument.

The second limitation of our study is that the sample of critical care patients observed was small. Future studies will have to include more patients.

We conclude that the present study provides evidence that the BPS has good psychometric properties. This instrument might prove useful to measure pain in uncommunicative critically ill patients and to evaluate the effectiveness of analgesic treatment and adapt it. Further studies are required to determine whether the use of this scale can really improve management of pain in the critical care setting.

The authors gratefully acknowledge all the nurses and physicians who participated in this study, Dounia Benzarouel for her assistance with data collection, and Younès Lahrech and Khalil Zakari for their help during the writing of this manuscript.