The Reliability and Validity of the Face, Legs, Activity, Cry, Consolability Observational Tool as a Measure of Pain in Children with Cognitive Impairment

Abstract

Pain assessment remains difficult in children with cognitive impairment (CI). In this study, we evaluated the validity and reliability of the Face, Legs, Activity, Cry, Consolability (FLACC) tool for assessing pain in children with CI. Each child’s developmental level and ability to self-report pain were evaluated. The child’s nurse observed and scored pain with the FLACC tool before and after analgesic administration. Simultaneously, parents scored pain with a visual analog scale, and scores were obtained from children who were able to self-report pain. Observations were videotaped and later viewed by nurses blinded to analgesics and pain scores. One-hundred-forty observations were recorded from 79 children. FLACC scores correlated with parent scores (P < 0.001) and decreased after analgesics (P = 0.001), suggesting good validity. Correlations of total scores (r = 0.5–0.8;P < 0.001) and of each category (r = 0.3–0.8;P < 0.001), as well as measures of exact agreement (κ = 0.2–0.65), suggest good reliability. Test-retest reliability was supported by excellent correlations (r = 0.8–0.883;P < 0.001) and categorical agreement (r = 0.617–0.935; κ = 0.400–0.881;P < 0.001). These data suggest that the FLACC tool may be useful as an objective measure of postoperative pain in children with CI.

The difficulty in assessing pain in individuals who cannot verbalize or self-report their pain, such as those with cognitive impairment (CI), continues to pose a significant barrier to effective pain treatment. A recent study found that pain was rarely assessed for children with CI after spine fusion surgery and that there was a significant disparity in analgesic administration and acute pain service consultation between children with and without CI ⁽¹⁾. These findings were consistent with those of a previous study that found significant undertreatment of pain in adult patients with CI compared with cognitively intact adults after hip fracture ⁽²⁾. These studies suggest that individuals with CI may be particularly vulnerable to undertreatment of pain.

Although self-report of pain is considered to be the “gold standard” for pain assessment, children with CI lack the skills necessary to reliably self-report pain location and intensity ⁽³⁾. An investigation demonstrated that patients with CI exhibit more behavioral indicators of pain than those who are cognitively intact ⁽²⁾. Furthermore, several studies have described pain behaviors that are common to individuals with CI ^(4–7), suggesting that behavioral tools may be useful to facilitate the objective measurement of pain in this population. Most recently, Breau et al. ⁽⁷⁾ demonstrated the reliability and validity of an extensive behavioral checklist to identify pain in children with severe CI. However, the length and nature of this checklist limit its utility for routine postoperative pain assessment in clinical settings. Soetenga ⁽⁸⁾ investigated the use of the Wisconsin Children’s Hospital Pain Scale in a small sample of preverbal and nonverbal children and demonstrated reasonable reliability and validity. However, the scoring for this tool is determined by a global rating of behavior (zero to five points), rather than an ordinal ranking of each behavioral category, which may limit the tool’s precision. Furthermore, this study combined preverbal infants with older CI children, making it impossible to determine whether pain assessment in the CI group was more or less reliable than in the infants. Consequently, there remains a need to identify a clinically useful behavioral scale that objectively and reliably measures pain in children with CI. The Face, Legs, Activity, Cry, and Consolability (FLACC) behavioral pain scale (see Table 1) was previously found to have excellent validity and reliability for pain assessment in young, cognitively intact children ⁽⁹⁾. This tool requires scoring each of five behaviors on simple 0–2 ordinal scales, to provide a composite ordinal pain score of 0–10. The purpose of this study was to examine the validity and reliability of the FLACC pain scale as a measure of pain in children with CI. The hypothesis under study was that the FLACC tool is a valid and reliable measure of pain in this population.

Table 1: FLACC Behavioral Pain Assessment Tool

Methods

With approval from the IRB and written informed consent from a parent or legal guardian, children aged 4–18 yr with varying degrees of CI were studied after painful orthopedic or general surgery. Before surgery, the child’s demographic information was obtained, and the child’s developmental and communication levels were determined by review of the medical records and from parent interviews. Each child was evaluated for his or her ability to self-report pain by using either the simple Faces Scale ⁽¹⁰⁾ or a 0–10 numbers scale by using the following methods, adapted from Fanurik et al. ⁽³⁾. Testing was conducted only in children who were deemed able, by parent interview, to perform simple ordinal ranking tests, such as putting blocks in order from smallest to largest. Children were first asked to choose the smallest (or largest) of two blocks presented to them. If this task was accomplished, the child was asked to place three blocks in order from smallest to biggest. Children who were successful in ordinally ranking the blocks were then asked to distinguish the face of the child who is having the most pain (hurt) versus the face of the child who has no pain or the face of the child who is having a little pain. Children who understood numbers were also asked to demonstrate magnitude (“Which is bigger?”) between two and then three numbers and, last, to rank-order three and then five numbers.

After recovery from general anesthesia and before the administration of an IV analgesic, patients were observed and scored for pain behaviors by using the FLACC pain tool. All bedside nurses were experienced in scoring pain with the FLACC tool, as well as a variety of other pain tools. Observations were made while the child was awake and in the presence of a parent or guardian whenever available. The patient’s bedside nurse observed the patient’s behaviors for 2–3 min and assigned a FLACC pain score while the patient was videotaped. Analgesics were administered at the discretion of the bedside nurse in accordance with physician orders. In general, moderate to severe pain was treated with morphine 0.05–0.1 mg/kg, and mild pain was treated with acetaminophen 10–15 mg/kg or ketorolac 0.5 mg/kg. Fifteen to thirty minutes later, patients were observed, videotaped, and scored for pain behaviors by using the same methods. Children who did not receive an analgesic were not observed a second time. All observations were made while the child was awake and alert. During each observation, parents independently, yet simultaneously, recorded a global rating of their child’s pain by using a 0- to 10-cm visual analog scale (VAS). Additionally, for children who successfully completed all tasks through rank-ordering the faces, a self-reported pain score was obtained with the modified Faces Scale. For those who were successful in all testing through rank-ordering of numbers, a 0–10 pain score was obtained.

Videotapes were edited to randomly mix the segments and were later viewed and evaluated by two nurses expert in pediatric pain assessment and in the use of the FLACC. These nurses were blinded to the administration of analgesics and to the bedside nurses’, parents’, and patients’ assessments. Each nurse independently scored the child’s pain during each segment by using the FLACC behavioral tool. These nurses viewed 50 randomly selected video segments on a second occasion 2–3 mo after the first viewing to establish the test-retest reliability of the FLACC.

The FLACC scores provided by the bedside nurse and three independent observers were correlated by using Spearman’s ρ; κ statistics for each of the five categories (Faces, Legs, Activity, Cry, and Consolability) were also determined. Values of κ ≥ 0.40 were accepted as good agreement.

The total FLACC scores of each observer were correlated with the parent VAS pain scores by using Spearman’s ρ. Additionally, bias (i.e., the average difference between parent and nurse scores) and precision (the sd of the difference) were calculated.

Pain scores obtained before and after analgesic administration were compared by using Wilcoxon’s signed rank tests for paired data. The total FLACC scores and categorical scores assigned by the blinded observers at two separate viewings were compared by using Spearman’s ρ and κ statistics.

On the basis of previous studies that demonstrated correlation coefficients of approximately 0.4 between observational and parent ratings, it was determined that a sample of 51 observations would be needed to demonstrate a significant correlation between FLACC scores and parent ratings (α = 0.05; β = 0.10). Eighty children were included to account for potential differences in this population, to ensure parent availability, and to ensure appropriate variability in pain scores from mild to moderate and severe.

Results

Eighty children were recruited into the study; however, one child was never videotaped and was therefore excluded. One-hundred-forty observations were recorded in the remaining 79 children (aged 10.11 ± 4.3 yr), 51% of whom were boys. Ninety percent of the sample was Caucasian, 4% African American, 4% Hispanic, and 3% other. Forty children (51%) had severe spasticity. Parents were present for 113 observations. The blinded observers deemed 19 videotaped segments unusable because of poor quality, discussion of pain scores during the observation, or short duration of the segment. Interrater reliability and construct validity were therefore based on 94 observations.

Table 2 presents the developmental level of the sample and results of the evaluation of the child’s ability to self-report pain. The 13 children who completed all tasks had mild CI. Although 16 and 13 children were deemed able to use the Faces Scale and numbers scale, respectively, only 8 gave pain scores when asked to do so in the postoperative period. Four of these used the Faces Scale, three used a 0–10 numbers scale, and one used words (i.e., “a lot”) to express their pain. These data were insufficient to enable reliable comparisons with parent and FLACC scores.

Table 2: Developmental Evaluation of the Sample

FLACC scores of both the bedside nurse and the blinded nurses correlated significantly with parent scores (r₁₁₃ = 0.651; blinded nurses, r₉₄ = 0.609 and 0.519, respectively;P < 0.001), suggesting good criterion validity (Fig. 1). Parent scores, however, tended to be higher than the FLACC scores assigned by the bedside nurse (bias, 0.59; precision, ±2.3) and blinded nurses (0.51 ± 2.4 and 0.65 ± 2.6, respectively). Furthermore, the blinded nurses’ scores tended to be less than the bedside nurse’s scores (−0.2 ± 1.6 and −0.09 ± 2.4, respectively).

Figure 1: Relationship between Face, Legs, Activity, Cry, Consolability (FLACC) scores and parent visual analog scale (VAS) pain scores. Displayed are the regression line for the correlations and the 95% confidence intervals.

There was a decrease in FLACC scores after the administration of analgesics (5.3 ± 2.8 versus 2.0 ± 2.4 for the bedside nurses’ scores, P < 0.001; 5.1 ± 2.9 versus 2.2 ± 3.0 for the blinded nurses’ scores, P = 0.001), supporting the construct validity of the tool as a measure of pain in this group of children.

There were moderate and significant correlations between observers for total scores (range of r = 0.507–0.778;P ≤ 0.001) and for each of the FLACC categories (r = 0.339–0.826;P ≤ 0.001), with the best correlations in the Face and Cry categories (r = 0.505–0.698, κ = 0.303–0.448; and r = 0.638–0.826, κ = 0.434–0.652, respectively). Measures of exact agreement were acceptable for most comparisons in the Face, Cry, and Consolability categories (35%–94% exact agreement; κ = 0.267–0.652). Many of the comparisons reached significance; however, there was least agreement in the Legs and Activity categories (17%–88%; κ = 0.205–0.477). The best agreement in these two categories was for a score of 0 (70%–88%), whereas there was variable agreement for scores of 1 and 2 (17%–77%). Measures of exact agreement were most consistent between the two blinded observers for all categories (κ values: Face, 0.346; Legs, 0.477; Activity, 0.405; Cry, 0.652; Consolability, 0.555).

Pain scores were coded as mild (0–3), moderate (4–6), and severe (7–10) for further comparisons, because interventions may have differed on the basis of pain intensity. Table 3 presents correlations and measures of agreement for each of these pain severities. There was excellent agreement for mild and severe pain categories and good agreement for moderate pain. Furthermore, pain interventions varied according to the level of pain scored by the bedside nurse. Children with mild pain most often received no analgesic (64%) or non-opioids (i.e., ketorolac or acetaminophen, 18%), whereas those with moderate to severe pain most often received morphine (60%) or diazepam (6%) for muscle spasms.

Table 3: Reliability of the FLACC On the Basis of Severity of Pain

Test-retest reliability was supported by excellent correlations for total FLACC scores for the blinded observers (r = 0.8–0.883;P < 0.001) and excellent correlations and exact agreement for each category (r = 0.617–0.935; κ = 0.400–0.881;P < 0.001).

Discussion

Until recently, little attention has been given to the assessment and treatment of pain in the cognitively impaired. In fact, previous studies of pain assessment and management have virtually excluded this vulnerable population of patients. Despite the recent reports in literature addressing pain assessment in individuals with CI, no studies have produced a reliable and valid, yet simple, tool that can be used in routine clinical practice. Our study demonstrated that the FLACC observational tool is reliable and valid for measuring postoperative pain in children with mild to severe CI.

The inability to obtain self-reported pain scores in many children has prompted the development of observational tools that measure pain behaviors. Although several such tools have been developed ^(9,11–13), only one has included testing in a limited sample of children with CI ⁽⁸⁾. Several studies have, however, yielded qualitative descriptions of pain behaviors that are often exhibited in subjects with CI ^(4–7). In these studies, several pain behaviors have been consistently reported. These include vocal expressions (e.g., moaning, crying, screaming, or yelling), not cooperating, irritability, facial expressions (e.g., frowning, eyes closed tight), activity (not moving or less active), and various bodily expressions (e.g., stiff, spastic, tense, or rigid; flinching or movement of body part away; arched head; or curls up). From these observations, an extensive behavioral checklist was developed by Breau et al. ⁽⁷⁾, which was shown to be reliable and valid for assessing pain in a small group of children with severe CI. Interestingly, the behavioral categories on their checklist are consistent with those on the FLACC, which attests to the content validity of the FLACC.

Although several observational pain tools are available for use with children, many are lengthy and lack the attributes necessary for easy implementation into practice. Such attributes include the relative advantage compared with other tools, the compatibility (degree to which the measure is consistent with the experience of clinicians), and the complexity (the degree of difficulty in understanding or using) ⁽¹⁴⁾. On the basis of these attributes, the FLACC pain tool was developed as a measure of pain behaviors that could be easily learned and assimilated into clinical practice ⁽⁹⁾. This study demonstrated that the FLACC has excellent construct validity, particularly considering the blinded nature of scores, and good correlations with parent VAS scores. The tool also contains reasonable interrater and excellent test-retest reliability in assessing pain in children with CI. The categories of Legs and Activity had poorer agreement than the other categories. This may be partially attributed to limitations imposed by our videotaped observations. The blinded observers were disadvantaged by not having a baseline of the child’s behavior for comparison or the ability to directly assess muscle tension or tone of extremities. Indeed, the bedside nurse’s FLACC scores were better correlated with the parent VAS scores, which may have been a result of better knowledge of the patient’s physical status. Breau et al. ⁽⁷⁾ also found less reliability in their subscales of Body and Limbs categories. Children with CI often have physical disabilities, including spasticity, that may hamper the clinician’s observations of these behaviors in assessing pain. Furthermore, dysmorphic facial features, dystonia, and facial muscle spasms may, in some children, make it difficult to interpret subtle facial expressions of pain. It is therefore important to consider the child’s baseline behaviors when assessing pain indicators. Parents or primary caregivers can assist with interpretation of these behaviors.

Other investigators have suggested that behavioral responses in individuals with CI may be conflicting ^(15,16). Gilbert-MacLeod et al. ⁽¹⁶⁾, for instance, reported that children with CI displayed less vigorous responses to everyday pain events and instigated fewer social responses (e.g., seeking help from adults) compared with nondelayed children under similar circumstances. These investigators suggested that such differences in behaviors may lead to an underestimation of pain in these children by care providers. Data from a study by Biersdorff ⁽¹⁷⁾ suggested that as many as 37% of individuals with CI were hyporesponsive to pain, i.e., had a high pain threshold, had a slow reaction time, or displayed unusual pain behaviors. Such data suggest that pain assessment measures for those with CI should be patient specific or must consider a change from the patient’s baseline status to better interpret behavioral signals for pain ⁽¹⁵⁾. Modifications of existing behavioral tools or, perhaps, adjustment of tools for individual children may, therefore, be necessary to improve the precision of pain assessment in children with CI.

Children with CI often have accompanying physical disabilities that require surgical intervention. It is essential that the pain associated with such procedures be well managed to enhance the child’s comfort and facilitate rehabilitation and recovery. Incorporation of the FLACC pain assessment tool into the postoperative care of children with CI may enhance the objective assessment of pain, thereby facilitating pain management. Refinement of the FLACC by incorporation of specific behaviors from Breau et al.’s checklist may further improve its precision in assessing pain in this challenging population.