Share this article on:

The Use of the Behavioral Pain Scale to Assess Pain in Conscious Sedated Patients

Section Editor(s): Takala, JukkaAhlers, Sabine J. G. M. MSc*; van der Veen, Aletta M. MScN; van Dijk, Monique PhD; Tibboel, Dick MD, PhD; Knibbe, Catherijne A. J. Pharm D, PhD*‡

doi: 10.1213/ANE.0b013e3181c3119e
Critical Care and Trauma: Research Reports

BACKGROUND: Assessing pain in mechanically ventilated critically ill patients is a great challenge. There is a need for an adequate pain measurement tool for use in conscious sedated patients because of their questionable communicative abilities. In this study, we evaluated the use of the Behavioral Pain Scale (BPS) in conscious sedated patients in comparison with its use in deeply sedated patients, for whom the BPS was developed. Additionally, in conscious sedated patients, the combination of the BPS and the patient-rated Verbal Rating Scale (VRS-4) was evaluated.

METHODS: We performed a prospective evaluation study in 80 nonparalyzed critically ill adult intensive care unit patients. Over 2 mo, nurses performed 175 observation series: 126 in deeply sedated patients and 49 in conscious sedated patients. Each observation series consisted of BPS ratings (range 3–12) at 4 points: at rest, during a nonpainful procedure, at retest rest, and during a routine painful procedure. Patients in the conscious sedated state also self-reported their pain using the 4-point VRS-4.

RESULTS: BPS scores during painful procedures were significantly higher than those at rest, both in deeply sedated patients (5.1 [4.8–5.5] vs 3.4 [3.3–3.5], respectively) and conscious sedated patients (5.4 [4.9–5.9] vs 3.8 [3.5–4.1], respectively) (mean [95% confidence interval]). For both groups, scores obtained during the nonpainful procedure and at rest did not significantly differ. There was a strong correlation between nurses' BPS ratings and conscious sedated patients' VRS-4 ratings during the painful procedure (rs = 0.67, P < 0.001). At rest and during nonpainful procedures, 98% of the observations were rated as acceptable pain (VRS 1 or 2) by both nurses and patients. During painful procedures, nurses rated the pain higher than patients did in 16% of the observations and lower in 12% of the observations.

CONCLUSION: The BPS is a valid tool for measuring pain in conscious sedated patients during painful procedures. Thus, for noncommunicative and mechanically ventilated patients, it may be regarded as a bridge between the observational scale used by nurses and the VRS-4 used by patients who are able to self-report pain.

Published ahead of print November 6, 2009 Supplemental Digital Content is available in the text.

From the Departments of *Clinical Pharmacy, and †Anaesthesiology and Intensive Care, St. Antonius Hospital, Nieuwegein; and ‡Department of Pediatric Surgery, Erasmus MC-Sophia Children's Hospital, Erasmus University Medical Center, Rotterdam, The Netherlands.

Accepted for publication September 14, 2009.

Published ahead of print November 6, 2009

Address correspondence and reprint requests to Sabine J.G.M. Ahlers, MSc, Department of Clinical Pharmacy, St. Antonius Hospital, PO Box 2500, 3440 EM Nieuwegein, The Netherlands. Address e-mail to

Many critically ill patients in the intensive care unit (ICU) suffer from pain,1–3 notably those on mechanical ventilation.4,5 From 35% to 55% of nurses have been reported to underrate patients' pain,6,7 and a current practices study revealed that the observed rates of assessment during procedural pain in mechanically ventilated patients remain below 40%.8 Researchers have recognized that pain and inadequate pain relief are major causes of physiological adversity and emotional stress.9–11 Therefore, it would seem important to achieve effective management of analgesia, first by measuring pain in a valid and reliable manner.

Various pain scales are available, but there is insufficient evidence of their reliability in the diverse ICU population. The Society of Critical Care Medicine recommends self-reporting by communicative patients using the numerical rating scale (NRS, range 0–10).7 This scale requires a certain level of comprehension, so one may opt for an alternative, the 4-point Verbal Rating Scale (VRS-4), which has shown good reliability and validity.12 Postoperative patients even prefer the VRS-4 over the NRS because of its ease of use.12

The observational Behavioral Pain Scale (BPS, range 3–12), applied by nurses, has been validated in deeply sedated, mechanically ventilated patients.13,14 It is composed of 3 subscales: facial expression,1–4 movement of the upper limbs,1–4 and compliance with mechanical ventilation.1–4,13 The BPS reflects objective visible behavior at 1 specific time point, whereas the NRS represents a global impression of pain, including several contextual factors during a longer time period.15 Gélinas et al.16 developed the Critical Care Pain Observation Tool (range 0–8). Based on the BPS, the Critical Care Pain Observation Tool has not yet been tested among different critical care populations and requires additional validation.17

Apart from communicative and deeply sedated patients, a third group can be identified, i.e., conscious sedated mechanically ventilated patients. Current ICU practice strives to restrict sedation to a conscious level whenever possible, in agreement with the landmark report18 that showed that ventilated patients benefit from daily interruption of sedative infusions. Ventilation could be stopped earlier in these patients, resulting in shorter ICU stays, and they showed no adverse psychosocial outcomes.19

Self-reporting using the NRS or VRS-4 may be complicated or unreliable in these patients because of their temporarily limited capacities of abstraction and concentration, and lack of comprehension.7,11 Furthermore, the BPS has been validated only in deeply sedated and noncommunicative patients.

For this growing group of conscious sedated patients, an observational pain scale such as the BPS, which can be used by the nurse, can add value to VRS-4 scores, because patients' self-reporting may be complicated and/or unreliable. Therefore, we designed a study to compare use of the BPSnurse in conscious sedated patients and in deeply sedated patients, for whom the BPS was developed. Additionally, in conscious sedated patients, the combination of the BPSnurse and the patient-rated VRS-4 was evaluated.

Back to Top | Article Outline



A prospective, observational study was performed in a 30-bed surgical/medical ICU in a teaching hospital in Nieuwegein, The Netherlands. The Medical Ethical Committee of the St. Antonius Hospital approved the study protocol and waived the need for informed consent because the observational study design and pain measurements are considered as standard care.

Back to Top | Article Outline

Patients and Classifications

During the 2-mo study period, all patients admitted to the ICU were evaluated for inclusion in the study once a day (between 8:00 am and 12 noon). ICU patients who were 18 yr and older, sedated irrespective of sedation depth, and ventilated for at least 8 h before assessment were eligible for inclusion. Patients who received neuromuscular blocking medications or muscle-paralyzing drugs, who were unconscious after resuscitation, quadriplegic, had a critical illness (poly) neuropathy, or had an epidural catheter, were excluded.

Included patients were classified as “sedated” or “conscious sedated” on the specific day. Sedated patients were defined as patients who were not able to communicate during all 4 consecutive assessments (at rest, during nonpainful procedures, at retest rest, and during painful procedures) on that particular day. Conscious sedated patients were defined as patients who were able to communicate during at least part of the assessment. Patients could be included on multiple days, an approach also used in the first BPS validation study in nonresponsive critically ill patients.13

Eighty patients were included during the 2-mo study period. Fifty patients were classified as sedated on all study days, 17 as conscious sedated on all study days, and 13 as either sedated or conscious sedated on different days in the study period.

Back to Top | Article Outline

Pain Measurement Instruments


The BPS is an observational pain scale, preferably applied by the attending nurse. It has been validated for use in deeply sedated, mechanically ventilated patients.13,20 Easy to use and well accepted by nurses, the BPS contains 3 subscales: facial expression, upper limb movements, and compliance with mechanical ventilation (Table 1). Each subscale is scored from 1 (no response) to 4 (full response). Therefore, BPS scores range from 3 (no pain) to 12 (maximal pain).13 A BPS score of 6 or higher is considered to reflect unacceptable pain.2

Table 1

Table 1

Back to Top | Article Outline


The VRS-4 is a 4-point verbal rating scale (range 1–4) used for patient self-reporting. It was adapted from the Verbal Graphic Scale,21 which includes 4 categories: 1) free of pain (NRS 0), 2) mild pain (NRS 1–3), 3) moderate pain (NRS 4–6), and 4) severe pain (NRS 7–10). This shorter version was used because conscious sedated patients may temporarily lack full comprehension of the more complex 11-point NRS. Unacceptable pain using the 11-point NRS is defined as NRS >3 (moderate pain and severe pain),6,7 thus unacceptable pain using the 4-point VRS was defined as a score of 3 or 4.

In this study, the “BPSnurse” is defined based on a BPS rating by the attending nurse. The “VRS-4patient” is defined by the VRS-4 rating by the patient.

Back to Top | Article Outline

Study Procedures and Intervention

Pain was assessed during 2 routine nursing procedures. One was an arterial catheter dressing change, identified as a nonpainful procedure from a pilot study in our ICU. The second was turning, a procedure that patients have described as painful.22,23 In addition, pain was assessed at rest, i.e., before the first of these procedures, and in between these procedures, at least 30 min after the first procedure.

At each of these 4 points, a nurse researcher (AV, critical care nurse and student in nursing sciences) and an attending nurse simultaneously observed the patient for about 1 min, with the observers' assessments made independently. The attending nurse then determined the Ramsay Score (RS). Next, the nurse researcher and the attending nurse independently determined the BPSnurse score. Communicative patients were then asked to apply the VRS-4patient. This order was decided upon to prevent the nurses' scores from being influenced by the patient's score.

Demographic data such as gender, age, intensive care indication, and the Sequential Organ Failure Assessment score24 were collected.

Back to Top | Article Outline


The 72 nurses who participated in the study all attended a 4-h training session, given by a BPS-trained ICU nurse. Attention was given to the essentials of pain and the difficulties of scoring pain in ventilated and sedated patients. The use of the BPS was explained by means of pictures of ICU patients. All received a protocol explaining the study and the BPS.

Back to Top | Article Outline

Depth of Sedation

Depth of sedation was assessed by the RS, which is a single-item, 6-level scale (scores range from 1 to 6).25 The levels are: 1) patient anxious, agitated, restless; 2) patient cooperative, oriented, and tranquil; 3) patient drowsy or asleep, responds easily to commands; 4) patient asleep, brisk response to a light glabellar tap; 5) patient asleep, sluggish response to a light glabellar tap; and 6) patient asleep, no response to a light glabellar tap.9 The RSs were rated in the morning (between 7:30 and 8:00 am), whereas the pain assessments were completed between 8:00 am and 12:00 noon. In 8 patients, the RS was different during sedation assessment and pain assessment. The RS for the conscious sedated patients (median 6, range 3–6) was significantly lower (P < 0.001) than that for the sedated patients (median 3, range 2–5).

Back to Top | Article Outline

Standard Pain and Sedative Medication in the ICU

All patients received pain medication by protocol, i.e., 4 times daily 1 g of acetaminophen rectally, plus either 4 times daily 10 mg morphine subcutaneously if in moderate pain or 30–50 mg morphine per day by continuous IV infusion when in severe pain. Pain severity was evaluated on a daily basis. For procedural pain, patients received either no morphine or a bolus of 5–10 mg morphine, depending on the attending nurse's judgment. Patients were sedated preferably with propofol or midazolam, according to local standard practice.

Back to Top | Article Outline

Data Analysis

Data were analyzed using the SPSS software (version 15.0, Chicago, IL). The statistical analysis was performed by calculation on all measurements of all patients, including 1 measurement per day per patient. This approach was used by Payen et al.13 when they first validated the BPS in nonresponsive critically ill patients and can be justified because a critically ill patient's condition may rapidly change over 24 h, e.g., when taken off mechanical ventilation, with consequences in terms of organ failure, neurological or respiratory situation, sedation levels, pain levels, and communication abilities.

Kappa coefficients with quadratic weights were used to reflect agreement between the nurse researcher and the attending nurse regarding the BPS. Weighted kappa penalizes disagreement in proportion to its severity.26 Theoretically, the value of kappa can range from 0 (no agreement) to 1.0 (perfect agreement). A value larger than 0.6 was regarded as satisfactory.27 The 95% confidence intervals (CIs) for kappa coefficients were calculated.

Internal consistency, a measure of how the items within a scale are interrelated, was expressed in Cronbach's α. A high Cronbach's α value reflects high internal consistency. Generally, a value larger than 0.7 is regarded as satisfactory.28

The effect size is a standardized way to express the magnitude and meaning of an instrument's capacity to change, in this case, the BPS. The effect sizes of the BPS total and BPS items were calculated as the difference between the score at rest and the score during the painful procedure, divided by the standard deviation (sd) at rest.29 An effect size of around 0.20 is generally considered to be small, 1 of 0.50 indicates moderate differences, and those of 0.80 or above indicate large differences.30

Values are expressed as mean and 95% CI. Spearman nonparametric rank correlation coefficients (rs) were used to measure the degree of correlation for 2 ordinal variables. The unpaired t-test and the Mann-Whitney U-test served to compare differences in quantitative and nonparametric data, respectively. The test-retest procedure was analyzed by the paired Student's t-test. A P value of <0.05 was considered statistically significant.

Back to Top | Article Outline


Patients and Data

Table 2 shows the characteristics of the 80 enrolled patients, classified by state of sedation. The mean amount of propofol administered (±sd) was 130.4 ± 58.8 mg/h for conscious sedated patients vs 175.6 ± 72.6 mg/h for sedated patients (P < 0.05). The mean amount of midazolam administered in conscious sedated patients and sedated patients was 3.3 ± 1.2 vs 4.8 ± 2.5 mg/h (P = 0.32). The mean ICU stay at time of pain assessment (±sd) was 4.5 ± 3.6 for conscious sedated patients vs 5.4 ± 8.1 for sedated patients (P = 0.43). One hundred seventy-five observation series were completed: 126 in 63 sedated patients and 49 in 30 conscious sedated patients. The latter also included 49 VRS-4patient scores for 30 patients.

Table 2

Table 2

Back to Top | Article Outline

Interrater Reliability

Table 3 gives the quadratic weighted kappa and the exact agreement for the BPSnurse in sedated patients (126 observation series) and conscious sedated patients (49 observation series) between the nurse researcher and the attending nurse. Kappa values were excellent (0.80–0.83). There was no difference in exact agreement for sedated and conscious sedated patients (0.83 [95% CI: 0.76–0.87] vs 0.80 [95% CI: 0.72–0.88]).

Table 3

Table 3

Back to Top | Article Outline

Pain Scores in Conscious Sedated Patients and Sedated Patients


BPSnurse scores were significantly higher during painful procedures than at rest in both sedated patients (5.1 [95% CI: 4.8–5.5] vs 3.4 [95%: CI 3.3–3.5]) and conscious sedated patients (5.4 [95% CI: 4.9–5.9] vs 3.8 [95% CI: 3.5–4.1]) (Fig. 1). There was no difference in BPSnurse scores between the nonpainful procedure and rest in sedated patients (3.4 [95% CI: 3.3–3.6] vs 3.3 [95% CI: 3.2–3.4]) and conscious sedated patients (3.7 [95% CI: 3.5–3.9] vs 3.6 [95% CI: 3.3–3.8]). BPSnurse scores did not differ between sedated patients and conscious sedated patients at rest or during nonpainful or painful procedures.

Figure 1

Figure 1

Table 4 shows that the effect size for responsiveness of BPS total scores was large in sedated patients (126 observation series) and conscious sedated patients (49 observation series) (2.5 and 1.8, respectively). The effect size of the item “facial expression” was largest in both sedated patients (3.6) and conscious sedated patients (2.4). It was also large for “compliance with ventilation” (1.4 and 0.9) but moderate for “upper limbs” in both groups (0.7 and 0.5) (Table 4). During painful procedures, internal consistency was moderate in both sedated patients and conscious sedated patients (Cronbach's α 0.63 and 0.66, respectively).

Table 4

Table 4

Back to Top | Article Outline


In conscious sedated patients, VRS-4patient scores were significantly higher during painful procedures than at rest (2.2 [95% CI: 1.9–2.5] vs 1.1 [95% CI: 1.0–1.2]). Scores did not differ between the nonpainful procedure and rest (1.0 [95% CI: 1.0–1.0] vs 1.0 [95% CI: 1.0–1.0]).

Back to Top | Article Outline

Comparison Between BPSnurse and VRS-4patient in Conscious Sedated Patients

During the painful procedure, there was a strong positive correlation between BPSnurse and VRS-4patient (rs = 0.67, P < 0.001, 49 observation series) (Fig. 2). The 4 boxes in Figure 2 each have been divided into 4 quadrants, separating acceptable pain and unacceptable pain scores (unacceptable pain VRS-4 >2 and BPS >5).

Figure 2

Figure 2

During painful procedures, in 16% of the observations, the patient rated pain as acceptable (VRS scores, 1 or 2), whereas the nurse rated it as unacceptable (BPS >5). Conversely, in 12% of the observations, the patient rated pain as unacceptable (scores VRS >2), whereas the nurse rated it as acceptable (BPS 3–5). At rest, during the nonpainful procedure, and at retest rest, 98% of the observations were in the quadrant with acceptable pain scores. In these cases, both the patient and the nurse assigned acceptable pain scores.

Back to Top | Article Outline


The findings from this study are consistent with the notion that the BPS is reliable for pain assessment in conscious sedated patients. This is of interest in that so far the BPS has been validated for deeply sedated patients only.13 All ICU patients recovering from a deeply sedated state will pass through this conscious sedated state. Thereby, patients who experience agitation or delirium, in whom self-reporting will be complicated, benefit from this pain assessment in the conscious sedated state.

BPSnurse scores during painful procedures were significantly higher than those at rest in both sedated patients and conscious sedated patients. Payen et al.13 made a similar observation in deeply sedated patients, i.e., BPS scores were significantly higher for painful procedures such as turning or tracheal suctioning. Therefore, it would seem that the BPS can detect and discriminate pain and is a valid measure of pain in both sedated and conscious sedated patients. Furthermore, the internal consistency was comparable for observations in both groups, demonstrating similar homogeneity of the items. The fact that the effect size was large in both groups shows that the BPS is able to quantify change in clinical status and detect painful procedures. In both groups, the BPS subscale “facial expression” was the most sensitive to change, as in a previous study.20 The value of facial expression has been proven for both acute and chronic pain not only in adults31,32 but also in infants and children.33

Underestimation of patients' pain by nurses is a well-known problem.5 Surprisingly, using the BPS, nurses also tend to overestimate patients' pain. On the other hand, conscious sedated patients' pain scores are not always reliable. Therefore, use of the BPS in combination with the VRS-4 during painful procedures may lead to a more reliable rating of patients' pain. A previous study from our group34 also concluded that a combination of self-reporting and observational measures is recommended when credibility of self-reporting is doubted. Each method yields unique information. Self-reporting primarily reflects expressive pain behavior that is under control of higher mental processes. Observational measures capture behavior that is less subject to voluntary control and more automatic.34

The level of agreement between the research nurse and the attending nurse was high for both sedated patients and conscious sedated patients (kappa 0.83 and 0.80, respectively). The fact that the kappa values in this study pertained to 72 nurses and generally remained good shows that nurses can be trained to use the BPS in a reliable way in both sedated and conscious sedated patients.

In the ideal study design, nurses would be blinded to the nature of the procedure (painful or nonpainful) that is being performed at the point of assessment. This could be achieved by videotaping the scenes and having the nurses rate the scenes afterward. Care should be taken then to conceal the procedure. A limitation of video recordings is the likelihood that some aspects are missed because the general overview of the patients' situation is necessarily not provided.

In this study, we used the VRS-4 instead of the 11-point NRS because of the lack of capability in conscious sedated patients. This approach was inspired by a study from Briggs and Closs,12 who showed that postoperative patients prefer the VRS. However, it would be of interest to test whether our assumption that conscious sedated patients are indeed incapable of using an 11-point scale is valid.

In this study, most patients were in a sedated state, although it is desirable for patients to be in a conscious sedated state. This suggests that the health staff should give more attention to the sedation state of the patients in our ICU.

Nevertheless, because the BPS may both overrate and underrate patients' pain, and the patient's self-report is not always reliable, a combination of the nurse-rated BPS and the patient-rated VRS-4 is perhaps ideal for estimating patients' pain. Within this context, patients' sedation levels must be frequently assessed as well, and conscious patients' own self-reported pain scores must be considered the “gold standard.”

Back to Top | Article Outline


The BPSnurse is valid for use in conscious sedated patients during painful procedures. Thus, the BPS can be regarded as a bridge between the observational scale for noncommunicative and mechanically ventilated patients and the VRS-4 used by patients who are able to self-report pain.

Back to Top | Article Outline


The authors thank the staff and nurses of the intensive care unit of the St. Antonius Hospital for their contribution to this study. Ko Hagoort is thanked for editorial assistance.

Back to Top | Article Outline


1. Miner JR, Krauss B. Procedural sedation and analgesia research: state of the art. Acad Emerg Med 2007;14:170–8
2. Chanques G, Jaber S, Barbotte E, Violet S, Sebbane M, Perrigault PF, Mann C, Lefrant JY, Eledjam JJ. Impact of systematic evaluation of pain and agitation in an intensive care unit. Crit Care Med 2006;34:1691–9
3. Li DT, Puntillo K. A pilot study on coexisting symptoms in intensive care patients. Appl Nurs Res 2006;19:216–9
4. Puntillo KA. Pain experiences of intensive care unit patients. Heart Lung 1990;19:526–33
5. Turner JS, Briggs SJ, Springhorn HE, Potgieter PD. Patients' recollection of intensive care unit experience. Crit Care Med 1990;18:966–8
6. Hamill-Ruth RJ, Marohn ML. Evaluation of pain in the critically ill patient. Crit Care Clin 1999;15:35–54, v–vi
7. Sessler CN, Jo Grap M, Ramsay MA. Evaluating and monitoring analgesia and sedation in the intensive care unit. Crit Care 2008;12(suppl 3):S2
8. Payen JF, Chanques G, Mantz J, Hercule C, Auriant I, Leguillou JL, Binhas M, Genty C, Rolland C, Bosson JL. Current practices in sedation and analgesia for mechanically ventilated critically ill patients: a prospective multicenter patient-based study. Anesthesiology 2007 106:687–95; quiz 891–2
9. Lerch C, Park GR. Sedation and analgesia. Br Med Bull 1999;55:76–95
10. Fraser GL, Riker RR, Prato BS, Wilkins ML. The frequency and cost of patient-initiated device removal in the ICU. Pharmacotherapy 2001;21:1–6
11. Jacobi J, Fraser GL, Coursin DB, Riker RR, Fontaine D, Wittbrodt ET, Chalfin DB, Masica MF, Bjerke HS, Coplin WM, Crippen DW, Fuchs BD, Kelleher RM, Marik PE, Nasraway SA Jr, Murray MJ, Peruzzi WT, Lumb PD. Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult. Crit Care Med 2002;30:119–41
12. Briggs M, Closs JS. A descriptive study of the use of visual analogue scales and verbal rating scales for the assessment of postoperative pain in orthopedic patients. J Pain Symptom Manage 1999;18:438–46
13. Payen JF, Bru O, Bosson JL, Lagrasta A, Novel E, Deschaux I, Lavagne P, Jacquot C. Assessing pain in critically ill sedated patients by using a behavioral pain scale. Crit Care Med 2001;29:2258–63
14. Young J, Siffleet J, Nikoletti S, Shaw T. Use of a Behavioural Pain Scale to assess pain in ventilated, unconscious and/or sedated patients. Intensive Crit Care Nurs 2006;22:32–9
15. Ahlers SJ, van Gulik L, van der Veen AM, van Dongen HP, Bruins P, Belitser SV, de Boer A, Tibboel D, Knibbe CA. Comparison of different pain scoring systems in critically ill patients in a general ICU. Crit Care 2008;12:R15
16. Gélinas C, Fillion L, Puntillo KA, Viens C, Fortier M. Validation of the critical-care pain observation tool in adult patients. Am J Crit Care 2006;15:420–7
17. Cade CH. Clinical tools for the assessment of pain in sedated critically ill adults. Nurs Crit Care 2008;13:288–97
18. Kress JP, Pohlman AS, O'Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill patients undergoing mechanical ventilation. N Engl J Med 2000;342:1471–7
19. Kress JP, Gehlbach B, Lacy M, Pliskin N, Pohlman AS, Hall JB. The long-term psychological effects of daily sedative interruption on critically ill patients. Am J Respir Crit Care Med 2003;168:1457–61
20. Aissaoui Y, Zeggwagh AA, Zekraoui A, Abidi K, Abouqal R. Validation of a behavioral pain scale in critically ill, sedated, and mechanically ventilated patients. Anesth Analg 2005;101:1470–6
21. Blenkharn A, Faughnan S, Morgan A. Developing a pain assessment tool for use by nurses in an adult intensive care unit. Intensive Crit Care Nurs 2002;18:332–41
22. Puntillo KA, White C, Morris AB, Perdue ST, Stanik-Hutt J, Thompson CL, Wild LR. Patients' perceptions and responses to procedural pain: results from Thunder Project II. Am J Crit Care 2001;10:238–51
23. Stanik-Hutt JA. Pain management in the critically ill. Crit Care Nurse 2003;23:99–103
24. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Med. Intensive Care Med 1996;22:707–10
25. Ramsay MA, Savege TM, Simpson BR, Goodwin R. Controlled sedation with alphaxalone-alphadolone. BMJ 1974;2:656–9
26. Fleiss J. The measurement of interrater agreement statistical methods for rates and proportions. 2nd ed. New York: Wiley, 1981:212–35
27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74
28. Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297–334
29. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care 1989;27:S178–S189
30. Meenan RF, Kazis LE, Anthony JM, Wallin BA. The clinical and health status of patients with recent-onset rheumatoid arthritis. Arthritis Rheum 1991;34:761–5
31. Prkachin KM. The consistency of facial expressions of pain: a comparison across modalities. Pain 1992;51:297–306
32. Terai T, Yukioka H, Asada A. Pain evaluation in the intensive care unit: observer-reported faces scale compared with self-reported visual analog scale. Reg Anesth Pain Med 1998;23:147–51
33. Franck LS, Greenberg CS, Stevens B. Pain assessment in infants and children. Pediatr Clin North Am 2000;47:487–512
34. Hadjistavropoulos T, Craig KD. A theoretical framework for understanding self-report and observational measures of pain: a communications model. Behav Res Ther 2002;40:551–70
© 2010 International Anesthesia Research Society