Secondary Logo

Journal Logo

Benchmarking as a tool of continuous quality improvement in postoperative pain management

Meissner, W.*; Ullrich, K.*; Zwacka, S.

European Journal of Anaesthesiology (EJA): February 2006 - Volume 23 - Issue 2 - p 142–148
doi: 10.1017/S026502150500205X
Original Article
Free

Background and objective: Quality of acute pain management is far from being satisfactory. These deficits are not caused by the complexity of the medical problem but by difficulties in organization and hospital structures, sand procedures. Continuous quality improvement is a recommended tool to overcome such difficulties and to increase quality in the long run. This study reports the implementation of benchmarking-based continuous quality improvement to improve postoperative pain management at a university hospital.

Methods: A specialised pain nurse interviewed patients of three surgical departments on the first day after surgery, and continuously assessed process and outcome quality parameters. A multidisciplinary team of anaesthetists, surgeons, nurses and pharmacists implemented a regular procedure of data analysing and internal benchmarking. Results and suggested improvements were fed back to the healthcare teams.

Results: From 1998 to 2002, 6756 patients were assessed. Average pain on ambulation and maximal pain were 3.7 ± 2.4 and 5.0 ± 2.5 (mean ± SD) on a 11-point numeric rating scale. Pain intensity at rest was 1.9 ± 1.8. Over time, pain intensity on ambulation decreased (P = 0.022) whereas maximal pain and pain at rest remained unchanged. There was an increase in the number of patients who received non-opioid analgesia (P < 0.001).

Conclusions: A continuous quality improvement process could be established and is now successfully used in clinical routine. Cornerstones of this project were frequent assessments of process and outcome parameters, regular benchmarking and implementation of feedback mechanisms. Changes in organization of medical management and multidisciplinary teamwork seem to be more important than medical or technical aspects.

*Friedrich-Schiller-University, Department of Anaesthesiolgy and Intensive Care, Jena, Germany

Friedrich-Schiller-University, Department of Nursing, Jena, Germany

Correspondence to: Winfried Meissner, Department of Anaesthesiology and Intensive Care, Friedrich-Schiller-University Jena, Erlanger Allee 101, D-07740 Jena, Germany. E-mail: meissner@med.uni-jena.de; Tel: +36 41 9323353; Fax: +36 41 9323152

Accepted for publication 27 September 2005

First published online January 2006

Back to Top | Article Outline

Introduction

Reduction of postoperative pain is not only an ethical commitment but may improve surgical outcome under certain circumstances [1]. Surveys from various countries show that the quality of acute pain management is far from being satisfactory [2-4], although numerous reports about appropriate techniques and drugs have been published. Thus, the reasons for inadequate pain treatment are mainly deficits in organisation and personal resources and not medical problems: ‘… it appears that the solution to the problems of postoperative pain management lies not so much in the development of new techniques but in the development of an organisation to exploit existing expertise’ [5].

The American Pain Society recommended implementation of continuous quality improvement (CQI) strategies to detect and improve deficits in pain management [6,7]. Key elements of a CQI include continuous re-assessment and analysis of processes and outcomes. In the field of acute pain, several studies have shown the effectiveness of particular interventions. However, results have often deteriorated after time [8], and the use of CQI tools has only rarely been reported.

In our institution, an acute pain management project has been successfully implemented over 1.5 yr. However, following the end of this project, pain management activities as well as patient outcomes deteriorated again. Consequently, we initiated a CQI programme based on regular assessment of outcome and process quality parameters, repeated benchmarking and feedback mechanisms.

In a previous publication, we presented preliminary results of this one-time intervention and our first experience with the CQI programme [9]. The current article exceeds the former as further results and methodological aspects of continuous quality management initiatives are discussed. It was not our aim to conduct a representative survey of postoperative pain intensity, nor was it to study the impact of specific medical concepts on postoperative pain. Moreover, we are convinced that pain management does not fail because of shortage of pharmacological or technical solutions. Therefore, medical aspects are addressed only briefly.

Back to Top | Article Outline

Methods

Setting

The CQI programme was conducted on five wards of three surgical department (Department of Visceral Surgery, 86 beds: 3 wards; Department of Orthopaedic/ Trauma Surgery, 54 beds: 1 ward; Department of Cardiothoracic Surgery, 28 beds: 1 ward) of the University Hospital of Jena, Germany. Routine pain management on the participating wards was based on regular pain assessments and analgesic prescription algorithms. The medical concept consisted of routine use of non-opioid analgesics in combination with nurse-/patient-controlled intravenous opioid analgesia as well as regional anaesthetic techniques as described before [9].

Back to Top | Article Outline

Process and outcome data assessment

The main target of the CQI programme was to improve the quality of pain management not only after a single intervention but to maintain it over a longer period of time. A specialized pain nurse (S.Z.) visited a random sample of patients on the first postoperative day, excluding those who were transferred to the intensive care unit or those not being able to communicate properly. Patients who had been operated on before in our hospital were also excluded. To prevent selection bias, we daily randomized the wards that had to be visited. On these wards, each patient matching our inclusion criteria was included. The nurse conducted a semi-structured interview and documented postoperative pain treatment as well analgesia-related complications (Table 1). To avoid any interviewer-patient interaction bias, the nurse was instructed to inform patients that she was working independently from the healthcare team, that all information or judgement given in the interview would be treated confidentially, and that participation was voluntary.

Table 1

Table 1

Data were made anonymous immediately after the interview so that results could be attributed only to the time point and the ward but not to an individual patient.

Back to Top | Article Outline

Data analysis

Based on these data, an internal benchmarking process was performed in regular intervals. In detail, the mean values of the investigated parameters were calculated for periods of 3 months. From the beginning of 1999, all wards received their individual results compared with the averages of all participants. The results were compared with each other and discussed in a multidisciplinary pain management team composed of anaesthetists, surgeons of all departments, nursing staff and the head of the hospital pharmacy. Finally, feedback to the wards consisted of the results in process and outcome quality of each ward compared with those of all departments, performance over time, as well as recommendations of the multidisciplinary pain management team.

Pain intensities were obtained using an 11-point Numeric Rating Scale (NRS). Data were stored and analysed in a EXCEL data base. NRS scores are presented as mean ± standard deviation (SD). Process and outcome parameters were tested for change over time using the rank correlation test for non-parametric data (Kendall-Tau). Statistical significance was defined as P < 0.05.

Back to Top | Article Outline

Results

From February 1998 to December 2002, the CQI project included 6794 patients; of those, 6756 data sets (99.4%) were analysable (32% from the Department of Visceral Surgery; 45% from the Department of Orthopaedic/Trauma Surgery and 23% from the Department of Cardiothoracic Surgery). Further information on patient distribution is shown in Table 2. On average, 110 patients were interviewed per month. Data collection including data entry into a desktop computer required 10 min per patient. This adds up to a total of 50-60 min per day for all wards.

Table 2

Table 2

Back to Top | Article Outline

Process quality

In 70% of all patients, the prescription and assessment sheet for postoperative analgesia was in use. Regular non-opioid analgesia was given to 82% of patients (Fig. 1, Table 3). Over the 5 yr observation period, both parameters showed a positive trend which was significant for the application of non-opioid analgesics (P < 0.001).

Figure 1.

Figure 1.

Table 3

Table 3

Back to Top | Article Outline

Outcome quality

Pain on ambulation on the first postoperative day and maximal pain during the initial 24 h were 3.7 ± 2.4 and 5.0 ± 2.5 (mean ± SD), respectively (Tables 4 and 5). Both parameters improved over time, reaching statistical significance for pain on ambulation (P = 0.022, Fig. 2). Pain intensity at rest was 1.9 ± 1.8 on the first postoperative day and did not show any trend (Table 6).

Table 4

Table 4

Table 5

Table 5

Figure 2.

Figure 2.

Table 6

Table 6

Analgesia-related adverse effects and complications remained rare throughout the observation period (Table 7). Sedation (patient difficult to awake or respiratory frequency <8 min−1) was reported in 0.2% of patients.

Table 7

Table 7

The pain team met every 3-5 months with the head nurses and surgeons of all wards, and a representative of the hospital pharmacy. During these meetings, results were presented and discussed. A feedback example is presented in Figure 3. In the case of obvious deficits in process quality (i.e. decrease in the number of prescription sheets), precise actions were recommended to all involved parties (for instance, the ward nurse shall not accept the patient without this sheet). In the case of outcome impairment, changes in results were visualised and possible causes (i.e. change in pain management, staff fluctuation) were discussed. The results of these discussions were disseminated in written form.

Figure 3.

Figure 3.

Back to Top | Article Outline

Discussion

Surveys show considerable deficits in postoperative pain management throughout the world [2,3]. In an investigation of postoperative pain services in Europe, a few only reported on routine pain assessment and documentation, and occasional outcome measurements were performed in 40% of the hospitals only [10]. Few studies have reported on pre-post-improvement of acute pain management by single- or short-time interventions [11-13]. We report on the implementation of a CQI programme in postoperative pain management that has been performed over the past 5 yr and that included more than 6500 patients. Cornerstones of this project were continuous quality assessments, regular benchmarking and implementation of feedback mechanisms. Process as well as outcome quality parameters improved over time, partly reaching statistical significance.

We were able to show an improvement in pain on ambulation from 3.9 ± 2.3 to 3.6 ± 2.3 on a 11-point NRS over 5 yr. Maximal pain and pain at rest showed no change over time. In general, pain intensity was at least equal [4,13] or below [2,14,15] those published in recent reports of postoperative pain.

One could argue about the clinical relevance of such an improvement. Several projects studying one-time interventions showed more rapid changes in outcome parameters. In our preceding project, pain intensity decreased by 10% after 1 yr. However, the degree of improvement depends on the situation at the beginning of an intervention, allowing more progress when starting from a worse level of quality. Also, it seems to be very difficult to maintain such results over a longer period [8]. In many short-lasting interventions, success might be triggered not specifically by the CQI activity but by the psychological stimulus of healthcare providers who are pleased to receive attention as long as the activity is in progress but who are losing interest as soon as the project has finished (‘Hawthorne effect’) [16]. Thus, we are convinced that any measure that helps to assure a continuous level of quality for an extended period of time is of crucial importance. To our knowledge, there are no reports to date about similar CQI projects over a time period of several years in the pain setting.

In the field of quality improvement, recommended techniques have progressed from simple quality control to quality management consisting of continuous analysis, intervention and re-evaluation [17]. The plan-do-check-act (PDCA) cycle is a well-known model for continuous process improvement [18]. Continuous data analysis, routine feedback and benchmarking processes provide several advantages. First, the problem of influencing results by the evaluation itself can be reduced. Second, participants are encouraged to improve outcome not only for a limited time. Third, staff members become accustomed to this method of quality improvement, and, ideally, accept it as a means to develop good medical practice and clinical excellence.

Another advantage of continuous data assessment should be mentioned. Usually, benchmarking processes compare different competitors. In the beginning of our project, we also used this technique. However, obvious differences with other wards were often excused by co-factors that are difficult to adjust as, for instance, type of surgery, differences in patient demography or co-morbidity. Acceptance of feedback processes rose substantially when we started to compare wards not only with each other but with their own past performance; it was their own performance which now appeared to be the benchmark.

Back to Top | Article Outline

Outcome and process quality parameters

In a survey of more than 200 US institutions, more than 90% reported to have a continuos improvement programme in pain management [19]. On a closer look, however, many respondents did not collect outcome data as pain intensity or impact of pain on functioning. The authors underlined the overwhelming importance of assessing parameters of outcome in any CQI activities [19].

We focused on three parameters of pain intensity as outcome indicators. However, methodology of pain measurement is controversial [20,21]. We observed that even in patients with low pain intensity at rest, high levels of maximal pain or of pain on ambulation may occur. Apparently maximal pain and/or pain on ambulation are more sensitive indicators for insufficient pain management than pain at rest or current pain. It is debatable as to whether ‘absolute’ benchmarks (i.e. ‘pain intensity < 4 on the 11-point numerical scale’) are appropriate targets of acute pain management. ‘Satisfactory’ pain intensity levels will differ according to the extent of surgery and probably many other factors. A NRS of 3 may be adequate after appendectomy but too high after minor surgery. Moreover, studies that recommend a fixed pain intensity level as a target of pain therapy may not be transferable to other patient groups. Also, in many of these recommendations, it is not specified whether an approved pain level (i.e. 3 or 4 on the 11-point numerical scale) refers to acute, average or maximal pain at rest or on movement.

Outcome questions describing how pain interferes with important activities like deep breathing, mobilisation, or sleep might also be relevant clinical indicators for the quality of pain management [6,22] and could be used to define outcome benchmarks. Dietrick-Gallagher and co-workers suggested that a given number of patients (i.e. 90%) meeting functional outcome criteria might be a meaningful target in postoperative pain therapy [23]. However, standardised tools for the evaluation of functional aspects in postoperative pain control have not been evaluated yet.

Although improvement of outcome is the main purpose of CQI, comprising parameters of process quality may help to identify obstructive factors [22]. In one of the earliest studies on quality improvement in acute pain management, it was shown that introduction of the measurement of pain in itself improved pain intensity significantly [24]. We decided to incorporate two questions which may indicate possible inadequacies of the different team members who are involved in pain management, i.e. the number of missing prescriptions and assessment sheets at bedside; these may reflect deficits in the communication between anaesthetist, recovery room and ward. The number of patients not receiving their routine non-opioid analgesic may indicate shortcomings in prescription or nursing care. Parameters of process quality as well as structural quality should be determined according to their likely impact on outcome [25]. Unfortunately, process quality is often considered as an important value in itself. In contrast, what is important for the patient is not how but if successful and safe pain treatment is provided.

Our patients reported a remarkable low incidence of adverse effects like nausea, vomiting or sedation. This may partly be explained by the routine use of non-opioid analgesics which are part of our medical concept and that may reduce opioid requirements. Also, on the first postoperative day, most anaesthesia-related adverse effects have disappeared. However, it is possible that an earlier and more frequent survey would have shown a higher incidence of these complications.

It would be desirable to collect more information on adverse reactions of non-opioid analgesics as gastro-intestinal, renal or haematological adverse effects. However, most of these complications will develop later than during the first postoperative day, and assessment would require some laboratory or other exploration.

Our report is subject to several limitations. The method of randomization does not meet the requirements of a prospective, clinical study; however it seems to be an acceptable compromise for a CQI project performed in clinical routine practice. Patients who had surgery on a Friday were not equally represented since on weekends, no data collection could be performed. In our institution, different types of surgery are equally spread over all weekdays.

A critical issue of any survey of subjective parameters is the potential interviewer-patient interaction bias. Complete blinding of the interviewer was not feasible; therefore, we did not rely on data obtained by the staff of the wards, we separated assessments from clinical routine, and we undertook great efforts to train the interviewer for data collection.

Another possible limitation of our study was that hard outcome criteria, such as length of hospital stay or morbidity, were not included. We studied a wide range of surgical procedures including a number of minor operations where consequences on length of hospital stay or morbidity were unlikely. Also, improvement in pain control itself should be valued as an ethical goal of medical treatment, and discussion should not be limited to economic aspects.

It would have been highly desirable to implement more process and outcome parameters as well as possible complications and adverse effects. However, a CQI project that works under routine conditions must focus on a limited number of data. In our project, these parameters had to meet the following criteria: displaying outcome quality of pain therapy from the patient's perspective, indicating processes only that were thought to be important and being easily available. But even if much more possible co-factors would have been obtained it is questionable whether a continuous data collection like ours is suitable for discovering causal relationships between interventions and possible complications because there is no randomisation of treatment, and it is not possible to control all variables. This is a common problem in all performance measurement studies [13,26]. It should be emphasized that quality improvement procedures, as demonstrated in this study, must be framed by appropriate measures for feedback and ‘management of change’. This is probably an even more important part of a CQI programme than to establish data collecting and feedback.

Meanwhile, an extension of this project to other hospitals is supported by a grant from the German Federal Ministry of Health and Social Security. Outcome parameters were expanded by assessment of functional impact of pain, and some more clinical parameters (i.e. technique of pain management, type and duration of surgery, method of anaesthesia) were included. A web-based benchmark software was developed which allows immediate data analysis and feedback showing each participant his/her results compared with all others via a secured internet access.

In conclusion, a CQI process was established and has been successfully used in clinical routine for more than 5 yr. Some parameters of process and outcome quality improved over the observation period, others remained constant. Transparency and multidisciplinary teamwork as well as benchmarking and prompt feedback mechanisms seem to be key elements of the successful implementation of a quality management initiative.

Back to Top | Article Outline

Acknowledgements

This project was supported by the staff of the Departments of Visceral Surgery, Orthopaedic/Trauma Surgery and Cardiothoracic Surgery. We thank Sigrid Oehler, Head of the Surgical Nursing Team, for her special assistance and permanent support. The extension of this project is supported by a grant from the German Federal Ministry of Health and Social Security (217-43794-6/3).

Back to Top | Article Outline

References

1. Kehlet H. Effect of postoperative pain treatment on outcome-current status and future strategies. Langenbecks Arch Surg 2004; 389: 244-249.
2. Apfelbaum JL, Chen C, Mehta SS, Gan TJ. Postoperative pain experience: results from a national survey suggest postoperative pain continues to be undermanaged. Anesth Analg 2003; 97: 534-540, table of contents.
3. Rawal N. Acute pain services revisited- good from far, far from good? Reg Anesth Pain Med 2002; 27: 117-121.
4. Dolin SJ, Cashman JN, Bland JM. Effectiveness of acute postoperative pain management: I. Evidence from published data. Br J Anaesth 2002; 89: 409-423.
5. Rawal N, Berggren L. Organization of acute pain services: a low-cost model. Pain 1994; 57: 117-123.
6. American Pain Society Quality of Care Committee. Quality improvement guidelines for the treatment of acute pain and cancer pain. JAMA 1995; 274: 1874-1880.
7. Lack JA, White LA, Thoms GM, Rollin AM. Raising the Standard: The Royal College of Anaesthetists, 2000.
8. Allcock N. The use of different research methodologies to evaluate the effectiveness of programmes to improve the care of patients in postoperative pain. J Adv Nurs 1996; 23: 32-38.
9. Meissner W, Ullrich K, Zwacka S, Schreiber T, Reinhart K. [Quality management in postoperative pain therapy.] Anaesthesist 2001; 50: 661-670.
10. Rawal N, Allvin R. Acute pain services in Europe: a 17-nation survey of 105 hospitals. The EuroPain Acute Pain Working Party. Eur J Anaesthesiol 1998; 15: 354-363.
11. Coleman SA, Booker-Milburn J. Audit of postoperative pain control. Influence of a dedicated acute pain nurse. Anaesthesia 1996; 51: 1093-1096.
12. Tighe SQ, Bie JA, Nelson RA, Skues MA. The acute pain service: effective or expensive care? Anaesthesia 1998; 53: 397-403.
13. Bardiau FM, Taviaux NF, Albert A, Boogaerts JG, Stadler M. An intervention study to enhance postoperative pain management. Anesth Analg 2003; 96: 179-185.
14. McNeill JA, Sherwood GD, Starck PL, Thompson CJ. Assessing clinical outcomes: patient satisfaction with pain management. J Pain Symptom Manage 1998; 16: 29-40.
15. Miaskowski C, Nichols R, Brody R, Synold T. Assessment of patient satisfaction utilizing the American Pain Society's Quality Assurance Standards on acute and cancer-related pain. J Pain Symptom Manage 1994; 9: 5-11.
16. Shayer M. Problems and issues in intervention studies. In: Demetriou A, Shayer M, Efklides A (eds). Neo-Piagetian Theories of Cognitive Development: Implications and Applications for Education. London: Routledge, 1992: 107-121.
17. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705.
18. Deming WE. Out of the Crisis. Cambridge, MA: MIT Press, 1986.
19. Ward S, Donovan M, Max MB. A survey of the nature and perceived impact of quality improvement activities in pain management. J Pain Symptom Manage 1998; 15: 365-373.
20. Aubrun F, Paqueron X, Langeron O, Coriat P, Riou B. What pain scales do nurses use in the postanaesthesia care unit? Eur J Anaesthesiol 2003; 20: 745-749.
21. Jensen MP, Chen C, Brugger AM. Postsurgical pain outcome assessment. Pain 2002; 99: 101-109.
22. Crowley B, Lubesnick K, Sylwestrak ML. Clinical indicators: a tool for improving pain management documentation. J Nurs Care Qual 1991; 6: 40-46.
23. Dietrick-Gallagher M, Polomano R, Carrick L. Pain as a quality management initiative. J Nurs Care Qual 1994; 9: 30-42.
24. Gould TH, Crosby DL, Harmer M et al. Policy for controlling pain after surgery: effect of sequential changes in management. BMJ 1992; 305: 1187-1193.
25. Gordon DB, Pellino TA, Miaskowski C et al. A 10-year review of quality improvement monitoring in pain management: recommendations for standardized outcome measures. Pain Manag Nurs 2002; 3: 116-130.
26. Joint Commission on Accreditation of Healthcare Organizations. Improving the Quality of Pain Management through Measurement and Action. Oakbrook Terrace, IL: JCAHO, 2003.
Keywords:

QUALITY ASSURANCE; health care; BENCHMARKING; PAIN POSTOPERATIVE

© 2006 European Society of Anaesthesiology