The types of interventions devised and implemented in each of the hospitals are outlined in Table 4. Teaching formed a major component of the intervention in all the hospitals but one. Most often, the teaching included both physicians and nurses; in 2 hospitals, the teaching involved nurses only. Four hospitals developed local protocols; one was able to establish an Acute Pain Service and another to receive additional resources for an existing Acute Pain Service. Principal Investigators used a variety of sources in developing their interventions, including consultants from high resource countries with which they had been collaborating for some years, the IASP working groups, PAIN OUT, and resources from their own hospital.
Across the 14 wards and 4 interventions, there were 56 potential episodes for change. Pain assessment and administration of nonopioids on the ward reached maximal performance at baseline in 8 wards. One of these wards obtained maximal performance for both indicators. In PAIN OUT, both these measures have limited sensitivity to detect change because they are dichotomous, and so the activity registers as having been performed even if it was performed only once rather than regularly, as is the intention of the recommendation.
Of the potential remaining 48 instances for improvement, there were 17 (35%) episodes of improvement, 22 (46%) of no change and 9 (19%) where the intervention was performed less frequently compared with baseline. The interventions that improved most often, in 5 wards, were receipt of information and wound infiltration, whereas pain assessment improved in 3 wards. Administration of nonopioids improved in 2 wards but was performed less frequently, compared with baseline, in 4 wards (Table 5).
Of the 56 potential episodes for change in a PRO, there were 21 (37.5%) episodes of improvement and 30 (53.6%) episodes in which the PROs did not change and 5 (9%) where a PRO worsened compared with baseline (Table 5). Wish for more treatment improved most often, in 7 wards, ie, fewer patients would have wished for more treatment for pain after the intervention compared with baseline. The proportion of patients reporting severe pain was smaller in 6 wards, and the extent that pain interfered with activities in bed and nausea improved in 4 wards. All 4 PROs improved in 2 wards in the same hospital, and 3 PROs improved in 1 ward and 2 in another ward in the same hospital.
One PI reported that availability of quantitative findings from patients in his institution and presenting them to colleagues and the hospital administration served as a “wake up call,” an “eye opener,” allowing him to mobilize and engage colleagues from within his institution. The findings were also used to convince the hospital's administration to provide resources for setting up an Acute Pain Service. The PI in another center also presented findings to his hospital administration and received funding to develop an electronic database for the Acute Pain Service and to recruit additional personnel. He plans to work at the provincial and national levels to create standards for treatment. Most PIs would choose to upscale the project to other wards in their hospital.
Two PIs described team-level factors in their hospitals.
The PI, a clinician at the management level, reported being able to enlist collaboration from multidisciplinary providers, surgeons, anaesthesiologists, and nurses to implement change. It was the first time they were involved in a project of this format and so were eager to succeed and gain new skills. Baseline conditions were low. Structural changes were extensive, including teaching staff and patients, changing forms to allow for recording pain management routinely, changing treatment routines, and setting aside an area for recovery after surgery. This is one of the hospitals that completed the study within the designated 12 months. Two processes improved in each of the 2 participating wards, 1 worsened, 3 PROs improved in 1 ward, and 2 in another.
The PI was keen and enterprising but relatively junior who had a clear action plan in the form of creating a standardized order sheet for administering treatment after surgery. However, the PI was unable to enlist collaboration from physician colleagues who were more senior and from nurses, who lacked sufficient support from management. Staff on the ward and at the managerial level expressed lack of interest in the project. Departmental heads delayed providing evaluations of the proposed intervention. Surgeons regarded filling in the standardized order as redundant and a waste of their time, saying that their focus was on managing the patients' surgical features of care and less on pain. Nurses did not cooperate with performing the recommendations written on the standardized order sheet. They considered the work as burdensome, adding to their existing workload and not being within their job description. Changes in small effect sizes took place. Surgeons on 1 of the 2 wards changed their practice to the extent that the proportion of patients receiving wound infiltration increased. Administration of nonopioids, in the second ward, increased, but report of receipt of information decreased. A PRO in each ward improved. Baseline performance for some treatment practices on these wards was good; they achieved maximal scores for assessment of pain at baseline; and nonopioids were given to a high proportion of patients.
Principal investigators presented findings for the work in their hospitals at scientific meetings. A nursing student submitted a master's thesis based on the work performed in her centre and published the findings in a peer-reviewed journal.7 Work in another hospital served as the basis for a PI's Doctoral thesis.
Our findings demonstrate that multidisciplinary providers, working in hospitals in developing countries, were able to perform work to improve care of perioperative pain in their institutions. The large number of applicants responding to the call for participation in the project suggests that providers in developing countries are interested in undertaking QI work. The project proved achievable but challenging. Although we had funding for ten hospitals, 8 completed the program. At baseline, staff from the 8 hospitals collected data from 15 wards, and after intervention, data collection continued in 14 of these wards. Interestingly, although local teams worked independently of one another, teaching staff about management of perioperative pain formed the major component of the intervention in all but one hospital.
The overall low rate of missing data records indicates that surveyors and patients in the different institutions were able to collect data and fill in the project questionnaires for the variables we analysed. This is consistent with findings in an earlier study where we examined missing scores for all variables in the process and IPO questionnaires.50 Wound infiltration records were missing in 2 hospitals, for both treatment phases, indicating that recording this variable was challenging. Surgeons perform wound infiltration and document this in the surgical record. However, surveyors report that this information is not consistently available, even if the infiltration is performed. Missing records for age and surgical code indicate that surveyors in those wards may have required further training.
Interestingly, when PROs improved, they tended to cluster in the same ward and same hospital. Wound infiltration at the end of surgery was 1 of the 2 processes that changed the most. A PI attributed this change initially to teaching, and later, it was reinforced by the surgeons experiencing, first hand, that the intervention was effective in alleviating pain. By contrast, the frequency of administration of nonopioids on the ward decreased in 4 wards, and these were all wards where the proportion of wound infiltration increased. The PI attributed this to the wound infiltration being so effective, at least during the first hours after surgery, that surgeons may have regarded writing an order for nonopioids as unnecessary. Alternatively, nurses on the ward, finding the patients comfortable, may have decided that it was unnecessary to administer the nonopioids. However, as ratings of worst pain changed in only 2 of these 5 wards, it seems that the surgeons and/or nurses' observations did not align with the patients' reports.
After the intervention, the proportion of patients reporting they received information about their pain treatment options increased in 6 wards. It is possible that members of staff were more informed as a result of the teaching they received, and consequently, patients benefited from this. As shown by other studies,5,37 staff education is an important component in improving patient outcomes. Findings from a large study including 138 hospitals and over 21,000 patients in Germany showed that informing patients about postoperative management options demonstrated a consistent and positive effect on PROs.26 Providing information about pain treatment options and involvement of patients in decisions about treatment were key predictors of patient satisfaction with postoperative pain therapy.38 By contrast, it is not clear why the proportion of patients who reported receiving information was reduced in 4 wards in the second part of the study. Two indicators, assessment of pain and administration of nonopioids, demonstrated a ceiling effect in 7 wards. At present, we do not know whether this suggests good management, leaving little room for further improvement for these indicators on these wards or a limitation of the dichotomous method used in PAIN OUT for registering these variables. Our work50 and other studies15,25,28 suggest that there is little relationship between pain assessment and documentation as performed in the clinical routine and PROs, making this a weak quality indicator. However, it is possible that pain assessment in the clinical routine is not implemented as recommended by guidelines, eg, it is not followed up by treatment and/or reassessment. Further inspection of the data demonstrated that patients were not administered full doses of nonopioids, thereby, indicating that there is room to improve how this treatment process is performed.
At the current stage of developing this change management program, we are unable to formally suggest which factors and conditions contributed to change, or lack of change, in the processes or PROs in the different hospitals or wards. A Cochrane review19 found that QI projects using auditing and feedback (A&F) yielded a median 4.3% increase in provider compliance with practice recommendations (interquartile range: 0.5%–16%), indicating that the effect of A&F on professional behavior and on patient outcomes ranged from little or no effect to a substantial effect. Conditions in which A&F may be most effective include low baseline performance, when the person responsible for performing the A&F is a supervisor or colleague, if it is provided more than once, when it is given both verbally and in writing, and if it includes clear targets and an action plan. Whether A&F is more effective when combined with other interventions, such as teaching and reminders, is still uncertain.20 Although A&F is a widely used QI approach in health care, there is still need to identify the key ingredients for successful A&F interventions and to understand the mechanisms of action of which lead to effective A&F interventions.9,24
Positive “organizational culture” is increasingly understood as fundamental to achieving high performance in health care settings,3 although both culture and performance are challenging concepts to define, operationalize, and measure.39 According to Vaughn et al.,42 characteristics of high performing organizations include a positive organizational culture, in the form of norms, values, and basic assumptions of an organization, which embraces change. This flexibility may accelerate adoption of initiatives that improve care. In addition, in high performing organizations, change is led by committed individuals who support and respect employees. On the other hand, poor organizational culture is one factor leading to lack of change within health care organizations. PAIN OUT has not addressed organizational culture to date. It is possible that features of organizational culture facilitated the changes in some hospitals and wards and prevented them in others. Future projects within PAIN OUT may consider adopting methodology that would address this issue formally.
Medical faculty from high resource countries are increasingly involved in programs for teaching local physicians, residents, and allied personnel in developing countries and coupling this with QI projects.34,44 PAIN OUT methodology has been used to perform single A&F projects in Kenya, Rwanda, and Gaza.49 Our findings indicated the feasibility of data collection, but they also revealed some challenges. Staff whose pay is low were not keen to take on additional work without remuneration. Furthermore, costs of paper and ink on which to print questionnaire was covered, at times, out of pocket. Both factors restricted the scope of these audits and of implementing longer term QI projects. Funding for QI projects from either academic sources or from pharmaceutical companies is not readily available. The educational grant from the IASP, as a professional society, facilitated the current project.
Commitment of the PIs was notable given that the project was performed on a low budget, and that PIs and their collaborators were required to volunteer their time in addition to their regular clinical duties. An additional challenge was that communication between staff in each hospital, and the sponsors was predominantly through email.
In this study, we used one of the most commonly used pre–post study designs, an uncontrolled before and after study or a quasiexperimental design.17 This study design is often used where there are practical and ethical barriers to conducting randomized controlled trials.18 This is a relatively simple study design to conduct and is superior to observational studies. However, it is a weak evaluative design, in that secular trends or sudden changes make it difficult to attribute the observed changes to the intervention. Furthermore, the intervention can be affected by confounders such as the Hawthorne effect (the nonspecific beneficial effect on performance of taking part in research), which, in turn, can lead to an overestimate of the effectiveness of an intervention. This was the first multicenter QI project coordinated by PAIN OUT. The experience gained indicates that we may need to improve the study design, and that we should continue to learn from other fields where improvement science is maturing as a field.20,24
Principal investigators and their teams resided in 7 countries, the geographical distance and differences in work culture meant that they were working independently and could not support each other. The study phases in most hospitals took longer than the planned 12 months. Both geographical distance and providing support are currently being addressed by designing projects where groups of up to ten hospitals from the same geographical region or country work together. The timeframe for the project has been increased, providing leeway to accommodate for unforeseen difficulties. In the course of the project, collaborators collected extensive data about perioperative treatments and patients' experiences of pain in the different hospitals, offering a unique opportunity to learn about a wide range of practice patterns and patient evaluations. We used only a fraction of the data collected in this current study, leaving further work for subsequent studies.
Sustainability of this project can be viewed on 3 levels. Some PIs expressed interest to continue with the project in other surgical wards at the hospital or regional level. For PAIN OUT, the project has led to establishing networks of hospitals, worldwide. The IASP was able to promote research, education, and QI.
The authors have no conflict of interest to declare.
Supplemental digital content
Supplemental digital content associated with this article can be found online at http://links.lww.com/PR9/A38.
. Bardiau FM, Taviaux NF, Albert A, Boogaerts JG, Stadler M. An intervention study to enhance postoperative pain
management. Anesth Analg 2003;96:179–85.
. Benhamou D, Berti M, Brodner G, De Andres J, Draisci G, Moreno-Azcoita M, Neugebauer EA, Schwenk W, Torres LM, Viel E. Postoperative Analgesic Therapy Observational Survey (PATHOS): a practice pattern study in 7 central/southern European countries. PAIN
. Bradley EH, Brewster AL, Fosburgh H, Cherlin EJ, Curry LA. Development and psychometric properties of a scale to measure hospital organizational culture for cardiovascular care. Circ Cardiovasc Qual Outcomes 2017;10:e003422.
. Brennan F, Carr DB, Cousins M. Pain
management: a fundamental human right. Anesth Analg 2007;105:205–21.
. Chou R, Gordon DB, de Leon-Casasola OA, Rosenberg JM, Bickler S, Brennan T, Carter T, Cassidy CL, Chittenden EH, Degenhardt E, Griffith S, Manworren R, McCarberg B, Montgomery R, Murphy J, Perkal MF, Suresh S, Sluka K, Strassels S, Thirlby R, Viscusi E, Walco GA, Warner L, Weisman SJ, Wu CL. Management of postoperative pain
: a clinical practice guideline from the American Pain
Society, the American Society of Regional Anesthesia and Pain
Medicine, and the American Society of Anesthesiologists' Committee on Regional Anesthesia, Executive Committee, and Administrative Council. J Pain
. Cooper SA, Desjardins PJ, Turk DC, Dworkin RH, Katz NP, Kehlet H, Ballantyne JC, Burke LB, Carragee E, Cowan P, Croll S, Dionne RA, Farrar JT, Gilron I, Gordon DB, Iyengar S, Jay GW, Kalso EA, Kerns RD, McDermott MP, Raja SN, Rappaport BA, Rauschkolb C, Royal MA, Segerdahl M, Stauffer JW, Todd KH, Vanhove GF, Wallace MS, West C, White RE, Wu C. Research design considerations for single-dose analgesic clinical trials in acute pain
: IMMPACT recommendations. PAIN
. Cui C, Wang LX, Li Q, Zaslansky R, Li L. Implementing a pain
management nursing protocol for orthopaedic surgical patients: results from a PAIN
OUT project. J Clin Nurs 2018;27:1684–91.
. Dibra A, Kellici S, Akshija I. Postoperative pain
management at Tirana University Hospital Center Mother Teresa, Tirana, Albania. J Biol Regul Homeost Agents 2012;26:539–44.
. Dixon-Woods M, McNicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the health foundation's programme evaluations and relevant literature. BMJ Qual Saf 2012;21:876–84.
. Fay MP. Confidence intervals that match Fisher's exact or Blaker's exact tests. Biostatistics 2010;11:373–4.
. Fletcher D, Fermanian C, Mardaye A, Aegerter P; Pain
and Regional Anesthesia Committee of the French Anesthesia and Intensive Care Society (SFAR). Pain
and Regional Anesthesia Committee of the French Anesthesia and Intensive Care Society (SFAR). A patient-based national survey on postoperative pain
management in France reveals significant achievements and persistent challenges. Pain
and Regional Anesthesia Committee of the French Anesthesia and Intensive Care Society (SFAR). A patient-based national survey on postoperative pain
management in France reveals significant achievements and persistent challenges. PAIN
. Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, Bero LA. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2000:CD000172.
. Gan JT. Poorly controlled postoperative pain
: prevalence, consequences, and prevention. J Pain
. Gan TJ, Habib AS, Miller TE, White W, Apfelbaum JL. Incidence, patient satisfaction, and perceptions of post-surgical pain
: results from a US national survey. Curr Med Res Opin 2014;30:149–60.
. Gordon D, Zaslanksy R, Meissner W. Overview of systems design and quality improvement
to improve outcomes and identify best practices. In: Carr DB, Arendt-Nielsen L, Vissers KCP, editors. Pain
. IASP Press, 2018.
. Graham JW. Missing data analysis: making it work in the real world. Annu Rev Psychol 2009;60:549–76.
. Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract 2000;17(suppl 1):S11–6.
. Hakkennes S, Green S. Measures for assessing practice change in medical practitioners. Implement Sci 2006;1:29.
. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;CD000259.
. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more “business as usual” with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci 2014;9:14.
. Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006:CD000259.
. Kehlet H, Dahl JB. Assessment of postoperative pain
—need for action! PAIN
. Kehlet H, Jensen TS, Woolf CJ. Persistent postsurgical pain
: risk factors and prevention. Lancet 2006;13:1618–25.
. Marshall M, de Silva D, Cruickshank L, Shand J, Wei L, Anderson J. What we know about designing an effective improvement intervention (but too often fail to put into practice). BMJ Qual Saf 2017;26:578–82.
. Meissner W, Coluzzi F, Fletcher D, Huygen F, Morlion B, Neugebauer E, Pérez AM, Pergolizzi J. Improving the management of post-operative acute pain
: priorities for change. Curr Med Res Opin 2015;31:2131–43.
. Meißner W, Komann M, Erlenwein J, Stamer U, Scherag A. The quality of postoperative pain
therapy in German hospitals. Dtsch Arztebl Int 2017;114:161–7.
. Moore RA, Straube S, Aldington D. Pain
measures and cut-offs “no worse than mild pain
” as a simple, universal outcome. Anaesthesia 2013;68:400–12.
. Mularski RA, White-Chu F, Overbay D, Miller L, Asch SM, Ganzini L. Measuring pain
as the 5th vital sign does not improve quality of pain
management. J Gen Intern Med 2006;21:607–12.
. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for Quality Improvement
Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2016;25:986–92.
. R Core-Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing, 2014.
. Rawal N. Acute pain
services revisited—good from far, far from good? Reg Anesth Pain
. Rawal N. Current issues in postoperative pain
management. Eur J Anaesthesiol 2016;33:160–71.
. Revelle W, psych: procedures for personality and psychological research. Evanston: Northwestern University, 2018. Available at: https://CRAN.R-project.org/package=psych
, Version 1.8.4. Accessed July 2018.
. Robinson S, Pawelzik V, Megentta A, Benimana O, Mazimpaka D, Ndoli J, Sendegeya A, Wong R. A case study: applying quality improvement
methods to reduce pre-operative length of stay in a resource-constrained setting in Rwanda. J Hosp Adm 2016;5:41–7.
. Rothaug J, Zaslansky R, Schwenkglenks M, Komann M, Allvin R, Backström R, Brill S, Buchholz I, Engel C, Fletcher D, Fodor L, Funk P, Gerbershagen HJ, Gordon DB, Konrad C, Kopf A, Leykin Y, Pogatzki-Zahn E, Puig M, Rawal N, Taylor RS, Ullrich K, Volk T, Yahiaoui-Doktor M, Meissner W. Patients' perception of post-operative pain
management: validation of the International Pain
Outcomes questionnaire (IPO). J Pain
. Schug SA, Palmer GM, Scott DA, Halliwell R, Trinca J; Working Group of the Australian and New Zealand College of Anaesthetists and Faculty of Pain
Medicine. Acute pain
management: scientific evidence. 4th ed. Melbourne: ANZCA & FPM, 2015.
. Schwenkglenks M, Gerbershagen HJ, Taylor RS, Pogatzki-Zahn E, Komann M, Rothaug J, Volk T, Yahiaoui-Doktor M, Zaslansky R, Brill S, Ullrich K, Gordon DB, Meissner W. Correlates of satisfaction with pain
treatment in the acute postoperative period: results from the international PAIN
OUT registry. PAIN
. Scott T, Mannion R, Marshall M, Davies H. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Res Policy 2003;8:105–17.
. Stamer UM, Stüber F. Postoperative epidural analgesia: how about quality assessment? Anesth Analg 2003;97:918–9.
. Taylor RS, Ullrich K, Regan S, Broussard C, Schwenkglenks M, Taylor RJ, Gordon DB, Zaslansky R, Meissner W, Rothaug J, Langford R. PAIN
-OUT investigators the impact of early post operative pain
on health-related quality of life. Pain
. Vaughn VM, Saint S, Krein SL, Forman JH, Meddings J, Ameling J, Winter S, Townsend W, Chopra V. Characteristics of healthcare organizations struggling to improve quality: results from a systematic review of qualitative studies. BMJ Qual Saf 2019;28:74–84.
. Vijayan R. Managing acute pain
in the developing world. In: Pain
clinical updates. Vol. 19. Seattle, WA: International Association for the Study of Pain
. Weinberg M, Fuentes JM, Ruiz AI, Lozano FW, Angel E, Gaitan H, Goethe B, Parra S, Hellerstein S, Ross-Degnan D, Goldmann DA, Huskins WC. Reducing infections among women undergoing cesarean section in Colombia by means of continuous quality improvement
methods. Arch Intern Med 2001;161:2357–65.
. Weiser TG, Regenbogen SE, Thompson KD, Haynes AB, Lipsitz SR, Berry WR, Gawande A. An estimation of the global volume of surgery
: a modelling strategy based on available data. Lancet 2008;372:139–44.
. Weiss D, Dunn SI, Sprague AE, Fell DB, Grimshaw JM, Darling E, Graham ID, Harrold J, Smith GN, Peterson WE, Reszel J, Lanes A, Walker MC, Taljaard M. Effect of a population-level performance dashboard intervention on maternal-newborn outcomes: an interrupted time series study. BMJ Qual Saf 2018;27:425–36.
. Werner MU, Søholm L, Rotbøll-Nielsen P, Kehlet H. Does an acute pain
service improve postoperative outcome? Anesth Analg 2002;95:1361–72.
. Woldehaimanot TE, Eshetie TC, Kerie MW. Postoperative pain
management among surgically treated patients in an Ethiopian hospital. PLoS One 2014;9:e102835.
. Zaslansky R, Chapman CR, Rothaug J, Bäckström R, Brill S, Davidson E, Elessi K, Fletcher D, Fodor L, Karanja E, Konrad C, Kopf A, Leykin Y, Lipman A, Puig M, Rawal N, Schug S, Ullrich K, Volk T, Meissner W. Feasibility of international data collection and feedback on post-operative pain
data: proof of concept. Eur J Pain
. Zaslansky R, Rothaug J, Chapman CR, Bäckström R, Brill S, Fletcher D, Fodor L, Gordon DB, Komann M, Konrad C, Leykin Y, Pogatski-Zahn E, Puig MM, Rawal N, Ullrich K, Volk T, Meissner W. PAIN
OUT: the making of an international acute pain
registry. Eur J Pain