Background
The concept of quality is easily understood; however, defining it is challenging because quality is highly contextual. Deming,1 who refused to define quality in a few words or a sentence, stated that “the quality of any product or service has many scales”.1(p.169) In the context of health care, the formulation of a definition of quality is difficult for the same reason and has been a perpetual problem among healthcare managers and researchers.1-6 This has led to two approaches to defining quality in health care – generic and disaggregated definitions.4 Generic definitions are broad and all-encompassing, whereas disaggregated definitions recognize the multi-dimensionality of the concept and focus on individual components.4 Quality improvement (QI) is “a structured organizational process for involving personnel in planning and executing a continuous flow of improvements to provide quality care that meets or exceeds expectations”.7(p.4) It follows that the measurement of quality is central to providing direction for any QI effort.7-9 Although a definition of quality is not required before attempting to improve it, it is essential to have a definition of quality before measuring it.10 Donabedian11 argues that “we cannot assess quality until we have decided with what meaning to invest the concept. A clear definition of quality is the foundation upon which everything is built”.11(p.450) Indicators can then provide quantitative measures and a basis for improvement in care and the process by which patient care is delivered.12 To avoid invalid measurement of quality, it is important to differentiate between activity, performance and quality indicators.10 Owen10 explains that “activity indicators merely allow descriptive analysis of activity …”,10(p.77) for example, the total number of emergency calls responded to by an ambulance service within a 24-hour period. “Performance indicators add a value component in a sense that they measure the degree to which endeavor is optimized, but this is not necessarily linked to a definition of quality […]”,10(p.77) for example, ambulance service expenditure per person, that is the total expenditure of an ambulance service per person in the population it serves. Quality indicators are used to make judgments about quality and are based on evidence or consensus that the indicator can be used to evaluate quality of health care.13 For example, for a patient with suspected acute coronary syndrome, a 12-lead electrocardiograph is acquired within 10 min of an ambulance service arriving on the scene. Quality indicators can be classified in a range of different ways; however, Donabedian's Model of structure, process and outcome of medical care is widely accepted as the pre-eminent model for the measurement of quality in health care.7,14 Donabedian15(p.46) defined these as follows:
Structure: “… the conditions under which care is provided. These include: 1. Material resources […]; 2. Human resources […]; 3. Organisational characteristics […]”.
Process: “… the activities that contribute to health care – including diagnosis, treatment, rehabilitation, prevention, and patient education – usually carried out by professional personnel, but also including other contributions to care, particularly by patients and their families”.
Outcomes: “… changes (desirable or undesirable) in individuals or populations that can be attributed to health care”.
Balancing indicators are a less frequent but valuable type of measure that looks at a system from a different direction. Murray and Provost16 describe that balancing indicators measure related aspects of the system as a particular outcome and/or process is improved, and could be used to identify unintended effects or explain other reasons for success.
Pre-hospital care is the care provided to patients with real or perceived emergency or urgent care needs from the point of initial contact with an ambulance service until care is concluded or transferred.10 Thus, ambulance services are often the first point of access to the healthcare system and are faced with a wide variety of health problems.17 At a minimum, an ambulance service consists of a communication and control center, equipped vehicles with appropriately trained clinicians to provide pre-hospital care including transport and receiving facilities to continue relevant care.18 Pre-hospital care systems are categorized into two main types, based on the the qualifications of the clinicians: the Anglo-American system staffed by paramedics or emergency medical technicians and the Franco-German system in which physicians provide pre-hospital care.19 The discipline of paramedicine has evolved significantly over the last few decades. The roles and capabilities of paramedics have gained increased recognition and are adapting to the broader needs of the community and healthcare systems.20-22 As a result, care for low-acuity patients can in some cases be concluded in the out-of-hospital setting or these patients may be transported to healthcare facilities other than emergency departments.23,24 Nevertheless, ambulance services continue to provide pre-hospital emergency care which often cannot be definitive, and therefore transport to appropriate hospitals is required.
Historically, most ambulance services have focused on meeting response-time and other time-interval targets as well as increasing the survival rate from out-of-hospital cardiac arrest (OHCA) as indicators of pre-hospital care quality. The Council of Ambulance Authorities (CAA) is the representative body for the principal statutory providers of ambulance services in Australia. Its activities include “developing a body of knowledge through research, exchange of information, monitoring and reporting and maximizing opportunity for the application of standards providing for improved quality”.25(para.3) Currently, the CAA only has a few nationally agreed indicators.26 These are forwarded to the Productivity Commission. Chapter 9 of the Commission's 2016 Report on Government Services reports on government services for fire events and emergency ambulance events (pre-hospital care, treatment and transport).27 The ambulance events performance indicator framework provides information on equity, efficiency and effectiveness.27 Within effectiveness, the framework provides a means to report on clinical quality by measuring indicators associated with the delivery of clinical interventions and treatment. These are limited to cardiac arrest survival rate, cardiac arrest survival to hospital discharge and pain management. The report states that “clinical indicators have been identified as a key area for development in future reports”.27(p.9.51) The Australian Commission on Safety and Quality in Health Care (ACSQHC) was created by Health Ministers in 2006 to lead and coordinate healthcare safety and QI in Australia.28 The ACSQHC has incorporated aspects of pre-hospital care into some of their clinical care standards and associated indicators.29,30 However, there are no specific indicators focusing on pre-hospital care quality.
While the few indicators currently being measured nationally address important aspects of clinical care bundles for potential or actual time-critical patients or victims of OHCA, there is a need for more evidence-based and less simplistic indicators reflecting the quality of pre-hospital patient care, especially in terms of outcomes.12,31-34 Furthermore, these indicators need to address a wider spectrum of patients and incorporate the expanding scope of pre-hospital practice. Considering the diverse settings in which this relatively young discipline provides patient care, the utilization of a more comprehensive set of acceptable, feasible, sensitive, valid and reliable quality indicators is not only challenging, but also urgently required. This scoping review forms the initial part of a wider research project that aims to develop a set of quality indicators to measure, evaluate and benchmark the quality of pre-hospital care in Australia.
Inclusion and exclusion criteria
Population
The current scoping review will consider definitions and descriptions of pre-hospital care quality and performance/quality indicators developed for ambulance services providing pre-hospital care. The services may be in the form of a paramedic system (Anglo American model) or an emergency physician system (Franco-German model). Only countries that are similar to Australia in terms of economy (high-income based on The World Bank open data35) and healthcare system (Western) will be considered.
Concept
The concepts of interest are quality and quality indicators. This review will consider indicators that are intended to measure performance and/or quality only, that is indicators utilized to measure activity will be excluded.
Context
The context is pre-hospital care provided by ambulance services. This scoping review will consider definitions and descriptions of pre-hospital care quality. It will consider performance/quality indicators developed specifically for ambulance services. Indicators developed for other emergency or medical organizations, such as fire and rescue services or emergency departments, will be excluded.
Types of studies and sources
The current review will consider primary and secondary research in any paradigm and utilizing any methods as well as text and opinion. The review will also consider documents developed by quality units in health care or benchmarking organizations/committees/groups and ambulance service governance bodies. Documents written by ambulance services for their own, service-exclusive purposes will be excluded.
Search strategy
The search strategy aims to find both published and unpublished studies. A three-step search strategy will be utilized for each of the two objectives of this review, that is two searches will be conducted. For each search, an initial limited search of PubMed and CINAHL will be undertaken followed by analysis of the text words contained in the title and abstract, and of the index terms used to describe the articles. A second search using all identified keywords and index terms will then be undertaken across all included databases. Third, the reference list of all identified reports and articles will be searched for additional studies. Only English language papers will be included in this review due to this being the only language all reviewers understand as well as time and budget constraints. Databases will be searched from the year 2000, as modern QI in health care developed mostly thereafter.7 To supplement the above structured search, website searches of quality in health care or benchmarking organizations/committees/groups and ambulance service governance bodies, such as Emergency Medical Services (EMS) Compass® (www.emscompass.org), Institute for Healthcare Improvement (www.ihi.org) and the Association of Ambulance Chief Executives (www.aace.org.uk), will be performed. Furthermore, experts in the field of study will be consulted.
The databases to be searched include the following:
- PubMed
- CINAHL
- Embase
- Scopus
- Cochrane Library
- Web of Science.
The search for unpublished studies will include the following:
- ProQuest Dissertations and Theses
- Open Thesis
- Networked Digital Library of Theses and Dissertations
Initial keywords to be used will be as follows: emergency medical service∗, emergency medical technician∗, EMS, medical emergenc∗, emergency care, pre-hospital, pre-hospital, out-of-hospital, at the scene, ambulance∗ and paramedic∗
Combined with the following initial keywords for objective 1 search: health care quality, access and evaluation
Combined with the following initial keywords for objective 2 search: health care quality indicator∗, outcome and process assessment, benchmarking, quality indicator∗, quality measure∗, performance indicator∗, performance measure∗, clinical indicator∗, clinical measure∗, effectiveness indicator∗, effectiveness measure∗, structure indicator∗, structure measure∗, process indicator∗, process measure∗, outcome∗ indicator∗ and outcome∗ measure∗
Selection for inclusion will be assessed by two independent reviewers in accordance with the above inclusion/exclusion criteria. Any disagreements that arise between the two reviewers will be resolved through discussion or with a third reviewer if required.
Data extraction
Two charting tables have been developed to extract information relevant to the two review objectives (Appendix I), as recommended by the Joanna Briggs Institute (JBI) reviewer manual for scoping reviews.36 These charting tables will be piloted before data extraction and may be further refined during the review.
Data synthesis
For review objective 1, results will be presented as a map of attributes of pre-hospital care quality. For review objective 2, the presentations will be in tabular form. The table will list indicators, show what type of indicator each is (structure, process, outcome or balancing) and categorize them in accordance with components of their respective frameworks. Furthermore, the table will indicate when and where the indicator was developed, what EMS system it was intended for, its purpose and method of development. A second table, with combined framework components and indicators (and their type and frequency of occurrence in the literature), will then be synthesized. Duplicate indicators will be eliminated. Elimination will be done by consensus between the reviewers and, in case of disagreement, a third reviewer. Each section (review objectives 1 and 2) will be accompanied by a narrative summary.
Appendix I: Draft charting tables
Acknowledgements
The authors would like to thank Mr. James Pearce who will be the second reviewer.
References
1. Deming WE. Out of the crisis. Cambridge, MA: Massachusetts Institute of Technology; 1986.
2. Donabedian A. Evaluating the quality of medical care.
Milbank Mem Fund Q 1966; 44 3:166–206.
3. Blumenthal D. Quality of health care – Part 1: Quality of care – what is it?
N Engl J Med 1996; 335 12:891–894.
4. Campbell S, Roland M, Buetow S. Defining quality of care.
Soc Sci Med 2000; 51 11:1611–1625.
5. Cooperberg MR, Birkmeyer JD, Litwin MS. Defining high quality health care.
Urol Oncol 2009; 27 4:411–416.
6. Australian Institute for Health and Welfare, Australian Institute for Health and Welfare. Definitions of safety and quality of health care. 2016; Available from:
http://www.aihw.gov.au/sqhc-definitions/. [Cited February 23, 2016].
7. Sollecito WA, Johnson JK. McLaughlin and Kaluzny's continuous quality improvement in health care. 4th ed.Burlington, MA: Jones & Bartlett Learning; 2013.
8. Steffen GE. Quality medical care: a definition.
J Am Med Assoc 2003; 260 1:56–61.
9. Harris W, Lohr K. Defining quality of care.
Medicare a strategy for quality assurance. Washington, DC: National Academic Press; 1990.
10. Owen RC. The development and testing of indicators of prehospital care quality. Manchester: University of Manchester; 2010.
11. Donabedian A. Explorations in quality assessment and monitoring volume 3: the methods and findings of quality assessment and monitoring. Ann Arbour, Michigan: Health Administration Press; 1985.
12. Mainz J. Defining and classifying clinical indicators for quality improvement.
Int J Qual Health Care 2003; 15 156:523–530.
13. Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care.
Qual Saf Health Care 2002; 11 4:358–364.
14. Lloyd R. Quality health care: a guide to developing and using indicators. Sudbury, MA: Jones and Bartlett; 2004.
15. Donabedian A. An introduction to quality assurance in healthcare. Oxford: Oxford University Press; 2003.
16. Murray S, Provost LP. The health care data guide. 2011; San Francisco: Wiley, 481.
17. Wankhade P, Mackway-Jones K. Springer International Publishing, Ambulance services: leadership and management perspectives. Cham: 2015.
18. Brennan JA, Krohmer JR. American College of Emergency Physicians. Principles of EMS systems. 3rd ed.Sudbury, MA: Jones & Bartlett Learning; 2015. 19.
19. Dick WF. Anglo-American vs. Franco-German emergency medical services system.
Prehosp Disaster Med 2003; 18 1:29–35. 37.
20. O’Brien K, Moore A, Dawson DA, Hartley PR. An Australian story: paramedic education and practice in transition.
Australas J Paramed 2014; 11 3:1–13. Article 4.
21. O’Meara P, Stirling C, Ruest M, Martin A. Community paramedicine model of care: an observational, ethnographic case study.
BMC Health Serv Res 2016; 16 1:39.
22. Pollock A. Wankhade P, Mackway-Jones K. Historical perspectives in the ambulance service.
Ambulance services – leadership and management perspectives. Cham: Springer International Publishing; 2015. 17–29.
23. Krumperman K, Weiss S, Fullerton L. Two types of prehospital systems interventions that triage low-acuity patients to alternative sites of care.
South Med J 2015; 108 7:381–386.
24. Snooks H, Kearsley N, Dale J, Halter M, Redhead J, Cheung WY. Towards primary care for non-serious 999 callers: results of a controlled study of “Treat and Refer” protocols for ambulance crews.
Qual Saf Health Care 2004; 13 6:435–443.
25. Council of Ambulance Authorities. Council of Ambulance Authorities Inc. 2015; Melbourne: Available from:
http://www.caa.net.au/. [Cited November 1, 2015].
26. Waters D. (Chief Executive Officer, Council of Ambulance Authorities) Enquiry. Message to: Robin Pap, University of Adelaide. 2016. [about 1 screen].
27. Australian Government, Productivity Commission, Steering Committee for the Review of Government Service Provision. Report on government services. Vol. D. 2016; Available from:
http://www.pc.gov.au/research/ongoing/report-on-government-services/2016/emergency-management/rogs-2016-volumed-emergency-management.pdf. [Accessed November 25, 2016].
28. Australian Commission on Safety and Quality in Health Care. Governance. 2016; Sydney: Australian Commission on Safety and Quality in Health Care, Available from:
http://www.safetyandquality.gov.au/about-us/governance/. [Cited May 21, 2016].
29. Australian Commission on Safety and Quality in Health Care. Acute coronary syndromes clinical care standard. 2014; Sydney: Australian Commission on Safety and Quality in Health Care, 6–7.
30. Australian Commission on Safety and Quality in Health Care. Acute stroke clinical care standard. 2015; Sydney: Australian Commission on Safety and Quality in Health Care, 7.
31. O’Meara P. A generic performance framework for ambulance services: an Australian health services perspective.
Aust J Paramed 2012; 3 3:4.
32. El Sayed MJ. Measuring quality in emergency medical services: a review of clinical performance indicators.
Emerg Med Int 2012; 2012:161630.
33. Price L. Treating the clock and not the patient: ambulance response times and risk.
Qual Saf Health Care 2006; 15 2:127–130.
34. Australian Council for Safety and Quality in Health Care. Charting the safety and quality of health care in Australia. 2004; Canberra: Commonwealth Australia, Available from:
http://www.safetyandquality.gov.au/. [Accessed November 25, 2016].
35. The World Bank. The World Bank open data. 2016; Washington, D.C.: The World Bank, Available from:
http://www.worldbank.org/. [Cited November 23, 2016].
36. Peters MDJ, Godfrey CM, McInerney P, Baldini Soares C, Khalil H, Parker D. The Joanna Briggs Institute Reviewers’ Manual 2015: Methodology for JBI Scoping Reviews. Adelaide: Joanna Briggs Institute; 2015.
