Secondary Logo

Journal Logo


Evaluations of healthcare delivery models in Australia: a scoping review protocol

Roseleur, Jacqueline1,2; Partington, Andrew2,3; Karnon, Jonathan1,2

Author Information
doi: 10.11124/JBISRIR-D-19-00094
  • Free



Health expenditure in Australia increases at a faster rate than economic growth. Over the last 25 years, government spending on health has increased from 15.7% of taxation revenue to 24.1% and health expenditure increased from 6.5% to over 10% of total economic activity.1 While there exists sensible debate as to whether this trend in spending is sustainable,2 there has been, and likely will continue to be, a scarcity of resources and an inability to fund all healthcare expected to be beneficial. It is important to assess the value of alternative options for the allocation of scarce healthcare resources across the whole health system. Economic evaluation is an important aspect in a decision-making framework, as the costs and health benefits of alternative investment options must be considered alongside other factors, such as the equitable allocation of resources.

In Australia, there are formal processes utilized to assess the value of new pharmaceuticals and medical services to inform decisions on the listing of new items on the Pharmaceutical Benefits Schedule (PBS) and Medicare Benefits Schedule (MBS), respectively. Some state health departments have sought to define the way in which economic information is used by health service planners and decision-makers; for example, in the contexts of “model of care” development and “horizon scanning” as part of a commissioning cycle.3-5 However, there are no formal processes for making decisions on whether to fund new, evaluated healthcare delivery models. This imbalance in decision-making processes for health technologies and healthcare delivery models undervalues the available evidence on healthcare delivery models, and leads to the sub-optimal allocation of resources between new health technologies and healthcare delivery models within the Australian health system.

Health service evaluations are undertaken across the Australian health system and are funded by the National Health and Medical Research Council, governments, local health services and charities.6 These evaluations provide evidence that should be assessed with the aim of informing funding decisions that could improve health services.7 The validity and relevance of such evaluations to local contexts needs to be considered, which is the role of formal decision-making processes.

The aim of this proposed scoping review is to identify and describe the evidence base of published primary, comparative evaluations undertaken in Australia of healthcare delivery models that require the employment of additional healthcare practitioners (either to replace existing practitioners of another type or to provide new services). For example, medication reviews by pharmacists in emergency departments or patients’ homes have the potential to reduce hospitalizations, but require the investment of additional healthcare practitioners.8,9 The identified evaluations will potentially be significant inputs to aid in formal decision making processes for the assessment of new healthcare delivery models. The review has a particular focus on healthcare delivery models that could be funded by primary health networks (PHNs) and local hospital networks (LHNs) in Australia. The PHNs fund programs aimed at improving access to coordinated, high-quality primary health care. On the other hand, the LHNs fund hospital-based services, including outreach services, such as home-based rehabilitation.

The proposed search strategy can be adapted to inform searches of international literature for evidence on health care delivery models in specific clinical areas. The review will also identify gaps in the evidence base to inform research funding priorities.

A search was conducted in February 2019 for broad scoping and systematic reviews of health care delivery models in the following databases: JBI Database of Systematic Reviews and Implementation Reports, Cochrane Database of Systematic Reviews, PubMed, Epistemonikos and the Cumulative Index to Nursing and Allied Health Literature (CINAHL). A protocol for a scoping review of systematic reviews of alternative service models for the delivery of healthcare services in high-income countries was identified.10 The proposed scoping review provides a more specific review of primary comparative studies (not systematic reviews) in only Australia. Therefore, the two reviews will be complementary. The broader review could be used to identify general international areas with strong evidence, while this review will identify specific Australian studies within selected areas.

Review questions

The following review questions will guide this scoping review:

  • i) What primary, comparative evaluations of healthcare delivery models that require the employment of additional healthcare practitioners (either to replace existing practitioners of another type or to provide new services) have been undertaken in Australia?
  • ii) In which clinical areas have these evaluations been undertaken?
  • iii) Which healthcare practitioner categories have been studied in these evaluations?
  • iv) In which healthcare settings have these evaluations been undertaken?
  • v) Which research methods have been used in these primary, comparative evaluations?

Inclusion criteria

The population, concept and context (PCC) framework has been used to guide this review.11


Interventions for all population groups with an existing health condition (disease or risk factor), irrespective of age or sex, will be included. Interventions aimed at primary prevention will be excluded, including health promotion activities such as school-based exercise or nutrition programs, HIV testing in at-risk populations, screening in healthy populations, and immunization campaigns. Such interventions are excluded because the review aims to inform resource allocation decisions made by PHNs and LHNs in Australia and these entities are not responsible for the funding of primary prevention programs.


For the purposes of this review, included healthcare delivery models must aim to provide an existing service using an alternative set or new mix of healthcare practitioners, or a new service or form of treatment. To be included in the review, healthcare delivery models must require the employment of additional healthcare practitioners (either to replace existing practitioners of another type or to provide new services). This includes cases of task-shifting in which activities performed by one section of the workforce (e.g. doctors) are transferred to another (e.g. nurses), which would result in altered staffing structures within a service. An alternative example would be the use of telemedicine to provide services to a rural population with unmet needs. Interventions that aim to improve services with existing staffing structures, through either training or the introduction of protocols, will be excluded.


Included studies must be undertaken solely within Australia. Studies in all settings will be included. These may include, but are not limited to: primary care, specialist care, emergency departments, inpatient care, community care and aged-care facilities.

Types of sources

Included evaluations must include a comparator, involving the observation of outcomes over a similar period of time for studies in which either:

  • multiple groups of individuals receive treatment via two or more alternative healthcare delivery models for the condition of interest; or
  • one group receives treatment via a healthcare delivery model and another group receives no treatment for the condition of interest.

Appropriate study designs include randomized controlled trials, observational cohort studies and pre-post studies. Reviews, opinion papers and letters will be excluded.


The aim of mapping available evidence in this review is consistent with the purpose of scoping reviews.12 The JBI Reviewer’s Manual will be used to conduct this study and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Review (PRISMA-ScR) checklist will be used to report findings.11,13

Search strategy

A two-step search strategy was conducted. First, the review team conducted a number of small searches in PubMed to test various search terms and combinations of terms.14 Then, after identification of the most relevant terms, additional text words and index terms of relevant articles were identified and included in the final search strategy. The review team does not intend to search the reference lists of papers identified, or to contact the authors of articles that contain incomplete data. The final search strategy for PubMed is presented in Appendix I. Search strategies for the other databases used are available from the corresponding author.

Information sources

The following databases will be searched for citations published in English: PubMed, Embase and CINAHL. The search will be limited to articles published since 2008 to ensure relevance to the current Australian health system. No gray literature searches will be undertaken due to limited capacity.

Study selection

Citations will be imported into EndNote V8.2 (Clarivate Analytics, PA, USA) where duplicates will be removed. The remaining citations will be imported into Rayyan (Qatar Computing Research Institute, Doha, Qatar). As a large number of citations are expected due to the inclusion of general keywords, the title and abstract review will be undertaken in two steps. First, one reviewer will screen the full set of titles, with a second reviewer screening 10% of titles to ensure agreement between the two reviewers. Second, the results will be imported into Covidence (Covidence, Melbourne, Australia) where each citation abstract will be independently reviewed by two reviewers using the pre-specified inclusion criteria around the population, concept and context. Disagreements will be resolved through consensus. Thereafter, screening and selection of full-text articles will occur. The first 10% of full-text articles will be reviewed by two reviewers. The remaining full-text articles will be reviewed by a single reviewer. If a reviewer is unable to make a decision, the article will be discussed with a second reviewer. Reasons for excluding studies at this stage will be recorded and reported in the final review.

Data extraction

An extraction form will be developed using Microsoft Excel (Redmond, Washington, USA). A draft is presented in Appendix II. Data extraction will be performed by two reviewers for the first 10% of articles. The remaining data extraction will be completed by a single reviewer. Study characteristics will include authorship, year and journal of publication and study location (e.g. state(s) in which study was conducted). Study design will include type of study, clinical areas (as defined by the Medical Board of Australia), healthcare setting, intervention, whether the comparator was usual care, workforce category (e.g. category of healthcare practitioners) and study sample size. No outcomes will be extracted.

Data presentation

The results of the data extraction will be presented in a table. The extracted data will be reviewed to inform the aggregation of data categories, for example, clinical area, intervention type and sample size. Descriptive statistics will be utilized to summarize the characteristics of the studies, presenting percentages and frequencies relating to the year of publication, study type, clinical area, healthcare setting, intervention, comparator usual care (yes/no), workforce category and study sample size. Univariate and multivariate descriptive statistics will be presented; for example, analyzing clinical area, healthcare setting and workforce category by study type and sample size.

A narrative summary of the studies will be prepared, focusing on the frequency and study design features of published studies by clinical area, healthcare setting and workforce category. Case study examples of how the results of the review could be used to inform more detailed reviews of studies in specific areas will be presented from a PHN and LHN perspective.


This research is conducted by JR, AP and JK, Flinders University for the NHMRC Partnership Centre for Health System Sustainability (Grant ID #: 9100002) administered by the Australian Institute of Health Innovation, Macquarie University. Along with the NHMRC, the funding partners in this research collaboration are: The Bupa Health Foundation; NSW Ministry of Health; Department of Health, WA; and The University of Notre Dame Australia.

Although the NHMRC, The Bupa Health Foundation, NSW Ministry of Health, Department of Health, WA and The University of Notre Dame Australia, have provided in-kind and financial support for this research, they have not reviewed the content and are not responsible for any injury, loss or damage however arising from the use of, or reliance on, the information provided herein. The published material is solely the responsibility of the authors and does not reflect the views of the NHMRC or its funding partners.

Appendix I: Search strategy for PubMed

Appendix II: Draft data extraction tool


1. Australian Institute of Health and Welfare. Health expenditure Australia 2016–17 [Internet]. 2018 [cited 1 March 2019]. Available from:
2. Richardson JR. Can we sustain health spending? Med J Aust 2014; 200 (11):629–631.
3. NSW Agency for Clinical Innovation. Understanding the use of health economics: an ACI framework [Internet]. 2013 [cited 1 March 2019]. Available from:
4. SA Health. Health Technology Assessment Policy Directive [Internet]. 2015 [cited 1 March 2019]. Available from:
5. Department of Health & Human Services (Victoria). Evaluation guide, Centre for Evaluation and Research [Internet]. 2017 [cited 1 March 2019]. Available from:
6. Fradgley EA, Karnon J, Roach D, Harding K, Wilkinson-Meyers L, Chojenta C, et al. Taking the pulse of the health services research community: a cross-sectional survey of research impact, barriers and support. Aust Health Rev 2019; doi: 10.1071/AH18213. [Epub ahead of print.].
7. Karnon J, Partington A, Horsfall M, Chew D. Variation in clinical practice: a priority setting approach to the staged funding of quality improvement. Appl Health Econ Health Policy 2016; 14 (1):21–27.
8. Briggs S, Pearce R, Dilworth S, Higgins I, Hullick C, Attia J. Clinical pharmacist review: a randomised controlled trial. Emerg Med Australas 2015; 27 (5):419–426.
9. Roughead EE, Barratt JD, Ramsay E, Pratt N, Ryan P, Peck R, et al. Collaborative home medicines review delays time to next hospitalization for warfarin associated bleeding in Australian war veterans. J Clin Pharm Ther 2011; 36 (1):27–32.
10. Jessup RL, O’Connor DA, Putrik P, Rischin K, Nezon J, Cyril S, et al. Alternative service models for delivery of healthcare services in high-income countries: a scoping review of systematic reviews. BMJ Open 2019; 9 (1):e024385.
11. Peters MD, Godfrey CM, McInerney P, Soares CB, Khalil H, Parker D. Aromataris E, Munn Z. Chapter 11: Scoping reviews. Joanna Briggs Institute, Joanna Briggs Institute Reviewer's Manual [Internet]. Adelaide: 2017.
12. Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 2015; 13 (3):141–146.
13. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018; 169 (7):467–473.
14. Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs 2014; 114 (5):49–56.

Australia; decision-making; health services; health workforce; healthcare delivery models