Abstracts for the GIN and JBI Conference, Adelaide Australia, 2019

Methodological rigor of public health guideline recommendations on vitamin D and calcium intakes – a pilot study

Zhaoli Dai¹, Cynthia Kroeger¹, Sally McDonald¹, Lisa Bero¹

¹The University of Sydney

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: We compared vitamin D and calcium recommendations in public health guidelines and appraised methods used for the recommendations described.

Background/Importance: It is unclear to what extent vitamin D and calcium recommendations among adults at risk of osteoporosis vary across guidelines and whether guideline recommendation development follows a consistent and rigorous process.

Method: We conducted a comprehensive systematic search to locate food-based dietary guidelines (FBDGs) and bone health guidelines (BHGs) published in 2000–2019. We adopted 25 criteria in the following domains from the World Health Organization (WHO) Handbook for Guideline Development (2014) to evaluate included guidelines: guideline development group representation, conflict of interest (COI), systematic methods, evidence substantiation, recommendation development, and external review. Two reviewers independently performed data extraction.

Results/Lessons Learnt: Among ten FBDGs and ten BHGs from various countries, recommended intakes differ for vitamin D (range: 400–2000IU/d) and for calcium (range: 600–1500 mg/d). Ten FBDGs and four BHGs recommend dairy consumption for calcium, and all BHGs recommend supplementation of vitamin D (except Japan) and of calcium. One BHG (India) provides specific recommendation for sun exposure (vitamin D synthesis). Further, adherence to all 25 WHO criteria ranged from 16–88% with the top three being Australia FBDG (88%), US FBDG (84%) and Australia BHG (80%). Only 40% of the guidelines disclosed COI, 24% assessed quality of body of evidence and 20% were reviewed externally.

Conclusion: Our preliminary results suggest that recommendations for vitamin D and calcium vary substantially among countries and that very few guidelines meet all criteria recommended by the WHO.

Overcoming the evidence practice gap for objective cardiovascular assessment – A framework for developing consensus-based recommendations

Bethany Gower¹, Danielle Girard¹, Andrew Maiorana^2,3, Kade Davison¹

¹University of South Australia,²Curtin University,³Fiona Stanley Hospital

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To develop an e-Delphi protocol to provide consensus-based recommendations in the absence of research evidence for the assessment of cardiovascular function to inform therapeutic exercise interventions.

Background: We systematically reviewed all published guidelines and found almost no evidence-base to inform objective assessments for therapeutic exercise prescription. This means that many clinicians are operating within an evidence practice gap.

Method: Experts from the Australian Exercise Physiology and Physiotherapy community were identified by the research team based on pre-defined criteria or allowed to self-nominate from relevant professional networks to participate in a three-round online survey (e-Delphi). Experts first listed and then rated objective assessments of cardiovascular function that they deemed necessary in clinical assessment to inform safe and effective exercise prescription.

Results: A total of 31 experts were recruited to complete the online survey. 40 objective assessments were suggested. Only 6 items reached consensus for use. The third round allowed experts to reconsider others’ ratings and rationale and shift their rating toward the median and potentially reach threshold consensus, or to reaffirm their position and validate the consensus process. Experts provided extensive rationale to support their assertions and allow informed but unbiased consensus development.

Conclusion: In the absence of research evidence, the collective experience and insights of an expert panel can fill an evidence practice gap using the protocol outlined here. Further research is needed to establish an accepted framework for the translation of expert consensus outcomes and guiding statements for use by practitioners and consumers.

Impact of network meta-analysis on decision making in nice clinical guidelines: an example from the VTE prophylaxis guideline

Sedina Lewis¹, Jessica Glen², Dalia Dawoud³, Carlos Sharpin¹, Xavier Griffin⁴

¹National Guideline Centre, Royal College Of Physicians,²Health Research Council of New Zealand,³Clinical Pharmacy Department, Faculty of Pharmacy, Cairo University,⁴Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To assess the value of network meta-analyses (NMAs) in informing decision-making during guideline development.

Background: Hospital-acquired venous thromboembolism (VTE) is an important health concern in the UK and worldwide. NICE published a clinical guideline (NG89) in 2018 that aims to provide guidance for healthcare professionals on how to prevent hospital-acquired VTE. NG89 updates its predecessor (CG92) which was published in 2010. For two of the surgical populations covered within the guidelines, NMAs were conducted.

Method: Evidence identified in CG92 and after its publication was systematically reviewed. Bayesian NMAs were conducted across two orthopaedic surgery populations (elective hip replacement and elective knee replacement) for critical outcomes: deep vein thrombosis, pulmonary embolism and major bleeding. Summary statistics including ranks for VTE prophylactic strategies compared to no prophylaxis and their associated 95% credible intervals (CrIs) were calculated.

Results: The NMA results were presented to the guideline committee including rank plots which provided a visual aid for decision-making. There was uncertainty around the NMA results as demonstrated by overlapping 95% CrIs. The results informed the economic models developed to assess cost-effectiveness, integrating clinical and economic evidence. The committee utilised the NMA and economic model results when making recommendations. The uncertainty of the NMA results was noted during the decision-making process and contributed to the strength of the recommendations. The incorporation of new evidence in the NG89 analyses led to the introduction of new prophylactic strategies in clinical practice.

Conclusion: When appropriate, NMA should be conducted by guideline developers to inform evidence-based recommendations.

Survey of Practice guidelines on the International Practice Guideline Registry Platform

Xufei Luo^1,2, Mengshu Wang³, Chenhao Guo⁴, Meng Lv², Xiao Liu¹, Jianjian Wang¹, Jingyi Zhang^1,2, Qi Zhou¹, Kehu Yang¹, Qianling Shi⁴, Yaolong Chen¹

¹Evidence-Based Medicine Center of Lanzhou University; WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,²School of Public Health of Lanzhou University,³The First Hospital of Lanzhou University,⁴The First Clinical Medical School of Lanzhou University

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To investigate the registered guidelines on the IPGRP.

Background/Importance: The International Practice Guideline Registry Platform (http://www.guidelines-registry.org, IPGRP) was launched in 2013. Five years past, there is no detailed information about the registered guidelines.

Methods/Process: We searched IPGRP website manually from January 1, 2014 to March 20, 2019 and used Excel to perform data extraction and analysis.

Results/Lessons Learnt: There were 151 guidelines and consensuses included. Of these, 58 (38.4%) were identified as guidelines, 42 (27.8%) were expert consensuses, 31 (20.5%) were Chinese Medicine guidelines, 9 (6.0%) were rapid recommendation guidelines, 7 (4.7%) were patient guidelines and 4 (2.6%) were other guidelines. In terms of guideline grading system, about three-quarters guidelines (77.5%) used GRADE system, seven (4.7%) used the JBI (Joanna Briggs Institute) rating system, and seven (4.7%) used the OCEBM (Oxford Centre for Evidence-based Medicine, OCEBM) rating system. Sixteen guidelines (10.6%) did not report any rating system. Most of guidelines (87.4%) reported that their evidence based on systematic reviews. Only 48 guidelines (31.8%) submitted their protocols at IPGRP. 121 guidelines (80.1%) reported the funding information, of which 10 guidelines (6.6%) were funded by pharmaceutical companies.

Conclusion: During the past 5 years more and more guidelines from different medical associations were registered on the IPGRP. The IPGRP will provide additional methodological support to guideline developers. However, the platform only used by Chinese guideline developer, we should enhance the dissemination and let more people know the IPGRP.

Use of the results of systematic reviews, ‘overviews of systematic reviews’, and network meta-analyses to inform clinical practice guideline recommendations

Barbara Mintzes

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: We evaluated if and how systematic reviews (SRs), ‘overviews of systematic reviews’ (OSRs) and network meta-analyses (NMAs) are used in clinical practice guidelines (CPGs) and their included recommendations.

Background: Guidelines are systematically developed recommendations to assist practitioner and patient decisions about treatments for clinical conditions. High quality and comprehensive SRs, OSRs, and NMAs represent the best available evidence. Many guideline developers, such as the World Health Organization and the Australian National Health and Medical Research Council, recommend the use of research syntheses to underpin recommendations.

Methods: We searched for CPGs using TRIP and Epistemonikos. We included CPGs that provide at least two recommendations for the management of any clinical condition. We noted whether SRs, ORS, or NMAs were used and cited for each recommendation. We conducted a critical appraisal of how the guidelines developed recommendations.

Results: 478/691 records were screened at full text to identify 50 CPGs. The majority of CPGs were open access, published in a medical journal, and had conflicts of interest. Less than one quarter of CPGs used a SR process to inform the recommendations, and less than one percent used OSRs or NMAs. One third of CPGs reported: objectives and eligibility criteria based on PICO elements, a full search strategy, and a process for screening studies. Two thirds reported two or more databases searched. One third used or adapted the GRADE approach.

Conclusions: Our study found that less than one quarter of Clinical Practice Guidelines made the use of the different review types.

Assessing evidence from prognostic studies in guideline development: a pragmatic approach to improve the quality of guideline methodologists.

Irina Mostovaya¹, Julitta Boschman¹, Bernardine Stegeman, Elvira den Breejen¹, Koert Burger¹

¹Knowledge Institute Of The Federation Of Medical Specialists

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To evaluate the workshops on assessing prognostic evidence held at the Knowledge Institute of the Federation of Medical Specialists (KIFMS) in the Netherlands.

Background/Importance: Evidence from prognostic studies can be helpful in answering various clinical questions. It is often not clear, even for experienced methodologists, how to match a clinical question to a prognostic research question and how to assess the certainty of the body of evidence in guideline development.

Pending final guidance by the GRADE Working Group on the GRADE assessment of prognostic studies, a consistent and pragmatic approach to assess prognostic evidence is needed.

Methods/Process: The Special Interest Group (SIG) on guideline methodology at KIFMS developed a workshop for guideline methodologists to become more proficient in assessing prognostic evidence for guideline development. Relevant topics were selected from earlier consultations to the SIG and supplemented by topics raised in interviews with colleagues.

Results: The core elements of our approach are:

1. consider a prognostic research question for an indication (for diagnostic test or intervention) question;
2. formulate a research question that contains at least patient population and outcome;
3. focus on results indicating the performance of prognostic factors and/or impact of a model;
4. assess the quality of the studies and the certainty in the evidence by applying the GRADE criteria.

The workshop will be held and evaluated during the autumn of 2019.

Conclusion: An internal training program will show whether a pragmatic approach to assessing prognostic evidence will lead to a quality improvement of guideline methodologists.

Are WHO guideline panels gender and geographically balanced? A review of WHO guidelines published from 2008 to 2018

Joshua Vogel¹, Meghan Bohren², Dena Javadi³

¹Burnet Institute,²Melbourne School of Population and Global Health, University of Melbourne,³Alliance for Health Policy and Systems Research

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Background: One of WHO's core global health leadership functions is developing evidence-based guidelines. We aimed to determine the composition of WHO guideline panels by assessing the gender and country of affiliation of guideline contributors for all WHO guidelines (2008 to 2018).

Methods: Guidelines were identified from the WHO Guideline Review Committee database and PubMed. Two reviewers extracted guideline-level (title, publication year) and individual-level (contributor name, role, gender, country of affiliation) data. Guideline roles included: guideline development group (GDG) member, GDG chair, WHO steering group, external reviewer, or methodologist. Descriptive statistics were calculated to assess gender and geographical balance. Outcomes of interest were the proportion of GDG members and chairs that were female, and geographic distribution.

Findings: 230 guidelines were included (13,329 individuals). The median proportion of female GDG members per guideline was 47.1% (IQR: 35.7–56.3%). Seventy-five guidelines (34.2%) had less than 40% female GDG members. Among 243 GDG chairs, 145 (59.7%) were male and 96 (39.5%) were female. Most GDG members were affiliated with institutions in the Americas (31.1%) or Europe (27.2%). The median proportion of GDG members from low- and middle-income countries (LMICs) per guideline was 42.9% (IQR 33.3–53.1%).

Conclusion: Women are underrepresented across most WHO guideline roles, suggesting systematic gender inequalities. Individuals affiliated with institutions in LMICs in the Eastern Mediterranean, African, Southeast-Asian and Western-Pacific regions were underrepresented. This analysis is a starting point to reflect on improving gender and geographical diversity in guideline development and ensuring WHO “walks the talk” on equity and diversity.

Updating Australian guidance on the management of early breast cancer - development and application of a ‘meta-guideline’ approach

Sarah Norris¹, Debra Hector³, Kristina Harvey¹, Agnes Wilson¹, Zoe Thomas¹, Melina Willson², Christine Giles³, Helen Zorbas³

¹Hereco,²Cochrane Breast Cancer Group, NHMRC Clinical Trials Centre, The University of Sydney,³Cancer Australia

CS1 - Guideline Development Advances and Processes, Riverbank Room 2, October 31, 2019, 11:00 AM - 12:30 PM

Background: Cancer Australia published Clinical Practice Guidelines for the management of early breast cancer in 2001. Cancer Australia sought to update this guidance using rigorous, transparent methods avoiding the resource-intensive, time-consuming processes typically used to develop/update Clinical Practice Guidelines (CPGs).

Objective: The development of a streamlined method to produce up-to-date guidance for the management of early breast cancer in Australia.

Method: A ‘meta-guideline’ approach was developed, broadly consistent with the ‘Adopt, adapt or start from scratch’ NHMRC (2017) methodology. The ADAPTE framework was varied to allow the inclusion of multiple source guidelines and to eliminate the need for reappraisal of supporting systematic reviews. The meta-guideline approach was applied to 84 evidence-based international/national guidelines identified via comprehensive searches. Forty-seven guidelines had employed adequate evidence-to-recommendation methods (AGREE II) and were included as source guidelines. All recommendations from these guidelines were extracted and sorted by topic and sub-topic. Source recommendation(s) were then selected for each sub-topic based on recency of the evidence review, consistency with similar recommendations, and applicability to the Australian healthcare context. Source recommendations were adopted or adapted accordingly, or new recommendations were noted as requiring development.

Findings: The meta-guideline approach is predicated on the availability of multiple, high-quality, relevant guidelines across the range of required topics. Although informed by internationally recognised principles of guideline development, the resulting guidance is not a true CPG.

Conclusion: The meta-guideline approach reduces duplication of effort while maintaining currency of recommendations. It is a pragmatic solution for developing high-level guidance with a broad clinical scope under resource-constrained conditions.

Reducing the evidence-practice gaps in stillbirth prevention across Australian maternity hospitals; The Safe Baby Bundle

Christine Andrews^1,2, David Ellwood⁹, Philippa Middleton³, Adrienne Gordon⁴, Michael Nichol⁵, Caroline Homer⁶, Jonathan Morris⁷, Glenn Gardener², Michael Corry², Miranda Davies-Tuck⁸, Vicki Flenady^1,2

¹Mater Research Institute- University Of Queensland,²Centre of Research Excellence in Stillbirth,³South Australian Health and Medical Research Institute,⁴University of Sydney,⁵Royal North Shore Hospital,⁶Burnet Institute,⁷Kolling Institute of Medical Research,⁸Hudson Institute of Medical Research,⁹Griffith University

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To reduce stillbirth rates from 28 weeks’ gestation in Australia.

Background: The Safe Baby Bundle (SBB) has been proposed to address the priority evidence practice gaps in stillbirth prevention. Similar bundles in the UK have shown a 20% reduction in stillbirth rates.

Methods: Phase 1: Survey of Australian maternity services and expert panel workshops to inform bundle elements and implementation strategies. Phase 2: Baseline data collection and implementation in partnership with jurisdictional health departments of New South Wales, Victoria, and Queensland (commencing mid-2019). An interrupted time series analysis will examine the change in stillbirth rates. Pre and post-implementation surveys will evaluate women's experience of care, clinician acceptability and experience, and impact on hospital resource use. Benchmarking of key performance indicators will flag priority areas for practice change through audit and feedback.

Results: Five priority areas identified: 1) improved detection and management of impaired fetal growth; 2) increasing awareness and management of women with decreased fetal movements; 3) smoking cessation support; 4) provision of maternal safe sleeping advice; and 5) improved decision-making around timing of birth for women with risk factors. Jurisdictional implementation strategies will vary, with Victoria using Institute of Healthcare Improvement collaborative methodology and other states utilising existing frameworks for change.

Conclusion: We anticipate the SBB will save the lives of 100 babies each year across the participating centres. Evaluating the SBB implementation across three Australian States affords a unique opportunity to assess variation between jurisdictions and effectiveness of strategies used to facilitate uptake.

A systematic survey to identify and evaluate the effects of approaches to integrate guideline and quality assurance scheme development

Miranda Langendam¹, Thomas Piggott², Holger Schünemann²

¹Amsterdam UMC/University of Amsterdam,²McMaster University

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Background: The European Commission Joint Research Centre was asked to develop a methodological framework, based on guideline (GL) development and quality assurance (QA), for improving the quality of care of colorectal cancer. To support development of this methodological framework we performed a systematic survey of the literature to identify and describe approaches to develop GL recommendations and quality indicators (QI) and/or QA schemes in an integrated framework, and to evaluate the effects of such an integrated approach on patient/individual, process and structure outcomes.

Methods: We identified a completed systematic review on this topic that was current until April 2010. We agreed with the author to collaborate on an update of that review applying the original eligibility criteria.

Results: The search (original and update) resulted in 22,606 references. We did not find articles on methods to integrate GL development and QA scheme. Twenty-five articles reported on methods of GL-based QI development. We did not find any evaluation studies. All but one method article started with describing the QI development process and how evidence from GL was used. In one article developing recommendations was followed by formulating QI for each recommendation, it was unclear however how recommendations and QI were linked. None of the articles reported a framework where QA/QI development was part of the question formulation of developing the recommendations.

Conclusion: We found approaches where recommendations and QI development were linked, but none of the articles reported a well-defined conceptual framework where QA/QI development was fully integrated in the guideline process.

15 years of translating the evidence into practice: What the Veterans’ MATES program has achieved and the lessons learned.

V Tammy LeBlanc¹, Natalie Blacker¹, Anna Moffat¹, Jemisha Apajee¹, Mhairi Kerr¹, Kerrie Westaway¹, John Barratt¹, Emmae Ramsay¹, Nicole Pratt¹, Lisa Kalisch Ellett¹, Elizabeth Roughead¹

¹Quality Use of Medicines and Pharmacy Research Centre, University Of South Australia

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Aim: The Veterans’ Medicines Advice and Therapeutics Education Services (Veterans’ MATES) program aims to improve the health of the veteran community through better use of medicines and health services.

Background: For 15 years, the Australian Government Department of Veterans’ Affairs (DVA) has provided Veterans’ MATES; a knowledge translation program that utilises administrative claims data, evidence based medicine and stakeholder engagement to bridge the evidence-practice gap.

Methods: Utilising DVA health claims data, we provide patient-specific feedback to medical practitioners. Supporting evidence-based educational material is developed by a clinical panel, peer reviewed, and overseen by a national editorial committee. Veterans are mailed educational materials. A national call centre and website support inquiries from stakeholders. Reference group meet twice a year to provide advice on topics, interventions, practitioner and consumer needs. Evaluation includes surveys and observational studies.

Results: Fifty-four educational topics targeting more than 300,000 veterans and 33,000 doctors have been implemented. Successes include 116,000 fewer patient months of treatment with hypnotics, 3, 800 additional bone mineral density tests and 38,000 fewer patient months of treatment with proton pump inhibitors. Engaging veterans leads to more success; we found a 27% relative increase in the uptake of renal function tests (1.27, CI:1.08–1.50, p=0.003) among high risk veterans who became aware of the issue as a result of the program.

Conclusion: Factors contributing to Veterans’ MATES success include the program's grounding in behavioural theory, a focus on clearly identified problems, strong stakeholder engagement and the promotion of a consumer - health professional partnership.

Formal criteria to assess the validity of performance indicators

Craig Grime¹, Mark Minchin¹, Gill Leng¹

¹National Institute for Health And Care Excellence

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To agree criteria to assess the validity of performance indicators.

Background: The NICE indicator programme develops indicators suitable for use in national performance frameworks for primary and secondary healthcare in England. Traditionally, informal criteria have been used by our advisory committee to assess the validity of indicators in development. As part of a process review, proposals were raised for formal validity criteria to help ensure transparency and consistency.

Methods: A literature review was undertaken to identify existing checklists or tools to formally appraise the validity of indicators. Key components from an existing framework were discussed by a working group consisting of academics with expertise in health informatics, and a modified framework was made subject to public consultation.

Results: An indicator appraisal framework was developed across 6 domains consisting of 14 criteria for the assessment of the validity:

• Importance (national priorities, known variation in practice, clinical outcomes and under- or over-treatment)
• Risk (unintended consequences)
• Evidence base (quality and alignment)
• Specification (components and minimum population level)
• Feasibility (repeatability, accuracy and data burden)
• Acceptability (attribution and potential for improvement).

Conclusion: Developers of performance frameworks or indicators should use formal criteria to assess the validity of indicators in development and those being considered for retirement.

Using stratification to develop pay-for-performance indicators

Mark Minchin¹, Craig Grime¹, Gill Leng¹

¹NICE

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To develop pay-for-performance (P4P) indicators for people with diabetes that support individualised care and reduce the risks associated with under and overtreatment.

Background: Financial incentives in the form P4P schemes are an increasingly common method to support quality improvement¹. Whilst guidelines for the care of people with diabetes increasingly highlight the need for individualised care², until recently the national P4P indicators used in England's QOF were applied uniformly across all people with diabetes. Using a single set of P4P indicators across a diverse population risks overtreatment in people with more complex care needs and undertreatment in people with less complex care needs^3,4.

Method: New indicators were developed through a multidisciplinary group. The new indicators include adjustments based on an individual's frailty status. For example, incentivised HbA1c targets for people without frailty were informed by targets in clinical guidelines. For people with moderate and severe frailty, who are at greater risk of overtreatment, indicators were developed to focus on individualised care.

Result: The new stratified indicators were negotiated into the 2019/20 QOF supported by approximately £40 million of incentives.

Conclusion: Stratifying P4P indicators helps support individualised care, reduces the risk of both under and overtreatment, and can mitigate against some of the unintended consequences of using P4P to support quality improvement.

1. Eijkenaar. (2012) P4P health: an international overview. MCRR 69(3)251–76.
2. T2 diabetes:management, NICE 2007
3. Kearney et al. (2017). Overtreatment and undertreatment. BJGP 67(663):442–443
4. Strain et al. (2018). T2 diabetes in older people. DM 35:838–845

Bringing Two Worlds Closer Together: A critical analysis of the opportunities for an integrated guideline and quality assurance methodological framework for the European Commission guidelines and quality assurance programs

Holger Schünemann

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Background: Although quality indicators (QI) are frequently derived from guidelines the corresponding parties do not collaborate sufficiently. To optimize workflow, understanding of the needs of the partners and implementation of guideline recommendations through quality assurance (QA), the work should be aligned methodologically. Learning from the European Commission Initiative on Breast Cancer (ECIBC), our objective was to bring both worlds together under an overarching framework to inform future European Commission projects.

Methods: We undertook a mixed-methods study and held in-person key informant meetings. First, we conducted a feasibility study that included a survey, interviews and review of manuals for an integrated GL and QA scheme that would support the European Commission. This study drew from an assessment of the ECIBC in which the two expert groups worked mostly separately. Second, we updated a systematic review to identify and appraise methods for integrating GL and QA development. Third, we then used the findings to inform an evidence brief and to draft a methodological framework that was discussed in a three-day structured meeting with an expert group.

Using the same language to talk about quality assurance: a proposal for a common terminology

Holger Schünemann

CS1 - Quality Assurance and Performance Indicators, Riverbank Room 3, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To clarify terminology used in the quality assurance field and to agree on a common language to define indicators.

Background: In 2017, the European Commission Joint Research Centre was asked to develop a methodological framework, based on guideline development and quality assurance, for improving the quality of care of colorectal cancer (the European Commission Initiative on Colorectal Cancer). A group of experts was invited to collaborate in identifying and discussing potential challenges and gaps in its development. Inconsistency and confusion about terminology used to define indicators emerged from the beginning.

Process: 23 experts in the fields of guideline development and quality assurance participated in a 3-round consensus exercise: 1. face-to-face discussion starting from the European Commission Initiative on Breast Cancer glossary (ECIBC - https://ecibc.jrc.ec.europa.eu) and a systematic review on guideline-based development of quality indicators; 2. email exchange and teleconference discussion; 3. final agreement through teleconference.

Results: A 3-level terminology was proposed: 1. Quality Indicators, constructs used as a guide to monitor, evaluate, and improve the quality of services (Quality of Life); 2. Performance Measures, tools that quantify or describe measurable elements of practice performance (SF36); 3. Performance Indicators, measurable units of practice performance (score of 56 on SF36).

Conclusion: Lack of clarity in the way indicators are defined brings confusion in the field of quality assurance. This consensus exercise proposes structured definitions and creates a consistent terminology that could be easily adopted and recognised. It involves international experts and is based on definitions already in use, better explaining their meaning and interactions.

Meta-analysis of incidence rate ratios, challenges and issues

Timothy Barker¹, Zachary Munn¹

¹Joanna Briggs Institute, The University Of Adelaide

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: The aim of this research project is to explain the methods undertaken to extract, transform, analyse and synthesise a non-standard format of data, from a body of literature that was poorly reported.

Background: Meta-analysis is a widely used statistical procedure to combine data from multiple independent, yet comparable studies. Whilst meta-analyses can be complex in their statistical functioning, performing one has been simplified due to the use of multiple software tools and programs that require little statistical foreknowledge. One key requirement for these software programs to function, is the correct input of the appropriate data. This usually comes in the form of events over total sample (dichotomous) or the mean and standard deviation (continuous). Whilst there is simplicity in collecting these values from well-reported primary studies, the difficulty increases when these values are not clearly reported, or raw values are not provided. Whilst meta-analysis remains a requirement of many systematic reviews, performing one appropriately can present a significant challenge when synthesising non-standard forms of data.

Process: This research project conducted a meta-analysis using one of these non-standard forms of data; incidence rate ratios and the natural log of the standard error. In most cases, this data was indirectly obtained from the source, after careful statistical transformation and analysis.

Lessons Learnt: We present some recommendations for clearer and more structured reporting standards for data, and provide some tips and hints for other researchers who may be struggling to synthesise data from poorly-reported sources.

Advances in the 2019 2nd Edition of the Cochrane Handbook for Systematic Reviews of Interventions

Miranda Cumpston¹, Julian P T Higgins², James Thomas³, Jackie Chandler⁴, Tianjing Li⁵, Matthew J Page⁶, Vivian A Welch^7,8,9

¹Cochrane Public Health,²University of Bristol,³EPPI-Centre, UCL,⁴Bangor University, UK,⁵Johns Hopkins Bloomberg School of Public Health,⁶School of Public Health and Preventive Medicine, Monash University,⁷The Campbell Collaboration,⁸Bruyère Research Institute,⁹School of Epidemiology and Public Health, University of Ottawa

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: To outline key developments in the 2019 substantive update to the Cochrane Handbook for Systematic Reviews of Interventions.

Importance: The methods outlined in the Cochrane Handbook underpin systematic reviews used by many key guideline developers around the world. This first major update in a decade includes new chapters and reflects the state of knowledge and current directions in evidence synthesis methodology, driven by the need to answer questions of relevance to decision makers and guideline developers.

Process: Over 100 authors and editors contributed to the new Handbook, reflecting a decade of academic progress. The updates encompass core methods underpinning all reviews, such as framing questions and data collection, as well as advanced and specialised techniques.

Results: The Handbook includes new chapters covering network meta-analysis, synthesis without meta-analysis, methods to consider equity and complex interventions. A new approach to planning review questions and analysis incorporates in-depth consideration of comparisons, intervention components and outcome selection. To enhance the usability of the evidence, the Handbook includes improved guidance on reporting and updating of reviews and covers advances in risk of bias assessment (Version 2.0), reporting biases, incorporating non-randomised studies, qualitative evidence and living systematic reviews.

Conclusion: Review authors and users of reviews are encouraged to explore the guidance in the new Handbook and consider how it can enhance reviews and the guidelines they support, from straightforward clinical questions to complex questions about public health and health systems.

A Hybrid Systematic Review Methodology for Assessing the Suitability of Psychometric Tools

Sonia Hines¹, Mikaela Cibich¹, Timothy Carey¹

¹Centre For Remote Health, Flinders University

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: In order to assess the extent to which social and emotional wellbeing (SEWB) screening tools used for Indigenous Australians are consistent with an Indigenous-developed SEWB framework,¹ we pragmatically conducted a ’hybrid’ review.

Background: We needed to identify relevant instruments and assess the quality and frequency of use of these instruments in order to draw conclusions about the way that Indigenous SEWB is measured. No existing review methodology met all our needs.

Method: We used standard systematic review methods to construct and conduct searches, and screen for relevant articles. We added an additional level of screening to find studies that used a psychometric instrument and retrieved those articles and the associated instruments and their domain information. Some instruments were used in several different studies and the number of times used was also recorded. The instruments were then evaluated using the COSMIN instrument. The domains of validated tools were checked against the framework and color-coded green for ‘fits’, amber for ‘fits somewhat’, and red for ‘does not fit’.

Results: This enabled us to identify instruments in terms of the quality and frequency of their use and form conclusions about the quality of the instrument and its fidelity to Indigenous concepts of SEWB.

Conclusion: The addition of conceptual triangulation to elements of scoping and measurement tools review methods enabled us to draw conclusions about the state of the science in this field.

1. Dudgeon P, Milroy H, Walker R. Working together: Aboriginal and Torres Strait Islander mental health and wellbeing principles and practice. 2nd ed. Perth, 2014.

Evaluation of the JBI scoping reviews methodology by current users

Hanan Khalil¹, Marsha Bennett², Christina Godfrey³, Patricia Mc Inerney⁷, Zachary Munn⁵, Andrea Tricco⁴, Micah Peters⁶

¹Monash University, Monash Rural Health, The Centre for Chronic Diseases Management: a Joanna Briggs Institute Centre of Excellence,²LSU Health, New Orleans, School of Nursing,³Queens University, Faculty of Health Sciences,⁴University of Toronto, DALLA LANA School of public health,⁵Joanna Briggs Institute Faculty of Health and Medical Sciences The University of Adelaide,⁶School of Nursing and Midwifery I Division of Health Sciences University of South Australia,⁷WITS University, Centre for Health Science Education

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: The primary objective of the survey was to evaluate the experiences of users of the JBI scoping methodology.

Background/Importance: In 2014, the Joanna Briggs Institute (JBI) published comprehensive guidance for the conduct and reporting of scoping reviews based on previous frameworks and guidance. To assist with future refinements to the JBI guidance, this study was undertaken to evaluate users’, experiences of using the methodology.

Method: An online survey administered using Qualtrics, a secure survey platform was distributed Invitations to a total of 51 corresponding authors registered in the Joanna Briggs Database of Systematic reviews and Implementation reports. Data was analysed using descriptive statistics.

Results: Thirty-one participants completed the survey (response rate of 61%). Forty two percent participants reported that the scoping review they had been involved with had taken between 6–12 months. The majority of the respondents (up to 89%) also noted that the step by step approach in the JBI manual was well documented and detailed strongly agreed or agreed (about the adequacy of each step). Eighty seven percent of participants stated that their scoping reviews led to further work

Conclusion: The overall evaluation by the respondents of the JBI scoping review methodology provided positive but also constructive feedback on the JBI methodological guidance for scoping reviews. After reviewing the feedback, further improvement in inclusion criteria and presentation of the results sections have been identified as areas for further guidance.

Integrating quantitative and qualitative evidence: The JBI Mixed Methods Systematic Review methodology

Lucylynn Lizarondo

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Background: Mixed methods systematic reviews (MMSR) have become increasingly popular to clinicians, researchers and policymakers as they provide an innovative approach for addressing important questions in health care. In 2014, methodological guidance for undertaking such reviews was published by the Joanna Briggs Institute (JBI); this has undergone significant revisions to reflect current methodological developments.

Aim: To present JBI's revised methodology for conducting MMSR.

Methods: The JBI mixed methods review working group reconvened to update the JBI guidance. The group reviewed the literature and held teleconferences and face-to-face discussions to reach consensus on every step of the methodology. The revised guidance was ratified by the JBI International Scientific Committee in 2019.

Results: Two approaches to how quantitative and qualitative data are combined and the overall integration of quantitative and qualitative evidence were adopted – convergent integrated and convergent segregated. The convergent integrated approach refers to a process of combining extracted data from quantitative studies and qualitative studies and involves data transformation. Conversely, the convergent segregated approach consists of conducting a separate quantitative synthesis and qualitative synthesis, followed by integration of the evidence derived from the syntheses. The nature/type of question(s) that is (are) posed in the systematic review informs the approach the reviewer should follow for the mixed methods synthesis.

Conclusion: The methods for synthesis and integration in MMSR are complex and more work is required to shed light this emerging but rapidly evolving methodology. Further methodological work is currently being undertaken by the working group to advance this field.

Stratification by quality induced selection bias in a meta-analysis of clinical trials

Jennifer Stone¹

¹Australian National University

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: This study examines if restricting primary meta-analyses to studies at low risk of bias or presenting meta-analyses stratified according to risk of bias is indeed the right approach to explore potential methodological bias.

Background: Stratification by quality (known as sensitivity analyses or exploratory subgroup analyses) is a commonly advocated analytical strategy after risk of bias assessment. Support for this strategy followed research that reported inconsistency in the effect estimates between high- and low- quality trials across 25 quality scales. The inconsistency demonstrated across strata when using different scales has been attributed to quality scores, and stratification continues to be used using domain judgements.

Methods: Reanalysis of the impact of quality subgroupings in an existing meta-analysis based on 25 different scales.

Results: We demonstrate that quality stratification itself is the problem because it induces a spurious association between effect size and precision within stratum. Studies with larger effects or lesser precision tend to be of lower quality, a form of collider-stratification bias (stratum being the common effect of the reasons for these two outcomes) that leads to inconsistent results across scales. We also show that the extent of this association determines the variability in effect size and statistical significance across strata when conditioning on quality.

Conclusion: We conclude that stratification by quality leads to a form of selection bias (collider-stratification bias) and should be avoided. We demonstrate consistent results with an alternative method that includes all studies.

Mapping the MASTER scale for quality assessment against 6 comparative study design critical appraisal tools from Joanna Briggs Institute.

Jennifer Stone¹

¹Australian National University

CS1 - Systematic Review Advances, Hall C - Plenary, October 31, 2019, 11:00 AM - 12:30 PM

Aim: We aim to compare this scale against 6 of the Joanna Briggs Institute's (JBI) key critical appraisal tools, with specific interest in coverage of the MASTER scale and identifying areas for improvement in the JBI tools.

Background: Evidence synthesis requires assessment of the methodological quality of included studies. There are numerous quality assessment tools to choose from depending on study design and research topic. We previously developed a quality assessment scale, MASTER, which is intended for broad use across analytic study design and research topics.

Methods: The safeguards within each JBI checklist were compared to each of the 40 safeguards from MASTER and mapped to the MASTER scale safeguard that it matched. This mapping of safeguards was depicted in a frequency table. Stacked bar charts were used to classify the distributions of domains and subdomains based on the mapped safeguards across JBI checklists. Finally, the MASTER scale mapped safeguards were cross tabulated against the JBI safeguards stratified for each of the 6 JBI checklists. Safeguards that remained unmapped, safeguards that mapped to the same MASTER scale, and safeguards deemed not for use as a risk of bias item were described.

Results/Lessons Learnt: We found that the MASTER scale has good coverage of the 6 key comparative study design critical appraisal tools from the JBI and provides a unified framework for quality assessment in evidence synthesis.

Conclusion: We conclude by highlighting the merits and limitations of this approach, including areas for improvement in the next revision of JBI comparative study tools.

Use of a deliberative dialogue to facilitate knowledge translation in the assessment of harms associated with systemic treatment of mental health disorders during and after pregnancy

Kristina Harvey¹, Jenny Ring¹, Sue Campbell¹, Nicole Highet², Marie-Paule Austin³

¹Health Research Consulting,²Centre of Perinatal Excellence,³St John of God Perinatal Mental Health Research Unit & University of New South Wales

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop a systematic, transparent approach that used the knowledge of a range of subject matter experts, to supplement the available, but limited, outcomes data for systemic therapies in the perinatal period.

Background: The Australian clinical practice guidelines Mental Health Care in the Perinatal Period (2017) included a range of interventions to manage a range of mental health disorders: anxiety, depression, schizophrenia, bipolar disorder, postpartum psychosis, and borderline personality disorder. The development of recommendations regarding the use of systemic therapies (pharmacological, complementary, electroconvulsive therapy, transcranial magnetic stimulation) in pregnant and breast-feeding women requires particular consideration of potential harms to the fetus and breast-feeding infant as well as to the woman herself, but it was recognised that evidence in the perinatal period was likely to be lacking for particular interventions and/or disorders.

Methods: We pre-defined a governance structure, a process of deliberative dialogue, and an analytic framework for considering the possible adverse effects of systemic therapies on the mother and/or baby during the perinatal period. The body of evidence included data from pregnant women and evidence-based guidance for non-pregnant individuals with particular mental health disorders.

Results: The deliberative, evidence-informed approach was easy to follow for stakeholders and facilitated the development of recommendations, particularly the explicit, overall weighing of benefits and harms for specific interventions.

Conclusion

We developed a practical evidence-to-decision framework to support the development of trustworthy guidelines in an area with limited evidence. The approach may be useful when incorporating evidence from populations other than a guideline's target population.

Piloting the inclusion of qualitative and other forms of evidence in a national epilepsy guideline

Kay Cooper^1,2, Sarah Florida-James³, Carolyn Sleith⁴, Pamela Kirkpatrick^1,2

¹Robert Gordon University,²Scottish Centre for Evidence-based, Multiprofessional Practice: A Joanna Briggs Institute Centre of Excellence,³Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland,⁴Healthcare Improvement Scotland

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To pilot the inclusion of a range of evidence sources, including qualitative research findings, in the development of a SIGN guideline for the management of epilepsy in children.

Background: Guideline developers have traditionally relied on evidence from quantitative studies. However, the use of qualitative evidence for generating recommendations has increased in recent years. This allows guideline developers to consider issues such as feasibility, acceptability and service users’ perspectives, in addition to effectiveness, arguably making guidelines more relevant and implementable.

Method: During the initial phase of guideline development, two key questions were identified which could not be answered with quantitative evidence alone. A unique collaboration was established between academic researchers with expertise in mixed-methods systematic reviewing and the multidisciplinary, nationally representative guideline development group, in order to use the wide range of evidence sources identified to inform the recommendations for these questions. The recommendations were informed by a novel JBI mixed-methods systematic review and findings from a range of other sources, including various types of primary research and reviews.

Lessons Learnt: Including a wide range of evidence sources requires adequate resource and expertise. We encountered challenges with critical appraisal, grading the evidence, determining confidence in the findings, and presenting these aspects within the guideline format. Further methodological development work is recommended.

Conclusion: The methodology used to develop this guideline has received positive feedback from the guideline development group and during the independent peer-review process. The team are keen to further develop the methodology to overcome the limitations presented above.

Translating First Nations knowledge into cancer survivorship healthcare practices

Wendy Gifford¹, Delia McDonald¹, Chad Hammond², Roanne Thomas², Ian D. Graham³

¹University Of Ottawa, Faculty of Health Sciences, School of Nursing,²Univeristy of Ottawa, Faculty of Health Sciences, School of Rehabilitation Sciences,³University of Ottawa, Faculty of Medicine School of Epidemiology and Public Health

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To describe the process of translating Indigenous knowledge within a First Nations community into cancer survivorship healthcare practices to be culturally relevant and safe.

Background: First Nations peoples of Canada have demonstrated a resiliency to the historical effects of colonization, social and environmental exclusion, and lack of culturally safe and accessible healthcare services. Nevertheless, First Nations communities have higher rates of cancers, in part because of their ongoing marginalization and its health effects. Their realities, values, and knowledges has yet to be incorporated into the dominant policies and practices of Canada's health care systems.

Method: An advisory group from a Mohawk community was established (n = 8: Elders, traditional teachers, healthcare providers, cancer survivors, caregivers). Sixteen meetings over 14 months were held in community. Six meetings were audio-recorded and transcribed, including two focus groups (n = 24 participants). Transcribed meetings were analyzed inductively for key themes.

Results: The process involved: 1) collaboratively identifying community survivorship needs and proposing healthcare practices to address them; 2) incorporating traditional values and knowledge of health into the proposed practices; 3) honoring natural rhythms of tradition and community. Trust, data ownership and open consent needed to be established and continuously re-visited to develop collective action. Collaborative leadership was rooted in preserving traditional ways, creating a democratic voice that included diversity of minds, and sharing stories.

Conclusion: Our process facilitated translation of First Nations values and knowledge into healthcare practices to address community needs and build staff capacity to develop culturally relevant and safe healthcare.

Scoping for diverse forms of knowledge to inform responses to opioid overdoses

Karen MacKinnon^1,2, Bruce Wallace^1,3, Bernie Pauly^1,3, Carol Gordon^1,2, Diane Sawchuck^2,4, Sana Shahram^3,5, Rebecca Raworth¹, Heather Strosher³, Celeste Macevicius³

¹University of Victoria,²The University of Victoria Centre for Evidence-Informed Nursing and Health Care,³Canadian Institute for Substance Use Research,⁴Island Health,⁵Interior Health

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To share reflections on the process of completing a scoping review of health equity-oriented approaches that includes diverse forms of knowledge.

Background: Health inequities that result in opioid-related deaths are commonly reported in the literature and people who use substances frequently experience stigma and criminalization. However, there is a lack of knowledge about integrated approaches to guide responses by decisions makers and service providers.

Methods: A JBI affiliated group and a research centre focusing on substance use related harms collaborated for this review. The ultimate purpose of this scoping review was to identify literature that could be used to develop a comprehensive health equity-oriented approach for addressing the drug-related harms of overdose and stigma. Two research librarians and multiple preliminary searches helped locate relevant literature. We initially searched for existing approaches but also wanted to locate the research evidence that would support their use. In addition to research reports we identified expert opinion papers, health policy documents and program reports.

Lessons Learned: Discussion about complex concepts and diverse forms of knowledge was required. For example, we were not interested in studying participants who use substances in a problem-focused way but wanted to learn about strategies these experts identified. Piloting screening and data extraction forms was needed to help reviewers recognize a health equity-oriented philosophical approach.

Conclusion: This scoping review required careful consideration of complex concepts. The peer-review process for publishing our protocol in advance of the final review allowed us to improve the transparency and rigour of the process.

Engaging people with a learning disability (LD) in setting out priority areas for quality improvement in health and social care – experience to date.

Chris Bird¹, Anna Wasielewska¹, Stacy Wilkinson¹, Laura Delaney¹, S Pollard¹, Gillian Leng¹, Mark Minchin¹

¹National Institute for Health and Care Excellence

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: Evaluating lessons learned from involving people with a LD in setting out priority areas for quality improvement in health and social care.

Background: The National Institute for Health and Care Excellence (NICE) provides national guidance and advice on improving health and social care. Previously at GIN conference NICE presented the benefits of involving people with a LD in guideline development. This abstract describes lessons learned and benefits gained from engaging people with an LD as experts by experience in the process of developing quality standards (QS) which set out priority areas for quality improvement.

Method: The QS development process has been adapted to ensure experts by experience were engaged in a meaningful way. The adjustments included:

• external facilitation to allow experts by experience to prepare for committee discussions and provide feedback on draft documents
• extended committee meetings
• easy read materials presented during meetings
• easy read documents provided for comments.

Evaluation was built into the process to ensure lessons were learned from the start. Initial findings indicate:

• the process was a positive experience for the experts by experience
• committee members valued experts’ contribution
• extra time and resources were crucial to involve experts by experience
• simplified materials could benefit all committee meetings by reducing time commitment and resources.

Results: Evaluation is ongoing and additional findings will be available soon.

Conclusion: Involving experts by experience in developing QS added value and credibility to the process. It may support uptake and implementation of quality standards among relevant stakeholders.

A new approach for explicit judgments of values and preferences: define the benefit required for an intervention

Lise Helsingen¹, Reed Siemieniuk², Per Olav Vandvik³, Linan Zeng², Michael Bretthauer^1,3, Thomas Agoritsas^2,4, Gordon Guyatt²

¹Oslo University Hospital,²McMaster University,³University of Oslo,⁴University of Geneva

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Objective: We describe a seldom-used method for making explicit judgments of values and preferences, and use the approach taken in a recent guideline on colorectal cancer screening (BMJ Rapid Recommendations) as a worked example.

Background: Judgments about values and preferences must inform all guidelines. A key question is: “Given the expected harms and burdens, how much benefit do people in the target population require to choose an intervention?” Albeit crucial, this question is typically not addressed explicitly.

Methods: The guideline panel first considered harms and burdens of screening (faecal immunochemical testing (FIT) every year or every two years, sigmoidoscopy or colonoscopy) versus no screening, and completed surveys that informed thresholds of benefit above which most people would choose screening. Bearing these thresholds in mind, the panel examined the evidence and issued their recommendations.

Results: The panel inferred that at least half of people in the target population would require a colorectal cancer mortality reduction of 5 per 1000 to undertake FIT-screening, and a reduction of 10 per 1000 for sigmoidoscopy or colonoscopy. A difference in colorectal cancer mortality reduction of 10 or more per 1000 would prompt them to recommend colonoscopy or sigmoidoscopy over FIT, and a difference of 5 or more would prompt recommending FIT every year over FIT every two years and colonoscopy over sigmoidoscopy.

Conclusion: Setting a threshold for the required benefit before reviewing the evidence can minimize the influence of panelists’ pre-conceived beliefs regarding appropriate recommendations and enables explicit and transparent judgments regarding values and preferences.

Use of Mixed Methods Reviews to Address Service Design Issues in Clinical Guidelines

Jacqui Real¹

¹National Guideline Centre, Royal College of Physicians

CS2 - Diverse forms of knowledge to inform guidance, Riverbank Room 3, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To explore a mixed methods approach to complex service design issues in guideline development.

Background/Importance: Developing evidence-based recommendations on complex service design issues is a challenge in guideline development. The use of mixed methods reviews may be a suitable approach.

Methods/Process: The NICE chronic heart failure guideline update sought to address complex issues around the delivery of heart failure services. A mixed methods review approach was taken, incorporating both studies of interventions and qualitative studies to address a number of complementary review questions. Each component was synthesised separately before being brought together in the guideline committee's discussion and recommendations.

Results/Lessons Learnt: The mixed methods approach enabled the development of a cohesive set of guideline recommendations on the approach to care for people with heart failure. While specific recommendations about the necessary level of care were largely based on RCTs, the qualitative evidence was used to address related concerns of the committee around care continuity and managing transitions between care settings. Had a mixed methods approach not been taken, the guideline may have made isolated recommendations that failed to take into account patient preferences and experiences. The qualitative evidence was used to “join the gaps” between the evidence of an intervention's effectiveness and the patient's experience of that pathway.

Conclusion: The use of a mixed methods approaches successfully enabled the development of guideline recommendations on complex service design issues. This approach, while consistent with existing NICE methods and processes, could be used to greater effect in clinical guideline development.

A Certification and Credentialing Course for Guideline Panel Members: Setting a New Standard for Practice

Stephanie Duda¹, Robby Nieuwlaat¹, Matthew Ventresca¹, Nancy Santesso¹, Jan Brozek¹, Gillian Leng², Ina Kopp³, Holger Schünemann¹

¹McMaster University,²Guidelines International Network,³Association of the Scientific Medical Societies’ Institute for Medical Knowledge-Management (AWMF-IMWi), Philipps-University Marburg

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To illustrate the need for and benefits of guideline panel member certification as a requirement of panel participation.

Background/Importance: Individuals with varying backgrounds and level of experience currently develop guidelines. Their level of qualification is not managed or monitored. There is no widely accepted certification standard and guidelines are presently developed and published without specific formal training, impacting guideline quality.

Methods/Process: In partnership with G-I-N, McMaster University is developing a guideline development certification and credentialing program to train individuals and groups in guideline development methods. The first of four courses in the program, ‘Guideline Panel Member’ will be an introductory course covering select domains of the G-I-N-McMaster Guideline Development Checklist ^1,2. The course will be available as a half-day in-person workshop or as a self-paced online module, offering flexibility for trainees but will require passing a certification examination.

Results/Lessons Learnt: Course development is currently underway and we will present results in Adelaide. The certification program is expected to launch at the G-I-N 2020 conference in Toronto, Canada.

Conclusion: Guidelines are arguably more influential than single health care providers. The knowledge and competency required by all parties involved to do it well is sophisticated. Organizations that develop guidelines should require certification among panel members to ensure the production of high-quality guidelines and potentially reduce development time and resources.

1. GIN-McMaster Guideline Development Checklist. Available: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html

2. Schunemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ 2014;186(3):E123–42.

Develop the Reporting Guideline for Clinical Practice Guideline of Proposal: RIGHT for P

Xufei Luo^1,2, Xianzhuo Zhang³, Meng Lv², Kefeng Liu⁴, Yongjie Yang⁴, Xuan Yu¹, Kehu Yang¹, Yaolong Chen¹

¹Evidence-Based Medicine Center of Lanzhou University; WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,²School of Public Health of Lanzhou University,³The First Clinical Medical School of Lanzhou University,⁴The First Affiliated Hospital of Zhengzhou University

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop the extension of the RIGHT reporting guideline for proposal of CPGs.

Background/Importance: The proposals of clinical trial and systematic review can promote the transparency, avoid duplicated work and improve the quality, the same as Clinical Practice Guideline (CPG) proposal. However, how to report the proposal for CPG and what should be reported in a proposal are still unclear.

Methods/Process: The study was performed with the following steps: 1) systematically evaluate published proposals of guideline; 2) write protocols and register on the EQUATOR; 3) investigate the expectations of clinicians, researchers, and methodologists for the information of guideline proposal; 4) start three rounds of Delphi to select items; 5) hold a face-to-face consensus meeting.

Results/Lessons Learnt: We have finished the systematic analysis for CPGs’ proposals, and found that most published proposals were diverse in reporting contents and quality. In addition, we have established RIGHT for proposal working group including guideline developer, methodologists, clinicians, patients, medical editors and health economists, etc. Finally, the item pool of reporting guideline for proposal has been collected.

Conclusion: Nowadays more and more guidelines are developed to assists physicians and/or other health care practitioners and patients make decisions about appropriate health care for specific clinical circumstances. The checklist of RIGHT for P are expected to promote the transparency and improve the quality of CPGs.

A theoretical framework and competency-based approach to training in guideline development

Reem A. Mustafa^1,2, Rebecca Morgan², M. Hassan Murad³, Yngve Falck-Ytter⁴, Holger J. Schünemann², Philipp Dahm⁵, Shahnaz Sultan⁶

¹University Of Kansas Medical Center,²Department of Health Research Methods, Evidence, and Impact, McMaster University,³Evidence-based Practice Center, Mayo Clinic,⁴Case Western Reserve University, University Hospitals and VA Northeast Ohio Healthcare System,⁵Department of Urology, University of Minnesota,⁶Division of Gastroenterology, Hepatology, and Nutrition, University of Minnesota

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Background: There is an increasing requirement to develop guidelines using transparent, standardized, and rigorous methods. Consequently, a better understanding of the knowledge, skills and expertise necessary for guideline development is needed.

Objective: To describe a theoretical framework of knowledge and skills that are required for individuals to serve on a guideline panel or become a guideline methodologist.

Methods: Based on an iterative process and review of published manuscripts focused on guideline development, we identified competencies, sub-competencies, and milestones.

Results: Using a competency-based approach to training and the Dreyfus model of skill acquisition, we identified 3 competencies (and 12 sub-competencies): 1) Facilitate the development of guideline structure and setup; 2) Make judgments about the quality or certainty of the evidence; and 3) Transform evidence to a recommendation. Level 1 focuses on recognizing and acknowledging the importance of a specific skill or behavior. Levels 2 and 3 require learners to demonstrate progressive acquisition of knowledge and application to specific behaviors. Level 4 represents the individual who has acquired the requisite knowledge and can function independently while Level 5 represents the mastery/aspirational level.

Conclusion: We propose a competency-based education framework that will help (1) standardize the qualifications needed for individuals to serve on guideline panels or help to lead guidelines as guideline methodologists, (2) help develop curricula for teaching and training of guideline panel members, and (3) enable organizations to identify guideline methodologists with the relevant and appropriate level of knowledge and skills.

Overdue: Establishment of a Certification and Credentialing Program for Guideline Developers through a GIN-McMaster Partnership

Holger Schünemann

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To ensure the production of high-quality health care practice recommendations by training guideline group members, methodologists and trainers from around the world in a standardized and transparent fashion, with a focus on implementing modern, evidence-based guideline research.

Background/Importance: Guideline development processes vary substantially, and many do not meet basic quality criteria. Despite widespread and needed professional certification and training in health care, no certification program in guideline development or training in its methods exists.

Methods/Process: In response to the demand for high-quality healthcare guidelines, McMaster University in partnership with G-I-N will offer certification and credentialing courses in guideline development, including planning, implementation, evaluation and updating. Course material will be organised using the G-I-N-McMaster Guideline Development Checklist ^1,2. Four successive certification levels will be available: i) Guideline Panel Member; ii) Guideline Methodologist; iii) Master Guideline Developer; and, iv) Guideline Development Credentialing Instructor. Online and in-person training options will be offered for individuals or groups from guideline producing organizations.

Results/Lessons Learnt: Program development is currently underway and we will introduce the material in Adelaide. The four certification levels will be launched at the G-I-N 2020 conference in Toronto, Canada.

Conclusion: Benefits of certification and credentialing include the ability to improve guideline quality and potential to reduce time and resources required to develop a guideline.

1. GIN-McMaster Guideline Development Checklist. Available: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html

2. Schunemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ 2014;186(3):E123–42.

Extending the RIGHT statement for Reporting Adapted Practice Guidelines in Health Care: the RIGHT-Ad@pt Checklist

Yang Song¹, Monica Ballesteros², Laura Martínez García¹, Pablo Alonso-Coello^1,4,5, Robin WM Vernooij³, Thurayya Arayssi⁶, Soumyadeep Bhaumik⁷, Yaolong Chen^8,9, Francoise Cluzeau¹⁰, Davina Ghersi¹¹, Paulina Fuentes Padilla¹², Etienne V. Langlois¹³, Susan Norris¹⁴, Holger J. Schünemann⁵, Elie A Akl^5,15,16

¹Iberoamerican Cochrane Center,²Hospital Universitario Vall d’Hebron,³Department of Research, Netherlands Comprehensive Cancer Organisation (IKNL),⁴CIBER de Epidemiología y Salud Pública (CIBERESP),⁵Department of Health Research Methods, Evidence, and Impact, McMaster GRADE center, McMaster University,⁶Weill Cornell Medicine-Qatar,⁷The George Institute for Global Health,⁸Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University,⁹WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,¹⁰School of Public Health, Faculty of Medicine, Imperial College London,¹¹National Health and Medical Research Council,¹²Facultad de Medicina y Odontología, Universidad de Antofagasta,¹³Alliance for Health Policy and Systems Research, World Health Organization,¹⁴Guidelines Review Committee, World Health Organization,¹⁵Department of Internal Medicine, American University of Beirut,¹⁶AUB GRADE Center, American University of Beirut

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop an extension of the Essential Reporting Items of Practice Guidelines in Healthcare (RIGHT) statement for the reporting of adapted guidelines.

Background/Importance: Adaptation of guideline is increasingly used for the efficient development of contextualized recommendations. Although the RIGHT statement can be useful for reporting adapted guidelines, it does not address all the important aspects of the adaptation process.

Methods/Process: To develop the RIGHT-Ad@pt, we built on the RIGHT statement, used the available methodological research on guideline adaptation, and have been using a multistep process that includes: 1) establishment of a Working Group; 2) generation of an initial checklist; 3) optimization of the checklist; and 4) approval of the final checklist. To optimize the checklist we are conducting an initial assessment of adapted guidelines, and will be conducting semi-structured interviews, a Delphi consensus survey, an external review by guidelines developers and users, and a final assessment of adapted guidelines.

Results/Lessons Learnt: The interdisciplinary and intersectoral approaches have allowed the Working Group to finish the protocol and the initial checklist. We will present results of the initial assessment of adapted guidelines and the findings of the semi-structured interviews at the 15th G-I-N Conference (Adelaide, 2019).

Conclusion: RIGHT-Ad@pt checklist can improve the reporting of adapted guidelines, contribute to improve their quality, and facilitate their implementation. This checklist will be useful for guideline developers with their reporting, journals editors to improve the reporting of their publications, and guideline users to evaluate the completeness of the reporting of adapted guidelines.

How to most effectively and efficiently quality assure a guideline development process?

Nichole Taske

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To describe the NICE quality assurance process for guideline development and explore its impact on guideline quality.

Background: The NICE guideline quality assurance (QA) process has evolved organically since the guideline programme was established in 2001 and now comprises a number of key steps throughout the guideline development process. Independent quality assessment of NICE guidelines consistently report that NICE guidelines are high quality compared to other organisations (Tan et al. 2017). With an increasing focus on efficiency, evaluation on how to operationalize these QA steps more efficiently without compromising guideline quality is warranted.

Methods and process: An evaluation of the current NICE quality assurance process was conducted against international standards such as NEATS, AGREE II and RIGHT. Mapping was carried out to identify:

• Overlaps between international standards and NICE QA standards
• International standards not covered by NICE QA standards
• Additional NICE QA standards NICE not covered by international standards.

Results/Lesson learnt: NICE QA process has steps or standards that cover almost all international standards. Also, NICE QA process also has additional steps or QA steps that go beyond what are recommended by international standards.

Conclusion: NICE has a very robust and detailed quality assurance process and it is believed that such a robust QA process, including additional QA steps, is the key for good quality NICE guidelines. A further evaluation will be conducted to explore how to operationalize these QA steps in a more efficient manner without compromising the quality of guidelines.

Quality appraisal of clinical practice guidelines on gastric cancer

Yangqin Xun^1,2, Nan Yang^1,2, Qi Zhou^1,2, Yaolong Chen^1,2, Xuan Yu^1,2, Kehu Yang^1,2

¹Evidence-based Medicine Center Of Lanzhou University,²WHO Collaborating Centre for Guideline Implementation and Knowledge Translation

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To analyze available CPGs on gastric cancer with RIGHT checklist in order to evaluate their reporting quality.

Method: Methodological quality of the gastric cancer CPGs was appraised using the Appraisal of Guidelines for Research and Evaluation II instrument.

Background: Gastric cancer is the 4th common cancer. In many countries, there is a lack of appropriate clinical practice guidelines (CPGs) for the diagnosis and management of gastric cancer or the quality of the CPGs has not been assessed.

Methods: We systematically searched electronic databases including PubMed, Cochrane Library from the inception to April, 2019. The GIN database, the NGC, the SIGN, NICE and Google also were searched to identify additional potential guidelines.

The RIGHT instrument was used by two independent assessors to create a systematic appraisal in 22 items to determine the guidelines fulfilling the inclusion and exclusion criteria. Each item was rated as “Yes” for total compliance, “Unclear” for partial compliance or “No” for non-compliance, respectively. The number and proportion of reported items for each item were also calculated. Statistical analyses were produced using SPSS version 15.0 for Windows. Methodological quality of the included gastric cancer CPGs was appraised using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument. Agreement among reviewers was assessed by intra-class correlation coefficient. The number of recommendations, strength of recommendation, and levels of evidence were determined.

Results and conclusions: This study is ongoing and results will be presented at the Evidence summit as available.

Development of Reporting Items for Patient/Public Version of Guidelines: RIGHT-PVG

Xiaoqin Wang^1,2, Yaolong Chen^1,2, Elie Akl³, Ana Marus[Combining Caron]ic[Combining Acute Accent]⁴, Ružica Tokalić⁴, Yngve Falck-Ytter⁶, Amir Qaseem⁵, Myeong Soo Lee⁷, Madelin Siedler⁸, Sarah Barber⁹, Mingming Zhang¹⁰, Edwin SY Chan¹¹, Qi Zhou^1,2, Kehu Yang^1,2, Susan Norris¹²

¹Evidence-Based Medicine Center of Lanzhou University,²WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,³Department of Epidemiology and Population Health, American University of Beirut,⁴Department of Research in Biomedicine and Health, University of Split School of Medicine,⁵American College of Physicians,⁶Division of Internal Medicine, Louis Stokes Veterans Affairs Medical Center, Case Western Reserve University,⁷Clinical Medicine Division, Korea Institute of Oriental Medicine,⁸ Division of Physical Education and Exercise Science, University of South Florida,⁹World Health Organization Centre for Health Development,¹⁰Chinese Evidence-Based Medicine Centre, West China Hospital, Sichuan University,¹¹Cochrane Singapore, Singapore Clinical Research Institute,¹²Health Metrics and Measurement, World Health Organization

CS2 - Guideline Credentialing, Quality and Reporting, Riverbank Room 4, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop the RIGHT-PVG as a reporting checklist for patient and public versions of guidelines (PVGs) in healthcare.

Background/Importance: Patient and public versions of guidelines (PVGs) can help with individual decision making and enhance the patient–clinician relationship by providing easily understandable and reliable information. An increasing number of organisations are developing PVGs. However, the reporting of PVGs varies widely across these organisations. Standardized reporting of PVG could help in making them a more comprehensive and useful tool in healthcare. We have registered the study protocol on the EQUATOR network (http://www.equator-network.org/library/reporting-guidelines-under-development/#84).

Methods/Process: We developed the RIGHT-PVG reporting checklist as an extension of the Reporting Tool for Practice Guidelines in Healthcare (RIGHT) statement. We built on the methods recommended by the EQUATOR network. These included conducting a literature review, establishing an international multidisciplinary team, and running a modified Delphi process.

Results/Lessons Learnt: The multidisciplinary consensus group consists of 17 international experts, including three representatives of the public. Following the literature review and two rounds of Delphi survey, the RIGHT-PVG checklist included 17 items addressing the following 12 topics: title/cover/copyright, contact information, summary, introduction of the target topic, purpose and scope, link to the source guideline, recommendations, the strength of the recommendations and quality of the evidence, questions to ask, terms and abbreviations, funding, and conflicts of interest.

Conclusion: The RIGHT-PVG reporting checklist represents international consensus on items for reporting PVGs. The RIGHT-PVG working group encourages research on the validity, acceptability, and usability of the checklist.

Living guidelines - the quest for currency. Lessons from 16 years of the SIGN/BTS British guideline on the management of asthma

Beatrice Cant¹

¹Healthcare Improvement Scotland, Scottish Intercollegiate Guidelines Network

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To highlight the benefits/challenges of producing a ’living’ guideline and to present possible solutions.

Background: Keeping guidelines up to date is a major challenge for guideline developers, particularly when resources are limited. The ’living’ guideline approach is one way of maintaining currency for topics with fast-changing evidence but presents challenges of its own. This paper sets out the main lessons learned from 16 years of producing the SIGN/BTS British guideline on the management of asthma using a ’living’ guideline type approach (annual/biennial updating using adapted SIGN methodology) and illustrates this with examples from the 2018–19 guideline update.

Process: An account based on first-hand experience of producing SIGNs only ’living’ guideline.

Lessons Learnt: Producing a ’living’ guideline has had demonstrable benefits in terms of maintaining the clinical usefulness and impact of the guideline. It is, however, labour and time intensive for SIGN and the clinicians involved and requires additional staff resources. After 10 years, annual publication via a continuous updating cyle proved unsustainable and biennial publication was introduced. Although more practical, challenges remain. Published systematic reviews, once central to the ’living’ guideline process now pose more problems than solutions due to the numbers being published, duplication, and overlap with existing evidence. Entrenchment is a risk given the limited time to stand back and take stock and could undermine the guideline's currency and credibility. Selective updating is essential but risks leaving out-of-date, withdrawn or superceded evidence in non-updated sections of the guideline.

Conclusion: Careful consideration is needed before adopting this approach.

Developing efficient evidence surveillance for ‘living’ Stroke Guidelines

Steve McDonald¹, Kelvin Hill², Loyal Pattuwage², James Thomas³, Tari Turner¹

¹Cochrane Australia,²Stroke Foundation Australia,³EPPI Centre, University College London

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To evaluate the feasibility, accuracy and reliability of a surveillance system to identify evidence relevant to the Australian Clinical Guidelines for Stroke Management.

Background: Living guidelines have the potential to revolutionise the rapid translation of health research into policy and practice by continual surveillance and integration of new evidence – a task potentially enabled by innovations, such as machine learning and citizen science.

Methods: We developed broad-scope PubMed searches for systematic reviews (SR), randomised trials (RCT), economic evaluations and studies of patient preferences and values. Citations are being independently screened then allocated to topic areas for further assessment. We will then 1) compare the accuracy and efficiency of the manual search with a machine learning classifiers for RCTs, SRs and economic evaluations, and 2) assess the reliability of relying on PubMed alone by comparing retrieval from Epistemonikos, Cochrane CENTRAL and DORIS (Database Of Research In Stroke).

Results: Screening of over 70% of ∼11,700 abstracts has been completed and over 500 records deemed relevant. These records are now being triaged for potential inclusion. The PubMed auto-alerts retrieve between 350–500 citations per month, of which 20–30% are duplicates (retrieved again following the addition of MeSH terms). Data on the accuracy and efficiency of machine classifiers, the reliability of restricting searches to PubMed will be available by October 2019.

Conclusion: As living approaches gain momentum, and machine learning tools become more routinely available, we need to continually evaluate the impact different approaches to evidence surveillance have on accuracy, efficiency, reliability and feasibility.

Developing an innovative model for meaningful consumer engagement in Australian Living Stroke Guidelines.

Peta Bates¹, Kelvin Hill¹, Jennifer Muller¹, Tari Turner², Toni Aslett¹

¹Stroke Foundation Australia,²Cochrane Australia

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Background: The Stroke Foundation has a history of strong consumer engagement generally and within guideline development, but we wanted to strengthen this further as we test a model of living guidelines.

Methods/Process: A review of historical consumer involvement was undertaken with the Chair of the Stroke Foundation Consumer Council. Input was sought from the full Consumer Council and other experts in consumer involvement, including Kidney Health Australia and Cochrane. A ‘Consumer panel’ model was proposed. In brief the model involves a panel of 20–30 consumers with different lived experiences, each providing a brief overview of their experience and areas of interest which helps guide the selection of 3–5 members when a specific guideline topic is deemed to need updating. The consumers will assist to develop the patient values and preferences summary in the GRADE profile and then be actively involved in decision-making on changes to the recommendation and any related practical considerations. In the future, we also intend to trial other ways (e.g. surveys) to validate published values and preferences with a wider range of consumers.

Results/Lessons Learnt: 66 consumers responded to the call for expressions of interest and 37 (28 stroke survivors, 9 carers) have joined the panel for up to two years. Five topics have been identified as needing updating. Further results will be available at the conference.

Conclusion: Living guidelines approaches can include innovative models for consumer input, ensuring consumers are at the center of stroke guidelines.

Prioritising knowledge translation for living stroke guidelines

Chris Lassig¹, Peta Bates¹, Kelvin Hill¹, Tari Turner², Sally Green²

¹Stroke Foundation,²Cochrane Australia

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To determine how to focus knowledge translation (KT) activities to ensure effective implementation of living clinical guidelines for stroke management.

Background: The Stroke Foundation and Cochrane Australia are testing a ‘living’ model of rapidly updating clinical guidelines in response to new evidence. It is not clear how living recommendations will impact the KT strategies needed to encourage their use in clinical practice.

Process: Interviews with three stroke-related KT experts were used to design a survey of 385 health professionals on their attitudes to adopting clinical guidelines. This was followed by a one-day workshop with 12 experts in stroke-related implementation research.

Lessons Learnt: The expert group agreed that, in terms of KT, the living approach is essentially a more rapid version of the conventional clinical guidelines review cycle, and is unlikely to introduce unique challenges. This was supported by survey findings of increased trust in living guidelines, but similar barriers and enablers. However, limited resources and the long lead time to design and enact KT strategies makes it necessary to focus on a subset of recommendations, prioritised according to strength of evidence, gaps in practice, importance to clinicians and consumers, feasibility and strategic impact, and measurability of the changes.

Conclusion: By linking KT priorities to recommendations with strong evidence there are unlikely to be frequent changes to those that are considered high priority for KT. Nevertheless, this and other factors will be reviewed annually to ensure a living approach to prioritisation and KT.

Experience of using real-world antimicrobial prescribing data in NICE guideline surveillance

Emma McFarlane¹, Charlotte Rawstrone¹, Kay Nolan¹, Fiona Glen¹

¹National Institute For Health And Care Excellence

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To describe the use of antimicrobial prescribing data in guideline surveillance.

Background: NICE antimicrobial prescribing guidelines (APGs) outline prescribing strategies for common infections. The NICE surveillance process usually includes searches for evidence to contradict or reinforce guideline recommendations. A new surveillance approach was developed for the APGs which focused only on the assessment of antimicrobial prescribing data and committee input to determine if changes in the recommendations are needed.

Method: We did a descriptive analysis of the main characteristics of the APGs and compared the recommendations included in the guidelines with the antibiotic resistance data available in the English Surveillance Programme for Antimicrobial Utilisation and Resistance 2018 report (ESPAUR). We identified changes relating to increase of antibiotic resistance and drugs recommended to determine the need to update the APGs.

The results were discussed with a committee including GPs, pharmacists and microbiologists to confirm the surveillance decision.

Results: 10 published APGs covering respiratory and urinary tract infections were included. 29 recommendations were identified. We did not identify data from the 2018 ESPAUR report that would have an impact on recommendations in terms of increase of antibiotic resistance and drugs recommended. The committee confirmed these results.

This approach highlights an alternative surveillance strategy focused on antimicrobial prescribing data to inform currency of recommendations rather than changes to the evidence base, providing a rapid assessment of whether guidelines are current.

Conclusion: The use of antimicrobial prescribing data could be enough to inform surveillance decisions.

Using research collaborations to fast-track guideline surveillance and updates

Emma McFarlane¹, Kay Nolan¹, Philip Alderson¹, Sarah Moss², Teresa Jones², Dorota Goble²

¹National Institute For Health And Care Excellence,²NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC)

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop a quicker process for identifying new evidence that may impact on guideline recommendations.

Background: Some topic areas are fast moving increasing the risk of guidelines having outdated recommendations. Rapid access to information, for example, results of ongoing studies, and assessment of their impact on guideline recommendations might be a relevant surveillance strategy.

Methods: NICE developed an event tracker containing ongoing studies relevant to guidelines. Ongoing studies are identified through collaboration with the National Institute for Health Research (NIHR), a UK research funder:

• NICE regularly receives a list of funded studies from NIHR that are near completion
• Studies are matched to guidelines and logged
• Once completed, NICE liaise with NIHR to receive a confidential copy of the draft publication to consider impact on recommendations
• Where an impact is anticipated, an exceptional surveillance review is undertaken which may lead to a decision to update the guideline.

Results: From 2016 to 2019, 41 NIHR-funded studies (primary studies and systematic reviews) on the event tracker were checked for impact resulting in 11 exceptional surveillance reviews and 8 decisions to update.

For NICE guideline NG36, an NIHR study available in April 2017, was included in an exceptional surveillance review and the updated guideline recommendations were issued in June 2018 illustrating the benefit of this approach. Overall, developing pragmatic strategies to incorporate new evidence into surveillance decisions and updated guidelines improves the currency of recommendations.

Conclusion: Collaboration with research funders can facilitate including practice changing evidence into guidelines quicker.

Improving expert engagement in guideline surveillance by developing an Expert Advisers Panel

Swapna Mistry¹, Emma McFarlane¹, Kay Nolan¹

¹National Institute For Health And Care Excellence

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To develop a single repository of topic experts across health and social care to support guideline surveillance.

Background: Topic expert involvement is a core principle of guideline surveillance to gain an understanding of how guidelines are being received in practice and any potential gaps in guidance. In 2015, the surveillance team were often missing certain specialisms where there was no expert contact and there was no system for keeping contacts up to date.

Method: An Expert Advisers Panel (EAP) was set up to recruit experts to support the surveillance reviews and for ad hoc advice. A two-strand approach to recruitment was taken that involved contacting previous committee members and an open competitive recruitment. Applications were assessed against core skills by an independent panel.

The panel members’ information is managed within a database including specialist areas and conflicts of interest. Currently, the database has nearly 1200 experts covering the entire guideline portfolio. A comparative analysis of expert engagement before and after establishing the EAP was used to determine improved engagement.

Results: Using the panel provides quick access to experts who are committed to supporting surveillance reviews. A comparative analysis of expert engagement for surveillance reviews before and after establishing the EAP demonstrated topic engagement increased from:

• average of 2 to 7 experts for clinical topics
• average of 2 to 6 for public health topics

Conclusion

Since establishing the panel, there has been a marked increase in engagement with experts for surveillance reviews.

Developing and applying a living guidelines approach to WHO maternal and perinatal health recommendations

Joshua Vogel¹, Femi Oladapo²

¹Burnet Institute,²World Health Organization

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To present findings from the first cycle of prioritization and updating of WHO's maternal and perinatal health recommendations, and its implications for future WHO guideline development.

Background: How should World Health Organization (WHO) most efficiently keep its global recommendations up to date? Over the past decade, WHO has issued over 400 maternal and perinatal health recommendations - the size of this portfolio is a major challenge to ensuring that all recommendations are up to date. A dynamic “living guidelines” approach was developed and applied for the first time to WHO's recommendations, allowing a more rapid response to new, important evidence in maternal and perinatal health.

Methods: WHO established an Executive Guideline Steering Group (GSG) that oversaw a prioritization and updating process across 10 guidelines. We conducted an online prioritization survey with international stakeholders and mapped new trials for supporting systematic reviews. The GSG met to agree on priorities for updating or creating new recommendations.

Results: We mapped eligible trials to 184 Cochrane reviews. The GSG reached consensus on updating of 36 high-priority questions. We established “living” guideline panels that updated individual recommendations on a rolling basis, with 25 new or updated recommendations published over a 20 month period.

Conclusion: Guideline developers are facing the challenge of ensuring that recommendations are up to date in the context of a rapidly growing evidence base. Through this “living guideline” approach, WHO can respond rapidly to important evidence without compromising the quality of its guideline development process.

A tale of two wikis: the user perspective of an online method for developing guideline-based quality indicators.

Louise Wiles^1,2,3, Peter Hibbert^1,2,3, Tamara Hooper³, Charlotte Molloy^1,2,3, Jacqueline H Stephens^4,5,6, William Runciman^1,2,3, Jeffrey Braithwaite¹

¹Australian Institute of Health Innovation, Macquarie University,²Australian Centre for Precision Health, University of South Australia,³South Australian Health and Medical Research Institute,⁴Aboriginal Health Equity, SAHMRI,⁵Adelaide Medical School, University of Adelaide,⁶School of Health Sciences, University of South Australia

CS2 - Living Evidence and Evidence Surveillance, Hall C - Plenary, October 31, 2019, 1:30 PM - 3:00 PM

Aim: To evaluate users’ perspectives of wiki-based processes for developing quality indicators (QIs).

Background/Importance: Guideline-based QIs are used to measure healthcare quality. Interpretation of QI data can have important consequences; therefore development processes need to be efficient, transparent, and engage a range of stakeholders to ensure QIs are current, and reflect what is reasonably expected of clinicians.

Methods/Process: QIs, extracted from guideline recommendations, were subjected to multi-round modified Delphi review by clinical experts. Two purpose-designed wikis, developed and deployed in 2015 (CareTrack Kids, NHMRC Partnership Grant APP1065898) and 2017 (STANDING Collaboration, NHMRC Program Grant APP1054146), were used to post draft QIs and capture experts’ feedback. User perspectives were sought via online survey.

Results/Lessons Learnt: Eighty-two experts undertook wiki reviews, and 51 (62%) completed user surveys. Two-thirds of respondents (33/51, 65%) reported the wiki-based methodology was suitable for developing QIs. Cited advantages included: seeking multiple perspectives (n = 15), efficiency of the process (n = 9), ability to view others’ feedback (n = 9), and flexibility/convenience to complete in own time (n = 6). Users described disadvantages as: lack of opportunity for discussion (n = 7), potential for personal beliefs to predominate over evidence (n = 5), and difficulty navigating through layers of information presented online (e.g. experts’ comments, n = 5).

Conclusion: Users reported that wiki-based methodologies are suitable for achieving consensus in QI development and convey several advantages over traditional approaches. Findings suggest that future wiki-based processes should be trialled as an adjunct to real time discussion forums, with technical refinements to enhance the user experience.

The European Breast Guidelines on screening and diagnosis of breast cancer: piloting of an updating strategy

Pablo Alonso

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To present the results of the piloting of the updating strategy for the European Breast Guidelines (EBGs).

Background: EBGs on screening and diagnosis are evidence-based guidelines developed within the European Commission Initiative on Breast Cancer aiming at ensuring quality of care across Europe. They are developed by a group (GDG) of experts and patients using GRADE.

An updating strategy was developed and piloted to keep providing up-to-date recommendations.

Methods: The updating strategy was piloted on a sample of seven healthcare questions (HQs). For each step, specific tools were identified and information about time for completion, understandability and easiness to rate has been collected using a 5-points Likert scale (1 very easy; 5 very difficult).

Results: Three HQs were judged as not needing update during the Prioritisation phase, the same happened to other two during the Surveillance phase. Two HQs completed the Updating phase and both were modified. Only few GDG members participated: from 11 to 5 out of 27 members depending on the phase. The UpPriority tool mean score for understandability was 2.65, and 2.75 for easiness to rate; mean completion time in minutes was 5.2. For the Surveillance Questionnaire, the mean scores for understandability and easiness to rate were both 1.78, and the mean completion time was 15.33.

Conclusion: The piloting served to highlight challenges and strengths of the updating strategy. The tools used were judged positively. Possible limitations: preselection of HQs; few GDG members participated. Future developments: identification of a sustainable EBGs’ life cycle (living-guidelines, two-year time-laps).

The Bermuda Triangle of guideline presentation: What are our readers missing?

Sonja Maria¹

¹Charles Sturt University

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To discuss how graphical design, language and text complexity can impact upon the end users ability to implement a guideline.

Background/Importance: Guideline development often focuses on the evidence base to make clinical decisions, with less emphasis upon the final layout or use of language and design for the end user.

Methods/Process: This was a study which used grounded theory methodology. Methods included semi structured interviews with participants as well as a discourse analysis on Paramedic guidelines.

Results/Lessons Learnt: In an effort to create fast and efficient-to-use guidelines, many shortcuts are taken within the presentation of the paramedic guideline. These guideline shortcuts may challenge paramedics to understand the expectations of care for patients and additionally they are created without acknowledging their heterogeneous education and training background.

Conclusion: Readability of a guideline should be emphasised during the guideline conception stage. Attention to final layout, language level and text complexity need to be suited to the intended audience.

The UpPriority Tool: A prioritisation tool for updating clinical questions within guidelines

Emma McFarlane¹, Hector Pardo-Hernandez^2,3, Andrea Juliana Sanabria^1,2, Monica Ballesteros², Carlos Canelo-Aybar^2,3, Ena Niño de Guzman², Katrina Penman¹, Margarita Posso², Marta Roqué i Figuls^2,3, Anna Selva⁴, Robin Vernooij², Pablo Alonso-Coello^2,3, Laura Martínez García²

¹National Institute for Health and Care Excellence (NICE),²Iberoamerican Cochrane Centre,³CIBER de Epidemiología y Salud Pública (CIBERESP),⁴Corporació Sanitària Parc Taulí

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To develop a pragmatic tool to prioritise clinical questions for updating within clinical guidelines (CG).

Background: Updating CGs is resource-intensive and time-consuming. Prioritising clinical questions in higher need of updating is a potential strategy to optimise this process.

Methods: Multi-step development process including: 1) establishing a Working Group; 2) generation of an initial version of the tool; 3) optimisation of the tool via a feasibility test, semi-structured interviews, a Delphi consensus survey, external reviewing, and pilot testing; and 4) agreement on the final tool version.

Results: A total of 83 participants contributed to the development process. The final version of the UpPriority Tool includes six items:

1. Impact of outdated recommendations on safety: evaluate whether potentially outdated recommendations entail safety concerns.
2. Availability of new relevant evidence: new, relevant evidence to the clinical question or recommendations.
3. Context relevance: relevance of contextual factors (e.g. disease burden, clinical practice variation, emerging care options).
4. Methodological applicability: review if the clinical question still addresses PICO components of interest.
5. Users’ interest: relevance (e.g. citations, downloads, media impact) for patients, health care providers, policy makers, and other stakeholders.
6. Impact on access to health care: evaluate whether recommendations have implications for access and coverage.

The tool includes guidance for rating these items, calculating and ranking priority scores, and reporting priority results and conclusions.

Conclusion: The UpPriority Tool could be useful for standardising prioritisation processes when updating CGs and for fostering a more efficient use of resources in the CG field.

Validation of five search filters for retrieval of clinical guidelines produced low precision

Barbara Mintzes

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To validate search filters for retrieval of CPGs in MEDLINE and Embase using the Ovid and PubMed interfaces.

Background: Given the difficulty in navigating information overload, clinical practice guidelines (CPGs) are important sources of evidence for clinical care. Search filters for databases such as MEDLINE may provide an efficient way to retrieve CPGs. To our knowledge, this is the first study to validate search filters for CPGs.

Methods: A search for filters for identifying CPGs was conducted in Google and InterTASC. To retrieve a random sample of CPGs to test sensitivity and precision of the filters, we used TRIP and Epistemonikos. The citations were screened independently by two researchers. The sensitivity and precision were calculated.

Results: Five search filters were retrieved: two from CADTH, two from the University of Texas, and one from the MD Anderson Cancer Library. A total of 109/478 CPGs were included which comprised the sample for testing sensitivity and precision. The sensitivity ranged from 87% to 98% for the five search filters with very low precision (<1%). The CADTH broad search filter had the highest sensitivity across all three databases.

Conclusion: Our analysis shows that it remains difficult to efficiently identify CPGs due to low precision of five search filters. We recommend searching guideline-specific resources as a more time efficient approach than searching bibliographic databases. Future research should focus on developing a search filter that balances high sensitivity with moderate precision when combined with a filter designed to find studies on a specific clinical condition.

Assessing the robustness of recommendations made in a guideline on specialist neonatal respiratory care in babies born preterm with threshold analysis

David Phillippo^1,2, Caitlin Daly^1,2, Sofia Dias^2,3

¹Population Health Sciences, Bristol Medical School, University of Bristol,²NICE Guidelines Technical Support Unit (TSU),³Centre for Reviews and Dissemination, University of York

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To pilot a recently published threshold analysis method during development of a National Institute for Health and Care Excellence (NICE) guideline, Specialist neonatal respiratory care in babies born preterm, to assess the robustness of recommendations informed by network meta-analyses (NMAs) of two outcomes.

Background: Concerns on the reliability of NMA results may arise if included studies are flawed in their conduct or reporting. Threshold analysis allows decision makers to assess the robustness of recommendations based on the NMA to potential biases in the included evidence.

Methods/Process: After the guideline committee discussed the evidence including any concerns regarding bias and drafted recommendations, there was concern of bias arising from treatment switching in all studies due to lack of blinding. Threshold analysis was therefore carried out for the best- and worst-ranked treatments, to assess the robustness of positive and do-not-do recommendations, respectively.

Results/Lessons Learnt: The recommendation to use the best-ranked treatment arising from the NMA on bronchopulmonary dysplasia was robust. However, for the mortality outcome the recommendation was sensitive to imprecision from a single study contributing to the NMA, and the committee considered this when re-evaluating the positive recommendation. The recommendation to not use the worst-ranked treatment, based on the results from the NMAs on both outcomes, was robust, providing some reassurance of this do-not-do recommendation.

Conclusion: Results of threshold analyses lead to further scrutiny of the evidence to which the recommendations are sensitive and may placate concerns of potential biases which would not affect the treatment recommendation.

Evidence-based Clinical Practice Guideline Development Processes of Korean Medicine

Hyo-Weon Suh¹, Jong Woo Kim¹

¹Department of Neuropsychiatry, College of Korean Medicine, Kyung Hee University

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To introduce the guideline development process for traditional Korean medicine (TKM).

Backgound: TKM is one of the subtypes of the East Asian traditional medicine (EATM). EATM including traditional Chinese and Japanese medicine is classified as an alternative medicine. Although TKM is commonly used in Korea, evidence is limited and variation in treatment is present. Therefore, government and patients recently requested the evidence-based clinical practice guideline (CPG) of TKM. Thus, the Guideline center for Korean Medicine (G-KoM) was established as an organization under the Ministry of Health and Welfare in 2016.

Methods: G-KoM selected thirty diseases to develop CPGs, and published a manual for developing CPG for TKM. The manual development committee was composed of methodologists and clinicians. In compliance with the manual, academic societies of TKM develop the CPGs.

Results: The guideline development process is as follows. CPG is developed based on systematic reviews. Searching databases include COre database (DB) and specialized DBs for TKM. The quality of evidence is basically same as GRADE approach, but G-KoM added level of Classical Text-based (CTB). This level means that the intervention only have classical literature references. The level of recommendation is based on USPSTF guideline (i.e. level A∼D), but level of recommendation can be upgraded considering applicability and cost. If developers recommend the interventions with CTB level, the level of recommendation should be Good Practice Point (GPP) used in SIGN guideline. In December 2019, nine of thirty CPGs will be published.

Conclusion: The guideline development process of TKM is well-established.

Towards efficiency in updating guidelines: results of the evaluation of a pilot (CATS-AIOS) project involving doctors in training.

Laura Viester¹

¹Kennisinstituut Van De Federatie Medisch Specialisten

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: Our aim is to share the evaluation of and lessons learned from a pilot project involving doctors in training (AIOS) in modular development and updating of health care guidelines by means of CATS (Critical Appraisal of a Topic).

Background/Importance: To keep clinical practice guidelines up-to-date there is a need for efficiency in the process of guideline development. In the Netherlands, CATS are part of most medical education programs. The results of a CAT (PICO, search, selection, and evaluation of evidence) are an important part of the process of guideline development. However, at present CAT's have not been incorporated in guideline development.

Methods/Process: This project was evaluated by means of a quality evaluation of the content of the guideline modules and a process evaluation. A pre-evaluation was carried out by the project team and additional input was collected from AIOS (n = 20).

Results/Lessons Learnt: The results of the process evaluation are presented in a blueprint and include the following topics:

• Feasibility of using CATS
• Quality and quality assessment
• Time investment
• Specific factors (e.g. support, training) for successful involvement.

Conclusion: The main conclusion of this pilot project is that the workload for AIOS involved in the process of guideline development is high. It is important to meet certain conditions, formulated as recommendations in the form of a blueprint, to ensure successful involvement of doctors in training in guideline development. This blueprint can be used as a guideline and practical tool in other medical disciplines when involving doctors in training.

The roles of WHO guidelines: a qualitative study

Zhicheng Wang¹, Quinn Grundy², Lisa Parker¹, Lisa Bero¹

¹Charles Perkins Centre, The University of Sydney School of Pharmacy, Faculty of Medicine and Health,²Faculty of Nursing, University of Toronto

CS3 - Guideline Development Advances and Processes, Riverbank Room 3, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To describe the roles of WHO guidelines in influencing health policy.

Background: Properly implemented evidence based clinical and public health guidelines can improve patient outcomes. The World Health Organisation (WHO) has been a major contributor publishing more than 160 guidelines on various topics since 2008.

Method: Seventeen interviews were carried out between March 2018 and December 2018 with WHO guideline developers, Headquarter staff, regional and country office staff. The deidentified transcripts were analysed through three phases of coding on NVIVO. Situational analysis was used to situate the issues of the adaptation process into the wider context.

Results: WHO guidelines were discovered to play a variety of roles in the work of all levels of WHO. The perception of the guidelines as lists of recommendations that need to be adapted and implemented in different contexts is limited in scope. This project has explored the intricate ways in which the guidelines are intertwined in WHO's work at every level. We categorised the uses of guidelines as: 1) Directly change policy, 2) Justify policy change, 3) Stakeholder engagement, 4) Guarantors of legitimacy, 5) Advocacy tools, and 6) Intertwined with WHO's various roles.

Conclusion: Seen as a guarantor of legitimacy by many countries in the world, WHO guidelines can be better positioned to influence health policy and practice change if their various roles are better understood. Packaging future WHO guidelines with operationalising guidance and multiple versions for the variety of audiences would likely help the guidelines fulfil their roles more effectively.

Adaptation of the British Medical Journal's (BMJ) Rapid Recommendation guideline “Oxygen therapy for acutely ill medical patients”: description of the development of an adaption process.

Anja Ussing^1,2, Henning Keinke Andersen¹, Maria Herlev Ahrenfeldt¹, Simon Tarp¹

¹Danish Health Authority,²The Parker Institute, Copenhagen University Hospital, Bispebjerg and Frederiksberg

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To develop a concept for adaption of Rapid Recommendation guidelines into a Danish context.

Background: The Danish Health Authorities have a desire to adapt high quality international guidelines into a Danish context to achieve new clinical guidelines in an accelerated time course without compromising transparency, systematics and quality. The BMJ has developed the concept “Rapid Recommendations” where new potential practice changing evidence rapidly is transformed into clinical recommendations using the GRADE approach. We selected these guidelines for adaption since their methods fulfilled our demands for quality and transparency.

Methods: In a pilot project, the recommendation for oxygen therapy¹ will be adapted to a Danish context. A working group representing relevant medical specialties and methods specialists has been set up. The group will develop recommendations adapted to a Danish context using the GRADE-method. Based upon the “body of evidence” identified by the Rapid Recommendation team, the working group will rate the quality of evidence, evaluate the importance of outcomes and finally formulate clinical recommendations adapted to a Danish context. Before publication the guideline will undergo external peer review and public hearing. Based on feedback from the working group, we will adjust the adaption process and select new Rapid Recommendations for adaption.

Expected results: We will evaluate the adaption process including a feedback process from patients, professionals and stakeholders during autumn 2019.

Conclusion: Our results will show whether the adaption concept is feasible. Based on this, we will make a final decision about continuation of the concept.

1. BMJ 2018;363:k4169.

Diagnosing and managing work-related mental health conditions in general practice – new Australian Clinical Practice Guidelines

Danielle Mazza¹, Samantha Chakraborty¹, Bianca Brijnath^1,2, Heather Nowak³, Trevor Brott⁴, Justin Kenardy⁵, Michelle Atchison⁶, David Gras⁷, Cate Howell⁹, Richard Buchanan⁸, Seyram Tawia¹⁰

¹Monash University, Department Of General Practice,²National Ageing Research Institute,³Mental Health Australia,⁴Western Industrial Screening and Accident Clinic,⁵Australian Psychological Society,⁶Royal Australian and New Zealand College of Psychiatrists,⁷Royal Australasian College of Physicians, Australasian Faculty of Occupational and Environmental Medicine,⁸Office of Industrial Relations - Queensland,⁹Royal Australian College of General Practitioners,¹⁰Comcare

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To develop a clinical guideline to assist general practitioners (GPs) with diagnosing and managing patients with work-related mental health conditions (MHCs).

Background: GPs are almost always involved in the care of patients with work-related MHCs. However, many GPs face challenges in providing this care, and have requested advice to assist them in addressing these challenges.

Method: The guideline was developed according to National Health and Medical Research Council standards and involved: (1)constructing key clinical questions from clinical dilemmas faced in practice; (2)undertaking systematic literature reviews for clinical questions; (3)assessing the body of evidence according to GRADE criteria; (4)developing a draft guideline; (5)revising the guideline following public consultation; and (6)assessment by independent methodologists and content experts.

Results: The guideline provides recommendations on work-related MHCs in general practice, specifically regarding:

• tools to assist diagnosis and assessment of severity of a MHC
• assessing if a patient is developing a comorbid or secondary MHC
• assessing if the MHC has arisen from work
• conveying a diagnosis of a MHC to the patient
• management of the condition to improve personal recovery
• considering if the patient can work
• communicating with the patient's workplace
• managing MHC and comorbid substance misuse
• considering why a MHC is not improving
• managing a MHC that is not improving

Conclusion: This guideline provides GPs with the best-available evidence to assist in diagnosing and managing patients with work-related MHCs. Guideline implementation will be facilitated through tailored implementation activities.

Updating of the guidelines for the disinvestment of obsolete technologies

Mariana Michel Barbosa¹, Jessica Yumi Matuoka¹, Jorgiany Ebeidala Emerick², Haliton Alves de Oliveira Junior¹

¹Hospital Alemão Oswaldo Cruz,²Brazilian Ministry of Health

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To verify the cases of disinvestment of technologies in health during guidelines updating process.

Background: By law, Brazilian clinical guidelines must be updated every two years ¹. The updating process of these guidelines allows the evaluation of new technologies that were not available at the time of the guideline's elaboration and the analysis of the disinvestment of obsolete medicines.

Methods: We conducted a search on Brazilian in National Committee for Health Technology Incorporation (CONITEC) website to identify and describe cases of disinvestment of technologies, which were structured during the guidelines updating process.

Results: Only one published disinvestment recommendation was found, the case of disodium pamidronate for Paget's disease (osteitis deformans). This recommendation was given because the evidences showed that the treatment with zoledronic acid was more effective, sustained and its administration more convenient when compared with the disodium pamidronate.

Conclusion: The continuous updating of the guidelines is an important tool for the disinvestment of obsolete technologies, ensuring the financial sustainability of the healthcare system but providing the best therapeutic options for the patients. However, in Brazil the disinvestment is still a new practice. Our analysis showed that more effort should be made to improve the efficiency in health technologies management and disinvestment should be considered when applicable.

References:

Brasil. Ministério da Saúde. Gabinete do Ministro. PORTARIA NO 2.009, DE 13 DE SETEMBRO DE 2012, aprova o regimento interno da Comissão Nacional de Incorporação de Tecnologias no Sistema Único de Saúde (CONITEC).

Addressing HIV-Related Stigma and Discrimination in the Healthcare Settings: Development and contextualization of guideline recommendations

Garumma Tolu Feyissa^1,2, Craig Lockwood², Mirkuzie Woldie^1,3, Zachary Munn²

¹Jimma University,²The Joanna Briggs Institute,³Department of Global Health and Population, T.H. Chan Harvard School of Public Health

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: This project was aimed to develop guideline recommendations to reduce HIV-related SAD in the Ethiopian healthcare settings.

Background: Stigma and discrimination (SAD) against people living with HIV in the healthcare setting is a common encounter.

Methods: A multidisciplinary panel of 13 experts was consulted through informal and formal meetings to determine the scope and content of the guideline. A systematic review of guidelines, tools, best practice documents and standards of practice and systematic reviews was conducted. This was followed by a systematic review of quantitative evidence. A content analysis of the included documents was conducted to develop initial tentative guideline recommendations. Strength and quality of evidence were assigned for each recommendation using a software package developed by Grading of Recommendations, Assessment, Development and Evaluation (GRADE) working group. The tentative recommendations were evaluated through a series of Delphi surveys. This was accompanied by an evaluation by an external panel using Guideline Implementability Appraisal (GLIA V2.0) checklist. Finally, we explored facilitators and barriers to implement the guideline using key informants interviews.

Results: A list of 31 tentative recommendations was drafted through a content analysis of the included documents. The recommendations were framed under the following themes: structural, information-based and skills building, contact and empowerment and biomedical interventions. Based on exploration of barriers and facilitators, we developed an implementation framework for the guideline.

Conclusion: The tentative recommendations were evaluated both by internal and external expert panel. Detailed information was sought to contextualize the guideline to the local health system of Ethiopia.

Differences in transparency of methods used to develop national dietary guidelines and in recommendations for fat intake: A pilot study comparing ten countries

Cynthia M. Kroeger¹, Zhaoli Dai¹, Sally McDonald¹, Lisa A. Bero¹

¹The University of Sydney

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To describe differences in fat intake recommendations and the rigor of development methods among national food-based dietary guidelines for Australia, Bangladesh, Brazil, Canada, Germany, India, Ireland, South Africa, Sri Lanka, and the US.

Background: Dietary guidelines are developed globally to help decrease the health and economic burden of non-communicable disease. However, recommendations differ among countries for reasons that remain unknown.

Methods: Recommendations for the intake of total, saturated, monounsaturated, n-6 and n-3 polyunsaturated, and trans fats, and cholesterol, as well as for dairy, meat, oils, and fat-containing processed foods and other plant-based foods, were extracted from each guideline. Methods for developing recommendations were extracted and compared to the World Health Organization (WHO) Handbook for Guideline Development. We assessed: discipline and diversity representation of the guideline development group, disclosure and management of competing interests, and evaluation of the scientific evidence base and formulation of recommendations.

Results/Lessons Learnt: Variable recommendations were developed by all countries for total and saturated fat, meat, and processed foods and some countries for unsaturated (n = 8) and trans fats (n = 8), cholesterol (n = 5), dairy (n = 9), oils (n = 9), and other (n = 6). Most guidelines described methods for discipline representation of the development group (n = 7) and linking recommendations to evidence (n = 8). Few countries disclosed (n = 2) or managed (n = 1) conflicts of interest, or systematically evaluated evidence (n = 3) or the strength and certainty of recommendations (n = 3).

Conclusion: All ten guidelines would benefit by improvements in methods reporting. Differences in recommendations may be explainable by differences in development methods and substantive evidence.

Inconsistency of Recommendations for Evaluation and Management of Hypertension

Peter Oettgen^1,2, Brian S Alper¹, Amy Price³, Esther van Zuuren⁶, Zbys Fedorowicz⁷, Glyn Elwyn⁴, Allen Shaughnessy⁸, Amir Qaseem⁵, Ilkka Kunnamo⁹, Urvi Gupta¹¹, Deborah D Carter⁹, Michael Mittleman¹⁰, Martin Mayer¹

¹EBSCO Health,²Cardiology Division BIDMC,³Department of Anesthesiology, Stanford School of Medicine,⁴The Dartmouth Institute for Health Policy and Clinical Practice,⁵American College of Physicians,⁶Department of Dermatology, Leiden University Medical Centre,⁷Veritas Health Sciences Consultancy Ltd,⁸Tufts University School of Medicine,⁹Murdy Consulting Group,¹⁰The American Living Organ Donor Fund,¹¹Stanford School of Medicine

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Objective:

Aim: To systematically assess the consistency of recommendations regarding hypertension management across clinical practice guidelines (CPGs) and electronic point-of-care (POC) resources

Method: We identified hypertension management recommendations from eight CPGs and two POC resources. We described discrete and unambiguous specifications of the population, intervention, and comparison states to define a series of reference recommendations. Three raters reached consensus on coding the direction and strength of each related recommendation made by each CPG and POC resource. For each recommendation, we analyzed the rate of consistency for direction and strength. We also assessed the CPG and POC resources for evidence of public and patient involvement, patient-facing information, and shared decision-making tools, and we involved patient and public representatives in this assessment.

Results: Considering all 10 recommendation sources, 12 of 71 recommendations (16.9%) were consistent in direction and strength, 21 (29.6%) consistent in direction but inconsistent in strength, and 38 (53.5%) inconsistent in direction. Considering only the CPGs, 25 recommendations (35.2%) were consistent in direction and strength, 13 (18.3%) consistent in direction but inconsistent in strength, and 33 (46.5%) inconsistent in direction. Excluding “insufficient evidence” ratings did not explain the inconsistency, and a leave-one-out sensitivity analysis suggested the inconsistency is not due to any single recommendation source.

Conclusion: Hypertension is a common chronic condition with widespread expectations surrounding guideline-based care, but CPGs have high degrees of inconsistency. Further investigation should determine the reasons for inconsistency, the implications for recommendation development, and the role of synthesis across recommendations for optimal guidance of clinical care.

Development of an evidence-based clinical practice guideline for deprescribing cholinesterase inhibitors and memantine

Emily Reeve^1,2,3, on behalf of the Guideline Development Team

¹Quality Use of Medicines and Pharmacy Research Centre, School of Pharmacy and Medical Sciences, University of South Australia,²NHMRC Cognitive Decline Partnership Centre, Kolling Institute of Medical Research, Northern Clinical School, Faculty of Medicine and Health, University of Sydney,³Geriatric Medicine Research, College of Pharmacy and Faculty of Medicine, Dalhousie University and Nova Scotia Health Authority

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To develop a guideline to assist healthcare professionals determine when to trial withdrawal of cholinesterase inhibitors (ChEIs) and memantine.

Background/Importance: Optimal medication use in people with dementia involves both prescribing medications and deprescribing (supervised withdrawal) of medications for which risk outweighs benefit. There has previously been a lack of evidence-based drug-specific deprescribing guidelines which likely acts a significant barrier to withdrawal of potentially harmful and unnecessary medications in clinical practice.

Methods/Process: The Guideline Development Team (GDT) consisted of content and methodology experts, clinicians, and consumers. We collaborated with, and followed the process developed by the Bruyère Research Institute Deprescribing Guidelines in the Elderly project, which is based on a comprehensive checklist for successful guideline development, the AGREE-II criteria and GRADE (Grading of Recommendations Assessment, Development and Evaluation).

Results/Lessons Learnt: Four recommendations were developed to guide clinicians in identifying individuals for trial deprescribing of ChEIs and memantine (such as those who don’t have an appropriate indication, have never experienced a benefit, appear to be no longer benefiting, or have severe/end stage dementia). Three additional practice points provide tapering and monitoring advice and other situations in which trial deprescribing could be considered. While there were limitations to the available evidence, an evidence-based deprescribing guideline was able to be developed and its recommendations were approved by the Australian National Health and Medical Research Council.

Conclusion: Implementation of this guideline may lead to enhanced deprescribing in practice and in turn, improved quality of life in people with dementia.

When safety is a concern: A rapid update of the Kaiser Permanente National aspirin guideline

Deborah Regidor¹, John Dunn^1,2, Qiana Amos¹

¹Kaiser Permanente Care Management Institute,²Kaiser Permanente Washington

CS3 - Guideline Exemplars, Riverbank Room 2, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To provide a case example of a rapid guideline update

Background: Kaiser Permanente (KP) National Guidelines are updated based on prioritization, with a typical timeline of 9 to 12 months. Recent publications reported excess bleeding with aspirin use in older adults. Due to potential safety issues, the guideline development team (GDT) determined an expedited updating process was needed.

Method: Interim recommendations via limited peer-review were published within one week until the GDT could fully vet new evidence.

Results: Major changes included a more conservative approach to aspirin therapy: A shift in the ages of target population and changes in recommendation strength. The GDT's considerations for justifying interim recommendations included:

• Safety issue potentially affecting large population
• Consistent results from multiple large studies
• Publication of a widely used professional society guideline that adheres to rigorous standards of methodological quality
• Providers communicating a need for rapid guidance

Conclusion: Aspirin use conveys small net benefit in younger adults at high cardiovascular risk, and potential net harm to patients ages >70 years. Issuing interim recommendations when warranted can potentially reduce the risk of serious bleeding from aspirin use in older people. Based on the aspirin experience, we are now developing methodological standards for identifying and creating recommendations requiring an expedited update. Guideline developers may want to develop a methodology to aid in identifying recommendations that are candidates for expedited updates. Criteria to consider may include safety concerns, publications of high-impact studies and/or guidelines, and a need for rapid guidance.

Setting Evidence on FHIR: Achieving Interoperability to Make Evidence-Based Practice Computable

Brian Alper^1,2, Ilkka Kunnamo³, Linn Brandt⁴, Zachary Munn^5,6

¹EBSCO Health,²University of Missouri-Coumbia,³Duodecim Medical Publication Ltd,⁴MAGIC,⁵Joanna Briggs Institute,⁶University of Adelaide

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim and Background: GINtech seeks to share technology solutions to improve efficiency of guideline development and implementation. Efforts to implement evidence-based practice are substantially limited because research findings and appraisals are not findable and re-usable without labor-intensive manual screening and repeated data entry. Interoperability can be achieved by establishing standards to communicate and exchange evidence concepts in machine-interpretable formats. Computable formats will provide an architecture to realize the Evidence Ecosystem.

Method: Health Level 7 (HL7) develops standards for electronic health information exchange. We are using and extending the HL7 Fast Healthcare Interoperability Resources (FHIR) standards to include components for evidence based medicine. The FHIR Resources for Evidence-Based Medicine Knowledge Assets project (EBMonFHIR) is an open, collaborative effort with working meetings twice weekly (https://confluence.hl7.org/display/CDS/EBMonFHIR).

Results: The technical and functional model for machine-interpretable expression of evidence is evolving. To make this usable for all components in the Evidence Ecosystem, we have included engagement from researchers, guideline creators and implementers. The current model includes descriptions of the research question, the statistical evidence, and its certainty (for example, GRADE ratings).

Discussion: Working together we can achieve interoperability for the Evidence Ecosystem in the computable era to realize the technological breakthroughs we see in other domains such as navigation support and finance. Common frameworks to represent evidence and its certainty will facilitate understanding, sharing and dissemination of EBM. Achieving interoperability with healthcare services will also enable easier dissemination into electronic health records (EHRs) and other clinical systems.

Finding Dory – fishing in the sea of NICE recommendations

Fiona Glen¹, Kay Nolan¹

¹National Institute for Health and Care Excellence

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To share our findings from pilot data analytic work to allow us to search related recommendations across NICE products.

Background/Importance: NICE supports the health and care system on a vast array of issues. With a portfolio of over 300 published guidelines NICE employs a detailed surveillance process that checks whether each guideline needs updating. Like Dory searching for her parents, finding related recommendations across our sea of guidance can be hard. Knowing related recommendations across our NICE portfolio supports efficient use of resources and is fundamental for the design of new evidence management platforms.

Methods/Process: NICE partnered with the Office for National Statistics. Working together, we designed the machine to be able to web-scrape content from our website and to use Natural Language Processing to classify and automatically link recommendations. Using different statistical techniques, we modelled the accuracy of the tool against human decision making when testing the accuracy of recommendation matching.

Results/Lessons Learnt: We will present findings on the accuracy and effectiveness of the tool plus any challenges we faced during design.

Conclusion: While increasing speed and accuracy of our surveillance process the tool can also be adapted to support automated comparison between guideline developers. As Dory needed Nemo and Marlin, the project was successful because of a multi-disciplined team relying on one another's different skills sets.

Making guidelines more valuable: exploiting a broader range of data sources and applying advanced analytics

Adrian Jonas¹

¹UK National Institute for Health and Care Excellence

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To update on the progress of the innovative programme to use broader data and applied analytics in guidance development.

Background/Importance: Guidance developers should seek to embrace the technological advances in the fourth industrial revolution and undertake analysis on a broader variety of different types of data to develop evidence to inform their guidelines; and to ensure these fit within a Learning-Health-System and thus potentially even offer guidance to a level of personalised care (based on the characteristics of a patient).

Methods/Process: A statement of intent was developed through internal collaboration and consultation, discussion with key external partners, and then an external consultation wider stakeholders, including the public.

Results/Lessons Learnt: The UK National Institute for Health and Care Excellence will soon be publishing a finalised statement of intent that explores:

• What kind of evidence does NICE currently use to develop guidance
• What broader types of data are available
• When and why should broader types of data be considered
• Practical considerations associated with data analytics.

Technical detail on methodological considerations are not included in the statement and will be developed at a later stage. A new (central) data and analytics team now also provides central oversight, advice and support to create the right ecosystem for implementation across 5 key strategic areas– data, tools, skills, collaboration and public trust.

Conclusion: Guideline developers should identify activities that will benefit from exploiting a broader range of data sources and applying advanced analytics.

How to Combine Artificial Intelligence with Clinical Practice Guideline: A Scoping Review.

Xufei Luo^1,2, Yujie Xiao³, Xiao Liu¹, Kehu Yang¹, Yaolong Chen¹

¹Evidence-Based Medicine Center of Lanzhou University; WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,²School of Public Health of Lanzhou University,³The Second Medical School of Lanzhou University

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To review how AI is now used to develop CPGs.

Background/Importance: With the development of Artificial Intelligence (AI) in medicine, the medicine journal has published many papers on AI and Clinical Practice Guidelines (CPGs). However, their current situation is unclear.

Methods/Process: MEDLINE and EMBASE were searched. Screening and data extraction were done in duplicate. Study characteristics and details about AI use were analyzed.

Results/Lessons Learnt: A total of 1354 published articles were identified, 98 met the inclusion criteria, about two fifths (38.8%) focused on the topic of AI tools or approaches (such as GLARE), 20 (20.4%) were computer algorithms on guideline developing, 16 (16.3%) introduced the clinical decision support systems in CPGs. 13 (13.3%) were machine learning and data mining on guidelines, 10 (10.2%) were integrating CPGs with medical records and 6(6.1%) were other sources about guideline developing using AI. In terms of study design, most of studies (76.3%) were literature review and experts’ opinions, only 13% studies were primary studies. Most of the studies were still exploring the method and framework of AI using CPGs.

Conclusion: While the use of AI to develop CPGs appears to be increasing, this review found that most of those studies were based on theory, and still can’t be used in practice. We put forward three directions for guideline developers involving AI. 1) To short the search times using AI when conducted SRs; 2) To develop the algorithms of quality evaluation for guideline; 3) To present the mapping of recommendations using AI.

A new digital and multilayered format of HTA reports for policy-makers, linked to systematic reviews and guidelines

Per Olav Vandvik¹, Linn Brandt², Stijn van de Velde³, Øyvind Melien³, Thomas Agoritsas⁴

¹Magic Evidence Ecosystem Foundation,²Department of Medicine, Innlandet Hospital Trust,³Norwegian Institute for Public Health,⁴University hospitals of Geneva

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To develop a prototype for a digital and multilayered HTA report presentation format, reusing and integrating content from systematic reviews and guidelines.

Background/Importance: Policymakers face major challenges in making timely and well- informed decisions for new technologies. Health Technology Assessments (HTA) reports represent key decision support tools but are typically inaccessible, lengthy and difficult to understand. Optimised presentation formats are therefore needed, ideally making use of advances made for guidelines, also to allow better integration of HTA with guidelines in the evidence ecosystem.

Methods: We developed the digital and multilayered format building on previous research and innovations for guidelines and aiming for a sufficiently generic and flexible format to align with various HTA frameworks and criteria. We applied a stepwise and iterative process with brainstorming, sketching and design through stakeholder feedback and usability testing. To demonstrate real life feasibility, we coordinated the prototype development with an ongoing national HTA on a stroke device performed by the NIPH and informed by MAGICapp/BMJ Rapid Recommendations.

Results: The prototype will be demonstrated at the conference. It was published in Norway March 2019 and used by policymakers in a face-to-face meeting. They were able to make a real-life decision after 10 minutes of exploring and discussing the content online. The initial positive feedback was supported by more formal evaluation.

Conclusion: Authoring and publication of HTA reports in such formats could greatly facilitate co-production, dissemination and updating of systematic reviews, HTA and guidelines, and ultimately sharing evidence and work in the evidence ecosystem.

A head-to-head comparison of GRADEpro and MAGICapp for supporting the development and dissemination of clinical practice guidelines

Jenny Ring¹, Sue Campbell¹, Kris Harvey¹, Sarah Norris¹

¹Health Research Consulting (hereco)

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Aim: To directly compare the capability of two guideline development tools (GDTs), GRADEpro and MAGICapp, to support all aspects of developing and disseminating clinical practice guidelines.

Background/Importance: A number of technical support tools have been designed to support the development and publication of guidelines. Many focus on supporting evidence identification and appraisal, and publication of recommendations. However, guideline development also requires high-level project and stakeholder management, and the relative ability of GDTs to support these activities was unclear.

Methods/Process: A framework for comparing the functionalities of the two GDTs was developed using broad domains: general secretariat and governance activities, stakeholder management, data management, communication support, content development, and publication and dissemination. Each tool was rated on the extent to which it performed specific functionalities within these domains.

Results/Lessons Learnt: Both GDTs support the development and presentation of synthesised evidence (content development). GRADEpro provides more support for early processes, including general governance, communication support and stakeholder management (e.g. PICO development) while MAGICapp provides more support for later processes of publication/dissemination (digital publication with interrogatable evidence tables and integrated interactive decision tools). Constraints identified include original study data not being accessible in GRADEpro evidence tables, and MAGICapp being configured for intervention questions only.

Conclusion: GRADEPro supports PICO development and more question types, whereas MAGICapp provides a more interactive online platform for users that supports regular updates. Choice of GDT should be informed by the relative importance to the developer of flexibility in evidence appraisal (GRADEpro) or ease of online publication (MAGICapp).

The PANELVIEW instrument for assessing panelists’ satisfaction with and perceived appropriateness of the development of practice guidelines

Wojtek Wiercioch¹, Elie Akl², Nancy Santesso¹, Yuan Zhang¹, Rebecca Morgan¹, Juan José Yepes-Nuñez³, Sérgio Kowalski⁴, Tejan Baldeh¹, Reem Mustafa⁵, Kaja-Triin Laisaar⁶, Ulla Raid⁷, Itziar Etxeandia-Ikobaltzeta¹, Alonso Carrasco Labra¹, Matthew Ventresca¹, Ignacio Neumann⁸, Maicon Falavigna⁹, Romina Brignardello-Petersen¹, Gian Paolo Morgano¹, Jan Brozek¹, Meghan McConnell¹⁰, Holger Schünemann¹

¹McMaster University,²American University of Beirut,³Universidad de los Andes,⁴Federal University of Paraná,⁵University of Kansas Medical Center,⁶University of Tartu,⁷Ministry of Social Affairs,⁸Pontificia Universidad Católica de Chile,⁹Federal University of Rio Grande do Sul,¹⁰University of Ottawa

CS3 - The Digital Evidence Ecosystem, Hall C - Plenary, October 31, 2019, 3:30 PM - 5:02 PM

Background: Guideline recommendations may be impacted by flaws in the process, inappropriate panel member selection, conflicts of interest and other factors. As no validated measurement tools exist to evaluate the guideline development process from the panelists’ perspective, our aim was to develop an instrument for guideline members to assess the structure, process and validity of the recommendations they create.

Methods: We began the instrument development with item generation through panelist surveys, contacting guideline developers, and a systematic literature search. We assessed face validity and reduced items through structured interviews, surveys with panelists, and guideline expert review. We tested the PANELVIEW tool with eight guideline panels as a pilot to assess instrument reliability.

Results: For item generation we surveyed 62 members from 13 guideline panels, and reviewed 20 documents identified in the literature and by contacting key stakeholders. Fifty-one additional key informants provided feedback about the generated items and response option phrasing. Item reduction resulted in 34 final items for the PANELVIEW tool. In the pilot evaluation with eight guideline panels, which used similar development processes, we found an overall test reliability coefficient of 0.35 across panels, with internal consistency ranging from 0.85 to 0.98 for the participating groups.

Conclusion: The PANELVIEW tool focuses on insight into and transparency of the guideline development process. In future research we aim to evaluate the use of the tool with a larger, heterogenous sample of guideline panels. Guideline developers can use the tool to identify areas for quality improvement in their development process.

Evidence-based learning: design, evaluation and conduct to optimise student outcomes

Timothy Barker¹, Alexa McArthur¹, Cindy Stern¹, Kylie Porritt¹, Matthew Stephenson¹, Bianca Pilla¹, Zachary Munn¹

¹Joanna Briggs Institute, The University of Adelaide

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: This research project aimed to document the use of the CREATE framework to develop an Evidence-based medicine (EBM) learning assessment tool, to evaluate the efficacy of the comprehensive systematic review training program.

Background: EBM involves the use of the best available evidence to inform clinical-decision making. Teaching EBM to students requires complicated and varied curricula, and this complicated learning needs effective evaluation and assessment, ensuring that the outcomes of students are being met and improved. 5 competencies integral to EBM have been identified, in which students must demonstrate aptitude; Ask; Search; Appraise; Integrate; Evaluate. However, there exists little uniformity in the types of assessment utilised to evaluate student learning (performance assessment or cognitive testing) and the assessment category in which evaluators are interested in (improve students’ knowledge of EBM or their attitudes towards EBM?). These differences were noted in 2011 and led to the development of the CREATE framework to guide development of EBM learning assessment tools.

Process: A working group of researchers formed within the Joanna Briggs Institute designed an EBM learning assessment tool using CREATE. This tool was designed to evaluate the efficacy of an advanced program aimed at improving participant's knowledge of the systematic review process offered by the Joanna Briggs Institute.

Conclusion: We highlight the advantages and limitations of using such a framework, and provide examples of how EBM can be assessed in unique and varied settings using CREATE.

Evidence Based Medicine in China—An Oral History Study

Bo Li^1,2, Yaolong Chen^1,3, Mengshu Wang⁴, Mr. Xufei Luo^1,3, Mr. Dazhen Zhao², Xuan Yu^1,3, Meng Lv^1,3, Kehu Yang^1,3

¹Lanzhou University,²Beijing Hospital of Traditional Chinese Medicine,³WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,⁴The First Hospital of Lanzhou University

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Objective: This oral history study aims to faithfully record the development process of evidence based medicine in China, describe the general situation, important historical figures and events of evidence based medicine in China to domestic and international peers.

Background: After being introduced in around 1996, evidence based medicine in China has made great progress. Although some articles have described the development history of evidence based medicine in China, to our knowledge no study has yet intended to comprehensively record the complete history of EBM in China by interviewing the leading experts. Inspired by the international oral history research of evidence based medicine, we therefore intend to conduct a similar study in China.

Method: Ten experts who are representative of the development of evidence based medicine in China have been invited to conduct oral history research and interviewed by pre-determined questions. Interview questions were reviewed and revised by the following experts who participated in the international oral history study: Gordon Guyatt and Brian Haynes, McMaster University, Canada; Sir Iain Chalmers and Sir Muir Gray, Oxford University, UK; Paul Glasziou, Bond University, Australia; and Kay Dickersin, Johns Hopkins University, USA.

Results: Ten interview questions were asked for the leading EBM experts and their answers are analyzed and will be presented at GIN conference.

Conclusion: This study will provide important videos and documents data for China and international people to know the development of evidence based medicine in China.

Evidence Based Practice in Graduate Nursing Education

Robin Christian¹, Michelle Palokas¹

¹University of Ms Medical Center, School of Nursing

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To discuss the successful implementation of systematic review (SR) methodology and project implementation in the curriculum of graduate nursing education.

Background: Research supports that the implementation of evidence-based practice (EBP) leads to high quality and cost-effective care; however variability in EBP persists. EBP concepts need to be included in graduate nursing education to provide the competencies and skills required for knowledge generation and translation.

Importance: In a recent study from the United States, nurses reported that they were not competent in the 24 EBP competencies, but a moderate positive association was found between EBP competencies and EBP knowledge.

Methods/Process: The Joanna Briggs Institute's (JBI) comprehensive SR training is embedded in a required program course. Ideally, the findings of the systematic review inform the project development and translation. Throughout the student's course work, they are completing a systematic review and project implementation. To facilitate student understanding of the process, we created a visual model to depict how the SR and project implementation fits into the steps of EBP.

Results: Students have a hands-on application of evidence-based practice competencies to prepare them for real-world use and translation.

Lessons Learnt: Lessons learnt address the following areas: policies needed for exceptions, development of a review panel, and how to develop a question applicable to a project implementation topic.

Conclusion: Curriculum enhancements can be replicated in other programs to improve student education in EBP competencies and knowledge translation to improve patient outcomes ultimately.

Impact of iCAHE Journal Club

Janine Dizon¹, Steve Milanese¹, Jeanette Routley², Catherine Turnbull²

¹Janine Dizon - University of South Australia,²Allied, Scientific and Health Office, SA Health

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: This research aims to evaluate the impact of the iCAHE Journal Club (JC) in terms of:

1. achieving the clubs’ goals in running JCs
2. perspectives of the iCAHE Journal Club members in running JC meetings particularly regarding evidence implementation

Background: The International Centre for Allied Health Evidence (iCAHE) Journal Club (JC) has been developed 10 years ago to facilitate the use of evidence into practice and to address one of the main barriers which is a lack of time to search and appraise papers. The iCAHE JC process was refreshed by including local context assessment to assist clinicians to adopt or contextualise the evidence to facilitate the implementation of evidence into practice.

Method: This is an evaluation research looking at impact of the iCAHE JC as a medium for evidence implementation. iCAHE JCs were asked for (1) their goals in running their clubs and their best measure of achieving their goals via short feedback surveys and (2) their perspectives of running journal club meetings in implementing evidence into practice via interviews.

Preliminary results: Different JC groups have different goals in running their JCs. The common goals are (1) continuing professional development (PD), (2) opportunity to have updates in practice and (3) provide structure to Quality Improvement (QI) projects and (4) facilitate knowledge translation and evidence implementation.

Conclusion: The iCAHE JC is a valuable medium in assisting clinicians in achieving their JC goals relating to professional development and evidence implementation activities.

Evidence-Based Practice Collaboration between a School of Nursing and a Hospital in an Academic Medical Setting

Michelle Palokas

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To discuss the development of a collaborative relationship between the School of Nursing (SON) and the Office of Nursing Quality in an academic medical center setting. The goal of the collaboration is to provide guidance and resources for bedside nurses to implement evidenced-based practice (EBP) at the point of care.

Background: Nursing holds a vital role in collaborative decision making for quality improvement. In an effort to improve quality measures, a need was identified to educate the nursing staff on EBP concepts.

Importance: Research supports that EBP leads to high-quality, cost-effective care. Benefits of collaboration and improved patient outcomes will support efforts in the journey to high reliability. Bedside nurses require resources such as faculty experts, students, and EBP tools. A collaboration with the SON can provide the needed education and training.

Methods/Process: Focus groups were held and a strategic plan was developed. Establishment of The CLUB (Clinical Leaders Using Best Practice) transpired. EBP workshops were conducted by faculty members from the SON and a Joanna Briggs Institute Centre of Excellence.

Results: The workshops were instrumental in engaging collaboration between the SON and the hospital. Continued partnership will include regular CLUB meetings to support EBP and quality improvement projects in the hospital.

Lessons Learnt: Lessons will address the following areas: desire and need for EBP, required buy-in, advisory board.

Conclusion: A structured process and shared resources between a hospital and SON aids in culture change and helps facilitate the implementation of EBP at the point of care.

Guideline Implementation Enablers and Challenges for Students in Nursing Education Programs: A Gap Analysis Study.

Elaine Santa Mina¹, Sherry Espin¹, Donald Rose¹

¹Ryerson University

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: This gap analysis study investigated the knowledge, attitudes, and perceived enablers and challenges of guideline implementation in learning, teaching and practice for nursing students.

Background/Importance: Over the past decade, individual faculty in a school of nursing idiosyncratically implemented guidelines within several courses to meet perceived learning objectives. This sporadic approach lacked the rigor of implementation science and could not ensure that students’ attained evidence based knowledge for practice upon graduation. Therefore, the researchers adopted an implementation science strategy, the first step of which was a gap analysis. Literature revealed a consistent ‘time-to-implement’ challenge for nurses to incorporate evidence-based practice recommendations ¹. Understanding of the complexity of that challenge, and others, have minimal exploration in nursing. Therefore, to support the development of strategies to implement guidelines in nursing curricula, a gap analysis of organization specific guideline implementation attitudes, perspectives, and knowledge enablers and challenges for students was essential.

Method: A mixed-methods study included an online survey developed by the researchers as informed by the literature. Focus groups for each of full and part time undergraduate and graduate nursing students followed.

Results/Lessons: Findings indicate that although over 80% of student participants value guidelines and are willing to implement in practice, only 20% self-identify as ‘very knowledgeable’. Thematic analysis revealed that easy guideline accessibility and brevity, supported by integrated online learning would improve implementation.

Conclusion: Findings direct the development of a student led implementation strategy and knowledge translation via our innovative Guideline Website.

1. Entry-level evidenced-based practice training in physiotherapy students: Does it change knowledge, attitudes, and behaviours? A longitudinal study K Long, M McEvoy, L Lewis, L Wiles, M Williams, T Olds. Internet Journal of Allied Health Sciences and Practice (2011) 9 (3), 5

Effectiveness of the nursing education programs for leaders of nursing institutions in implementation of evidence-based family caregiver's resilience model of stroke patients

Jui-Yuan Su

CS4 - Education for EBHC, Hall C - Plenary, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To evaluate the effectiveness of a nursing education program for leaders of nursing institutions in implementation of evidence-based family caregiver's resilience model of stroke patients

Background: In response to the increasing demand of an aging society in Taiwan, healthcare professionals play crucial roles in elderly and long-term care and quality assurance of services. This study was funded by Ministry of Health and Welfare in Taiwan to establish and evaluate a continuous education program with update knowledge and EBP for caring stroke caregiver's in nursing institutions.

Method: A quasi-experimental before-and-after study. 71 nursing leaders around Taiwan were participated. The 20 hours educational intervention was based on theoretical and EBP practice as well as scenario-based simulation education, and the use of the adult learning teaching strategies. The self-directed learning questionnaire was administered before and after the education intervention. The data of the course satisfaction evaluation questionnaire, and OSCE evaluation were collected after the education intervention.

Results: The course satisfaction evaluation questionnaire received high score, especially in terms of EBP skills, curriculum design, teaching and learning strategies, and engaged with clinical practice. The OSCE score in related to stroke patients’ assessment skills, family needs identification, and nursing integrated intervention skills show acceptable scores. Furthermore, there was a significant difference between the pre and post score of self-directed learning score respectively.

Conclusion: This EBP course may provide nurses in nursing institutions in Taiwan and policymakers with a useful model for integrating EBP and advanced integrated practice continuous education program in nursing institution.

Post-market safety evidence and SGLT2 inhibitors for diabetes: a comparative case study of regulatory actions in four jurisdictions

Alice Bhasale¹, Barbara Mintzes^1,3, Ameet Sarpatwari²

¹The University of Sydney Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health,²Program On Regulation, Therapeutics, And Law (PORTAL), Division of Pharmacoepidemiology and Pharmacoeconomics, Harvard University,³School of Population and Public Health, University of British Columbia

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To explore divergence in regulatory post-market safety communications for sodium glucose co-transporter-2 inhibitors (SGLT2 s) and implications for clinical guideline development.

Background: The United States Food and Drug Administration has required evidence of cardiovascular non-inferiority for new diabetes drugs since 2008. The resulting studies of SGLT2 s yielded evidence of both potential cardiovascular benefit and serious adverse effects, including lower limb amputation and diabetic ketoacidosis. Regulators in Australia, Canada, the European Union (EU) and the US differed in their decisions about, and communication of, SGLT2 post-market safety risks.¹

Method: We conducted a process-tracing case study of post-market safety communications for SGLT2 s using publicly available regulatory documents in each jurisdiction from marketing approval to July 2018.

Results: Of five new safety issues identified (diabetic ketoacidosis, amputation, serious genitourinary infections, acute kidney injury and bone effects) US regulators issued alerts for all five; Canadian regulators for four, and Australian and EU regulators for two. All warned about ketoacidosis and amputation. Decision-making rationales were not consistently reported, nor were the data used to arrive at different decisions clearly described by all regulators. The EU regulator was most transparent, Australia's regulator the least. To further substantiate the risks, regulators asked industry sponsors to conduct additional post-market studies.

Conclusion: Guideline developers may not have access to the same post-market safety evidence as regulators or the pharmaceutical industry and may therefore rely on regulatory warnings, which differ between jurisdictions. Greater transparency regarding regulatory decision-making is required.

1. Perry L.T et al JAMA Intern Med. 2019; doi: 10.1001/jamainternmed.2019.0294

The quality of systematic reviews (SRs) of rare diseases

Regina Leadley¹

¹Kleijnen Systematic Reviews Ltd

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: The aim of this research project was to investigate the proportion, type and quality of SRs conducted in rare diseases using a random sample of SRs and corresponding quality appraisals using the ROBIS tool from KSR Evidence (a database of over 100,000 SRs from 2015 to present).

Method: A random sample of SR publications from 2016 was screened and SR of rare diseases identified. Rare disease status was confirmed using ORPHANET (a unique resource for rare diseases and orphan drugs) to determine their specific unique identifier (ORPHA number). Data were extracted on review type, study characteristics and quality assessment.

Results: From 517 SRs, 10 (1.9%) reviews of rare diseases were identified. Rare diseases included cystic fibrosis, sarcoidosis, Guillain-Barré syndrome, myasthenia gravis, Kawasaki disease, systemic sclerosis, eosinophilic granulomatosis with polyangiitis, mucopolysaccharidosis type VI and rare cancers such as Burkitt's lymphoma, sarcoidosis and non-Hodgkin lymphoma. There were 7 interventional studies and 1 diagnostic study. The remaining studies investigated disease characteristics and disease characteristics, treatment practices, and treatment outcomes respectively. There were 8 high ROB, 1 low ROB and 1 unclear risk of bias SRs. The main areas of concern were not reporting search strategies, language limitations and no quality assessment of included studies.

Conclusion: Systematic reviews of rare diseases represent approximately 2% of all reviews in this sample. The majority of studies were at high risk of bias. Given the paucity of research in this area, it is important to encourage good quality research and address the areas of concern.

Von Willebrand Disease: An International Survey Assessing Topics of Importance to Decision-makers

Reem Mustafa¹, Mohamad Kalot¹, Mohammed Alkhatib¹, Nathan T. Connell², Veronica Flood³, Romina Brignardello-Petersen⁴, Cary Clark⁵, Jenny Castano⁶, Ellen Riker⁷, Fiona Robinson⁸, Mark Skinner Skinner⁷, Paula James⁹

¹University of Kansas Medical Center,²Brigham and Women's Hospital and Harvard Medical School,³Medical College of Wisconsin,⁴Department of Health Research Methods, Evidence, and Impact, McMaster University,⁵International Society of Thrombosis and Haemostasis,⁶American Society of Hematology,⁷National Hemophilia Foundation,⁸World Federation of Hemophilia,⁹Queen's University

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Background: Von Willebrand disease (VWD) is an inherited bleeding disorder caused by a quantitative or qualitative deficiency of the protein von Willebrand factor (VWF). There is a lack of clear guidance on best practices to inform the care of patients with VWD. The American Society of Hematology (ASH), the International Society on Thrombosis and Haemostasis (ISTH), the World Federation of Hemophilia (WFH), and the National Hemophilia Foundation (NHF) are collaborating on developing guidelines to provide guidance on the diagnosis and management of VWD.

Objective/Methods: To identify and prioritize the main topics to be covered in a collaborative guideline development effort, we designed a survey and distributed it to different stakeholders (patients, caregivers, healthcare providers (HCP)).

Results: 601 participants from 71 countries worldwide responded to the survey (49% patients/caregivers, and 51% HCPs). The highest priority topics identified were diagnostic criteria/classification, bleeding assessment tools, treatment options for women, and surgical patients. In contrast, screening for anemia and plasma-derived therapy versus recombinant therapies were rated the lowest priority topics. There is overall agreement among different stakeholders about the priority topics to be covered in a guideline effort for VWD. However, healthcare providers prioritized diagnostic classifications and criteria, while patients prioritized issues related to the diagnostic journey.

Conclusion: Involving the broader stakeholder community from an early stage in guideline development ensured that the scope of the guideline is in line with the stakeholders’ priorities. This could also be viewed as an important step for early stakeholder engagement and later dissemination of the guidelines.

A Survey of Barriers and Facilitators to Clinical Practice Guidelines Implementation among Practicing Physicians in the Bamenda Health District, North West Region-Cameroon

Etienne Ngeh Ngeh^1,2,3, Mr. Atabon Fidelis Ako^2,3, Vivian Ngwashono Mukere^2,3

¹St. Louis University Institute of Health and Biomedical Sciences,²Research Organisation for Health Education and Rehabilitation-Cameroon (ROHER-CAM)),³Regional Hospital Baemenda

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To assess contextual barriers and facilitators to the implementation and utilization of CPGs among physicians practicing in the Bamenda Health District

Background: Evidence-based clinical practice guidelines (CPGs) implementation and utilization improves clinical practice and patients outcomes. Developing effective strategies to facilitate the uptake and use of CPGs demands an understanding of contextual barriers and facilitators.

Method: We conducted a descriptive cross-sectional study involving 28 Physicians practicing in the Bamenda Health District from April 1st to 30th, 2019. A modified standard questionnaire was used to collect data on contextual barriers and facilitators on the implementation of CPGs. Data was analyzed using Epi Info version 7.0

Results: A total of 28 participants responded to this study. The mean age was 35.6 years, 21(75%) were males. Twenty seven physicians (96.4%) implemented CPGs during practice. Most (48%) implemented CPGs because it addresses most clinical scenarios in daily practice, 37% because they are evidence-based while 11.1% was due to their easy layout and user friendly nature. Majority (75%) admitted CPGs provide enough room for them to make their own conclusions. Major barriers were the lack of support by leadership (50.4%), infrastructural-related resources (79.2%), lack of time and motivation (36%) to adopt new routine and unsatisfactory dissemination (72%) of CPGs.

Conclusion: There is substantial evidence of implementation of CBGs in the Bamenda Health District. However, to add value to the uptake and proper implementation of evidence-based CPGs, Hospital Managers and guideline developers should consider reducing these barriers for better health care outcomes.

The alignment of Australian national funding arrangements with evidence-based recommendations for psychosocial and mental health screening in the perinatal period

Sarah Norris, Agnes Wilson¹, Nicole Highet², Marie-Paule Austin³

¹Health Research Consulting,²Centre of Perinatal Excellence,³University of New South Wales

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To describe utilisation of two Medicare Benefits Schedule (MBS) items for pyschosocial and anxiety/depression screening by providers of antenatal care, and interpret this use in the context of local health policy developments in perinatal mental healthcare.

Background/Importance: Australian evidence-based recommendations in favour of psychosocial screening and screening for anxiety and depression in the perinatal period were published in 2011 and strengthened in 2017 (see Mental Health Care in the Perinatal Period). Implementation of these recommendations required investment and changes to health service delivery for many providers of maternal and early childhood healthcare. Separately, the MBS Review Taskforce undertook a review of items for obstetric care and provided advice regarding new items for “Planning and management of a pregnancy” which included mental health assessment and screening for drug/alcohol use and domestic violence.

Methods/Process: Relevant MBS utilisation statistics and Australian Bureau of Statistics on numbers of confinements were sourced and analysed.

Results/Lessons Learnt: In the first financial year there were 128,034 occasions of use of the items, accounting for ∼63% of pregnancies. Whilst this represents rapid, substantial uptake, variations in uptake by State/Territory are apparent, with the number of services per 100,000 population varying from 177 to 404 for item 16590, and from 40 to 293 for item 16591

Conclusion: The availability of Australian evidence-based guidelines has supported the implementation of successful national funding arrangements for perinatal care including psychosocial and mental health screening, although variations in State/Territory funding is likely to be contributing to variations in overall care.

Rapid review evidence generation to answer questions raised by policy makers and dissemination to evidence use by Ethiopian Federal Ministry of Health

Sudhakar Morankar¹

¹Evidence Based Health Care Centre, Jimma University

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To describe the process for generating rapid review questions and involving policy makers in evidence generation and uptake of by policy makers.

Importance: Researchers generate evidence – primary research or using knowledge translation tools but they do not know how evidence is used for policy making or guidelines preparation. Ethiopian Evidence Based Health Care Centre (EEBHCC) experimented evidence generation and using it for policy making or guideline preparations or continuous medical education training and so on.

Process: EEBHCC established National Advisory Committee (NAC) in Ministry of Health and NAC established Health Policy Analysis Unit (HPAU) in planning directorate of FMOH. NAC chairperson is high level decision making policy maker and invited 5 questions from various directorates of FMOH to be answered through rapid review by EEBHCC in 10 weeks. EEBHCC involved the person who raised the question in the whole process of evidence generation process to avoid ambiguity of understanding the question and what policy maker wants. Five rapid reviews were produced in 10 weeks and dissemination meeting was held in 11th week. NAC members and FMOH experts from all 16 departments participated in the meeting.

Lessons Learnt: Policy makers admired that this is first time policy makers and researchers are meeting fruitfully. Recommendations were useful for their current policy modifications, training, guidelines preparations. Dedicated leadership promotes embedding evidence utilization culture creating demand/supply structures and making part of health system. Formally FMOH established HPAU to generate questions and use evidence.

Conclusion: This approach will useful to low and middle income countries.

The developing of evidence summaries for point of care: current knowledge, limitation and future perspective in China

Weijie Xing^1,2, Zheng Zhu^1,2, Yingfeng Zhou^1,2, Yu Chen^1,2, Yan Hu^1,2

¹Fudan University,²Fudan University Centre for Evidence-based Nursing, A Joanna Briggs Institute Centre of Excellence

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To search the ES for point of care published in Chinese and analyze their developing process and reporting method.

Background: Evidence summaries (ES) summarize existing international evidence on healthcare interventions or activities, which can be useful for improving practice at the point of care. In China, increasing number of ES have been developed in recent five years. However, the development process and reporting method of them are various.

Methods: Published ES from the establishments of the databases to May 2019 were searched in two Chinese Academic Databases. Included ES were analyzed from aspects of question refinement, literature search, screening and selection of studies, critical appraisal, evidence and recommendation description.

Results: A total of 35 evidence summaries were identified. Various types of evidence were involved, including clinical practice guideline, best practice report, evidence summary, systematic review, expert opinion and original research. Among them, twenty-six defined research questions using framework. Almost all studies reported searching words and methods. Only five of them provided flowchart of study selection. Twenty-seven reported critical appraisal approach. Thirty provided recommendation level for practice.

Conclusion and implication: The overall quality of ES in China is good, however, the inconsistency of developing and reporting methods exists. Which type of evidence can be included in the ES, how to assess the quality of ES and how to unify the evidence level are still unclear. The review methodology, quality appraisal instrument and reporting guidelines are needed to guide the development of evidence summaries in future.

Managing uninterpretable or inconclusive results from diagnostic tests in guideline development

Xiaomei Yao¹, Jinhui Ma², Ping Zhang³, Chongfan Zhang³, Jonathan Sussman¹

¹Mcmaster University, Department Of Oncology,²McMaster University, Department of Health Research Method Evidence and Impact,³Children's Hospital of Fudan University, Clinical practice guideline conduction and evaluation center

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To manage uninterpretable or inconclusive results when calculating diagnostic test accuracy in guideline development.

Background: It is common for guideline developers to exclude patients with inconclusive results when calculating diagnostic test accuracy, which may yield biased synthesized accuracy outcomes. We recommend a sensitivity analysis method for managing this issue.

Method: We add a third row (Cell “e” and Cell “f”) to the traditional 2x2 table to calculate accuracy outcomes for a diagnostic test. Patients in Cell “e” and “f” have uninterpretable or inconclusive results from a diagnostic test. After reference standard confirmation, patients in Cell “e” have disease and patients in Cell “f” do not. We recommend researchers report the diagnostic accuracy outcomes (such as sensitivity, and specificity, etc.) at both the worst and best case scenarios in addition to the diagnosis accuracy based on the data excluding the uninterpretable or inconclusive results. For the worst case scenario, patients in Cell “e” are treated as false negative and patients in Cell “f” as false positive; and for the best case scenario, patients in Cell “e” are regarded as true positive and patients in Cell “f” as true negative.

Results: We will give an example (i.e. a 2x3 table) to demonstrate how to use this sensitivity analysis to handle uninterpretable or inconclusive results when calculating diagnostic test accuracy in guideline development.

Discussion: With the lower and upper limits of the diagnostic test results generated at the worst and best scenarios, guideline developers can make a more robust recommendation.

The Role of Scoping Review in the Development of Clinical Practice Guidelines

Qi Zhou^1,2,3, Qianling Shi^1,2,3, Ruiying Hou¹, Meng Lv⁴, Yongjie Yang⁵, Kefeng Liu⁵, Yaolong Chen^1,2,3, Kehu Yang^1,2,3

¹The First Clinical Medical School of Lanzhou University,²Evidence-Based Medicine Center of Lanzhou University,³WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,⁴School of Public Health of Lanzhou University,⁵The First Affiliated Hospital of Zhengzhou University

CS4 - Guideline Development and Advances, Riverbank Room 2, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To investigate the use and role of ScR in clinical practice guidelines.

Background: Scoping Review (ScR) is a method for identifying and collating research evidence. It is used to fully understand the quantity, attributes and characteristics of existing research in the field of interest of researchers and has been increasingly applied.

Methods: The Web of Science database was retrieved in 2014–2018. Firstly, the “scoping review, scoping study, scoping literature review” is used as the search term for the cited reference search, and then the “guideline, recommendation, consensus, guidance, statement” for free word retrieval, and finally through the Boolean logical operator for the combination Identify relevant literature. The Thematic Synthesis was used to summarize the role of ScR in the guidelines.

Results: A total of 240 related studies were retrieved, and 25 guidelines that cited ScR were eventually included.; the guidelines refer to 16 ScRs (64%) in the recommendation basis, 5 ScRs in the background of the guidelines (20%), and 2 ScRs in the guidance dissemination and implementation section; In the role of guideline development, 7 (28%) guidelines used ScR as qualitative evidence to support the effectiveness of drugs, 5 (20%) guidelines used ScR to mapping evidence of disease, 2 (8%) guidelines used ScR to qualitative synthesis of patient preferences and values.

Conclusion: At present, the number of ScR adopted in guidelines is extremely low, but the role of ScR in the development of guidelines is extensive.

Patient Involvement in Best Practice Guideline Development

Megan Bamford¹, Giulia Zucal¹, Deborah Flores¹

¹Registered Nurses’ Association of Ontario

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To meet international guideline development standards, the views and preferences of patients should be sought and used to inform the development of clinical practice guidelines (Brouwers, 2017). The Registered Nurses’ Association of Ontario (RNAO) has incorporated the use of multiple strategies in order to involve patients at various stages of the guideline development process.

Background: RNAO is the professional body representing registered nurses, nurse practitioners and nursing students in Ontario, Canada. The Best Practice Guidelines (BPG) Program is a ministry-funded and signature program at the RNAO with a mandate to develop, disseminate, evaluate and support the active uptake of evidence-based clinical and healthy work environment BPGs. To date, 54 BPGs have been developed. Over the past year, the BPG Program has made a concentrated effort to develop BPGs that focus on health issues that have traditionally been underserved by the health system.

Process: Key informant interviews are conducted with patients and/or caregivers to gather perspective on the scope and goals of the BPG. Patients and/or caregivers are involved in all guideline development panels. Patient organizations, as well as individual patients and caregivers are actively sought to provide input at external review.

Lessons Learnt: Involving patients that have traditionally been underserved by the health system requires creative and thoughtful strategies. Developing BPGs within the context of an organization that has developed meaningful partnerships with patients and organizations is an asset.

Conclusion: It is important to have patient and caregiver input and/or involvement at all stages of BPG development.

Incorporating the Patient Experience and Public Involvement in SIGN Guideline 159: Epilepsies in children and young people.

Beatrice Cant, Karen Graham¹, Sarah Florida-James¹, Toni Gibson

¹Healthcare Improvement Scotland

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Background: Patient and Public Involvement (PPI) is an integral part of the SIGN guideline development process. A number of methods currently capture PPI in guideline development, but the guidelines do not adequately reflect this. With an increased emphasis on shared decision making, there was a need to review sections of the guideline where giving the patient perspective and shared decision-making could be highlighted.

Aim:

To evaluate the success of:

1. methods to incorporate patient experience e.g. stories/quotes
2. incorporating patient priorities within guidelines
3. a new approach for the ‘Provision of information’ section
4. a template for capturing PPI
5. To understand the resources required for this new approach.

Method:

1. A literature review of existing methods for incorporating patient experience stories into guidelines was undertaken. Multiple stakeholders were engaged and representative quotes were collected.
2. A patient-focused literature search was conducted and key topics discussed with young people and parents.
3. A template was for presenting information that included patient quotes, information-giving points and documentation on PPI.
4. Qualitative feedback on the value of including patient quotes and the template were obtained from multiple stakeholders.
5. Resource requirements were recorded throughout the process.

Results/Conclusion: The literature review showed a lack of research. The inclusion of patient quotes, experiences and the template were well received with consensus from multiple stakeholders that this added value to the guideline. Resource requirements included staff time to deliver the new approach. SIGN is considering adopting this approach for all guidelines.

Identifying and integrating patient and caregiver perspectives in clinical practice guidelines for percutaneous renal biopsy

Ms Talia Gutman^1,2, Pamela Lopez Vargas², Karine E Manera^1,2, Jonathan C Craig^2,3, Martin Howell^1,2, David J Tunnicliffe^1,2, Rob MacGinely⁴, Emily J See⁵, Jeffrey Wong⁶, David Voss⁷, John Saunders⁸, Solomon Menahem⁹, Shilpanjali Jesudason¹⁰, Allison Tong^1,2, Paul Champion DeCrespigny¹¹

¹Sydney School of Public Health, The University Of Sydney,²Centre for Kidney Research, The Children's Hospital at Westmead,³College of Medicine and Public Health, Flinders University,⁴Eastern Health Clinical School, Monash University,⁵Department of Nephrology, Monash Health,⁶Department of Nephrology, Liverpool Hospital,⁷Department of Renal Medicine, Counties Manukau Health,⁸Renal Unit, Royal Prince Alfred Hospital,⁹Department of Epidemiology and Preventative Medicine, Monash University,¹⁰Central and Northern Adelaide Renal and Transplantation Service and Department of Medicine, University of Adelaide, Royal Adelaide Hospital,¹¹Department of Nephrology, The Royal Melbourne Hospital

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To identify and integrate patient priorities and perspectives into the Kidney Health Australia – Caring for Australasians with Renal Impairment clinical practice guidelines for renal biopsy, to ensure patient-relevance.

Background: Percutaneous renal biopsy is often essential for providing reliable diagnostic and prognostic information for people with suspected kidney disease, however the procedure can lead to complications and concerns among patients.

Methods: We convened a workshop, consisting of three simultaneous focus groups and a plenary session, with ten patients who had undergone a renal biopsy and seven caregivers. Topics and outcomes prioritised by patients and their caregivers were compared to those identified by the guideline working group, which was comprised of seven nephrologists. Transcripts and flipcharts were analysed thematically to identify the reasons for participants’ choices.

Results: In total, 34 topics/outcomes were identified, 14 of which were common to the list of 28 previously identified by the guideline working group. Most of the new topics identified by patients/caregivers were related to communication and education, psychosocial support, and self-management. We identified five themes underpinning the reasons for topic and outcome selection: alleviating anxiety and unnecessary distress, minimising discomfort and disruption, supporting family and caregivers, enabling self-management, and protecting their kidney. A new topic on patient care and education was added to the guideline as a result.

Conclusion: Patient and caregiver involvement in developing guidelines on renal biopsy ensured that their concerns and needs for education, psychosocial support, and self-management were explicitly addressed; enabling a patient-centred approach to renal biopsies.

Using qualitative evidence to meet the needs and expectations of patients and families in ICU

Kathryn Kynoch^1,2, Mary-Anne Ramis^1,2, Ms Annie McArdle^1,2

¹Mater Health,²Mater Health-Queensland Centre for Evidence Based Nursing and Midwifery: a Joanna Briggs Centre of Excellence

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Background: The admission of a critically ill patient to ICU is a distressing life event for both patients and families. Literature reports patients in ICU want family close-by and actively involved in care, however the family's ability to make care decisions is often affected by their own values, needs, and preferences.¹

Objective: To synthesise qualitative data on the experiences and needs of families with a relative in an adult ICU with recommendations for care.

Method: A systematic review was undertaken utilising Joanna Briggs methods.² A previously published protocol guided the inclusion criteria.¹

Results: Twenty-seven phenomenological studies were synthesized with themes identified as: 1) environmental stress, 2) proximity, 3) participation in care, 4) meeting information, psychosocial and spiritual needs, 5) reassurance and 6) minimising daily life disruption.

Discussion: ICU clinicians increasingly need to engage patients and their families to ensure care and treatment is appropriate and consistent with patients and family preferences. This paper will discuss ways these recommendations from qualitative evidence can be used to meet the needs of family in ICU including facilitating proximity to the patient, open communication and shared decision making.

1. K. Kynoch, CJ. Cabilan, A. McArdle. Experiences and needs of families with a relative admitted to an adult intensive care unit: a qualitative systematic review protocol. JBI database of systematic reviews and implementation reports. 2016 Nov 1;14(11):83–90.

2. E. Aromataris, Z. Munn (Editors), Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute [Internet] 2017 [Cited November 20, 2017]; Available from: https://reviewersmanual.joannabriggs.org/

Social participation as a mechanism of transparency and patient empowerment in the elaboration of Brazilian guidelines

Sarah Silva¹, Jorgiany Ebeidalla¹, Rodrigo Sena¹, Camila Chacarolli¹, Carolina Neves¹, Vania Santos¹

¹Brazilian Ministry of Health

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Objective: To describe the experience on the insertion of patient representatives in elaboration and updating of the guidelines of the Brazilian public health system.

Introduction: The guidelines in Brazil are guiding access to technologies and standardize clinical conduct in all health services, assuming a normative role in the country. The patients are the final beneficiary of this document and their participation in moments of construction and revision of these documents is an opportunity to direct decision making and promote transparency to the process.

Methods: Report of experience of the technicians who accompanied the participation of patients in meetings of scope and panel of specialists in the period from January 2018 to April 2019 conducted by the Ministry of Health. The themes of the meetings involved diseases such as diabetes, rheumatoid arthritis, rare diseases, neoplasms and others.

Results: The participation of patients added new perspectives in the conduction of some research questions and judgment of some outcomes. The level of knowledge of the patients was considered high, revealing the empowerment of many of them about the disease. There was an opportunity to present to the patients the methodology used in the guidelines, highlighting the importance and commitment of the use of trustworthy evidence in decision making.

Lessons Learned: The participation of patients can add various information relevant to the process, beyond patient values and preferences.

Conclusion: The involvement of patients is important to guide technical discussions and reinforce the value of guidelines for an effective and safe treatment for patients.

Womens’ verdicts on consequences and labelling of gestational diabetes: a community jury

Rae Thomas¹, Anna Scott¹, Ms Rebecca Sims¹, Louise Craig¹, Leigh-Anne Claase², Julia Lowe³, Clare Heal⁴, Ms Leah Hardiman⁵, Paul Glasziou¹

¹Institute of Evidence-Based Healthcare, Bond University,²Therapeutic Guidelines Limited,³School of Medicine and Public Health, University of Newcastle,⁴School of Medicine and Dentistry, James Cook University,⁵Maternity Choices Australia

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To explore the values and preferences of women potentially impacted by a diagnosis of Gestational Diabetes (GDM).

Background: There is no international agreement about how to diagnose GDM. In 2015, Australia adopted a new definition and testing procedure. Significantly more women have been diagnosed with GDM but with little difference in health outcomes.

Method: We aimed to recruit 15 women from regional and rural areas within driving distance to the Gold Coast. The Community Jury (CJ) was conducted over two days. Women deliberated on:

• which important consequences of a diagnosis of GDM should be considered when defining GDM?
• what should Australian health practitioners call the condition known as GDM?

Results: Eight women attended the CJ and their recommendations were a consensus. Women were surprised that the level of risk for physical harms was low but emotional harms were high. The final ranking of important consequences (high to low) was: women's negative emotions; the burden of managing GDM; over-medicalised pregnancy; opportunity to minimise infant risks; opportunity to improve lifestyle; and macrosomia. To describe the four different clinical states of GDM, the women chose three different labels. Only one was GDM.

Conclusion: Definitions and guidelines often fail to ask the affected public about their values and preferences. The current glucose threshold for GDM in Australia is set at a cut-point for adverse risks including macrosomia and neonatal hyperinsulemia. When informed, women ranked macrosomia as the least important consequence. Community voices impacted by these decisions should be embedded in the decision-making process.

The perspective of patients living with chronic heart failure on self-management interventions: A mixed-methods overview for the COMPAR-EU project

Ms Ena Niño de Guzmán², Pablo Alonso-Coello¹, Ms Claudia Valli², Javier Pérez-Bracchiglione³, Ms Karla Salas-Gama², Andres Viteri-García⁴, Ms Laura Martinez-García²

¹Iberoamerican Cochrane Center (IIB Sant Pau-CIBERESP),²Iberoamerican Cochrane Center, Hospital Sant Pau,³Interdisciplinary Centre for Health Studies (CIESAL), Universidad de Valparaíso,⁴Cochrane Ecuador. Centro de Investigación en Salud Pública y Epidemiología Clínica (CISPEC). Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To identify, appraise and analyse the perspective of patients living with chronic heart failure (CHF) and their caregivers on self-management interventions (SMIs)

Background/Importance: SMIs are complex interventions considered critical to achieve high-quality care for patients with CHF. Incorporating the perspective of patients with CHF and their caregivers is essential for the development of trustworthy recommendations and for a successful implementation of SMIs. COMPAR-EU is a EU funded project that aims to identify, compare, and rank the most effective and cost-effective self-management interventions (SMIs) in four high-priority chronic conditions.

Methods/Process: We searched systematically in three databases up to March 2019. We included quantitative, qualitative evidence synthesis (QES) and mixed methods synthesis (MMS). Pairs of reviewers independently assessed their quality using the Joanna Briggs Institute Critical Appraisal Checklist and estimated the extent of overlap of primary studies. We applied a convergent parallel synthesis approach with narrative and thematic synthesis of findings, guided by an ad hoc SMI taxonomy.

Results/Lessons Learnt: We identified 1,900 unique references and included 47 SRs (published from 2007 to 2019). Most SRs were QES (45%) or MMS (34%). The more frequent synthesis methods in QES were thematic synthesis (33%) and meta-ethnography (29%). For MMS, it was the integrative approach (69%), and for quantitative SRs the narrative synthesis (80%). The degree of overlap across reviews was less than 25%.

Conclusion: Our findings will inform the SRs research agenda on SMIs field and will be incorporated in the development of recommendations and other decision-making tools in the COMPAR-EU project.

Incorporating patients’ preferences in the development of core outcome sets for four chronic conditions: COMPAR-EU project

Carola Orrego^1,2,3, Marta Ballester^1,2,3, Ana Isabel González-González^1,2,3,8, Estela Camus^1,2, Monique Heymans⁴, Pablo Alonso-Coello^6,7, Marieke van der Gaag⁴, Hector Pardo-Hernandez^6,7, Claudia Valli⁶, Ena Niño de Guzman⁶, Carlos Canelo-Aybar^6,7, Oliver Groene⁵, Rosa Sunol^1,2,3

¹Avedis Donabedian Research Institute (FAD),²Universitat Autonoma de Barcelona,³Red de Investigación en Servicios de Salud en Enfermedades Crónicas (REDISSEC),⁴Netherlands institute for health services research (NIVEL),⁵OptiMedis AG,⁶Iberoamerican Cochrane Centre-Biomedical Research Institute Sant Pau (IIB Sant Pau),⁷CIBER de Epidemiología y Salud Pública (CIBERESP),⁸Institute of General Practice, Goethe University

CS4 - Including Patients and Consumers, Riverbank Room 5, November 1, 2019, 11:00 AM - 12:30 PM

Aim: To develop core outcome sets (COS) for evaluating the effectiveness of self-management interventions (SMIs) among adults living with one or more of four prioritized chronic conditions: type 2 Diabetes (T2DM), Obesity, Heart Failure (HF), and Chronic Obstructive Pulmonary Disease (COPD). These COS are being used in the context of COMPAR-EU, a European project that aims to identify and rank the most effective and cost-effective SMIs for adults living with these conditions.

Background/Importance: Inconsistency and heterogeneity of outcomes measured in clinical trials limits the possibilities of comparing SMIs effectiveness. COS inform the selection of patient-important outcomes for the development of more trustworthy and implementable clinical guidelines.

Methods/Process: A list of outcomes for each condition was developed and ranked by patients and patient representatives using a modified Delphi Technique. Additionally, a scoping review of systematic reviews on patients’ values and preferences was conducted. The results of these activities provided a preliminary list of outcomes that were discussed and prioritized in a face-to-face consensus meeting with patients, patient representatives, healthcare professionals and researches.

Results/Lessons Learnt: Thirty-nine patients and patient representatives participated. Delphi survey results were combined with those of the overview of systematic reviews. The final selection of COS for each LTC included 16 outcomes for COPD, 16 for HF, 13 for T2DM and 15 for Obesity.

Conclusion: The proposed COS will foster informed, patient-facing clinical decision-making. They will also serve as the basis for developing recommendations in the COMPAR-EU Project.

Implementing work-related Mental health guidelines in general PRacticE (IMPRovE): Protocol for a pragmatic stepped-wedged cluster-randomised controlled trial

Danielle Mazza¹, Alex Collie², Justin Kenardy³, Bianca Brijnath⁴, Duncan Mortimer⁵, Joanne Enticott¹, Michael Kidd⁶, Lyndal Trevena⁷, Samantha Chakraborty¹

¹Monash University, Department of General Practice,²Insurance Work and Health Group,³University of Queensland,⁴National Ageing Research Institute,⁵Monash University, Centre for Health Economics,⁶University of Toronto,⁷The University of Sydney

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To implement clinical guidelines for the diagnosis and management of work-related mental health conditions (MHCs) in general practice.

Background: General practitioners (GPs) have an important and complex role in the recovery of patients with work-related MHCs. A new evidence-based clinical guideline has been developed to assist GPs with providing optimal care to patients with work-related MHCs, however unless the guideline is actively implemented it risks being underutilised.

Method: We will undertake a pragmatic cluster randomised controlled trial using a stepped-wedge approach to evaluate whether an intervention (comprising academic detailing, resource provision, and support through a digital community of practice) improves implementation of the guideline and patient outcomes. We will use guideline-based quality indicators to assess: (1) GP adherence to guideline recommendations at baseline, 12 months post-intervention and at 24-months follow-up; and (2) cost-effectiveness of the intervention. We will also assess health and return-to-work outcomes of participating patients at baseline, 6 months and 12 months following the initial GP consultation. An economic evaluation of the trial will be conducted and we will plan for sustainability and national scale-up.

Results: The trial commenced in April 2019 and is in the establishment phase. The trial builds on a body of work undertaken with key stakeholders defining the role of GPs in compensable injury, exploring their concerns and the development of evidence-based guidelines to address them.

Conclusion: This project will result in evidence-based improvements in GP care, improvements in health and return to work outcomes for patients, and sustainable policy change.

Can an impact framework be developed for NHMRC approved clinical practice guidelines?

Erica Crone¹, Ms Sally Wright¹, Geraint Duggan¹

¹National Health and Medical Research Council (NHMRC)

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To determine if it is feasible to develop and implement a framework to measure the impact of NHMRC approved clinical practice guidelines.

Background: The NHMRC Chief Executive Officer has the power to approve guidelines developed by an external body. NHMRC approval indicates to users that a guideline is of high quality, is based on the best available scientific evidence and has been developed to rigorous standards. They are recognised as representing current knowledge and best health practice.

Importance

NHMRC approved clinical practice guidelines represent a significant investment of public funds and volunteer labour and there is growing awareness of the importance of demonstrating the impact of these investments. However, a systematic method to measure the impact of NHMRC approved guidelines has not yet been developed. This project aims to determine whether this is feasible.

Method:

The protocol includes:

• Systematic literature review on impact measurement for clinical practice guidelines.
• Email questionnaire of organisations that developed and funded NHMRC approved guidelines in the last five years. The questionnaire will focus on stakeholders’ understanding of impact and evaluation measures and how they have included them in their guidelines.
• Draft impact framework for clinical practice guidelines developed using information from the systematic literature review and the email questionnaire. Stakeholders will be involved in the drafting of the framework which will be subject to public consultation.

Conclusion: This project will determine whether an impact framework can be developed for clinical practice guidelines developed to meet NHMRC requirements for approval.

Implementation Support for Guidelines and Health Technology Assessments (HTAs): Is it all the same?

Ms Brendalynn Ens¹

¹CADTH

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: This presentation will discuss how science-supported implementation strategies for (clinical practice guidelines) CPG implementation have common application for implementation of health technology assessment research (HTA). Despite some differences in research formats and processes, CPGs and HTAs each face implementation gaps and challenges to be consistently used in practice settings.

Background: The implementation of evidence-informed health care practice is a complex process undertaken in complex environments. Implementation science helps us to understand what works, for whom, in what environments, and under what circumstances in health systems to get evidence effectively and consistently used. Implementation strategies for particular types of evidence, such as clinical practice guidelines (CPGs), are often carefully planned and specifically focused. These same principles and approaches hold true for health technology assessment (HTA) implementation.

Methods: The presentation will compare and discuss several specific implementation approaches designed to support uptake and use of research products. Lessons learned and key ‘threads’ of commonality from HTA and CPG implementation approaches will be highlighted through use of an example.

Results/Conclusion: While clinical practice guideline (CPG) and health technology assessment (HTA) processes and their final document products may differ, these two worlds are not so far apart as once thought for implementation. Common learning for effective dissemination and implementation support can be considered for both, with the consistent outcome of effecting favorable practice change.

Implementing an evidence-based pre-admission clinic for elective surgery patients in a university private hospital in Australia: Challenges and roadblocks

Ms Jingjing He¹, Anne Scott¹, Ms Eliza Kenny¹, Retha Phillips¹, Ms Deborah O’Neill¹, David Gillatt¹

¹Macquarie University Hospital

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To explore barriers to the implementation of an evidence-based nurse-led pre-admission clinic (PAC) in a university private hospital in Australia.

Background: The PAC is an outpatient clinic to prepare patients for surgery and their hospital stay. In Australia, approximately 67% of elective surgery admissions were in private hospitals. However, there is a lack of standard practice guidelines for PACs in private hospitals in Australia. As a result of a funded projected in 2016, we developed an evidence-based PAC model to guide patient triage, pre-operative assessments, multi-disciplinary team review and transfer-of-care planning for surgical patients.

Methods: A mixed-methods evaluation was conducted to assess the first stage of implementation. Data were collected from hospital electronic medical records, incident reporting system, audit reports, observation and mapping of the PAC, document review, PAC staff self-reporting tool, brief staff interviews (n = 6) and an online survey to anaesthetists (n = 93). Quantitative and qualitative data were collected concurrently, analysed separately and were then compared and integrated in the final analysis to draw conclusions.

Results: Approximately 35% of patient admission forms were unavailable at the time of triage and there were variations in documentation quality. Other potential barriers to influence implementation are limited doctor engagement, lack of senior medical input in PAC service delivery, physical constraints of the workplace and IT system constraints.

Conclusion: Key factors to guide further implementation are 1) active engagement with clinicians and patients; 2) effective management and leadership; 3) a conducive work environment; and, 4) a functional IT system to facilitate PAC processes.

Opportunities and challenges in implementing best practice in health care: a South Australian experience

Saravana Kumar¹, Ms Esther Tian¹

¹School Of Health Sciences, University Of South Australia

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: This presentation, using a case-study approach, will showcase how health care stakeholders in South Australia have implemented best practice and achieved positive impacts on health care processes and outcomes.

Background: Health care today is confronted by a myriad of challenges. Successfully addressing these challenges requires a health system which is nimble, responsive to change and takes a system-wide focus. While achieving change in health care is complex, in South Australia we have successfully achieved this across a range of professions.

Method: Using a case-study approach, new models of care from a range of health disciplines (such as physiotherapy, occupational therapy, medicine) and clinical areas (such pediatrics, orthopedics) were implemented across South Australia, underpinned by a range of theories of implementation science. Quantitative (such as waiting time, cost, and safety) and qualitative (patient perspectives) were captured to measure the impact.

Results: The results these case-studies indicated a number of positive impacts on health care processes and outcomes. Informed by implementation science. these new models of health care reduced waiting times, improved efficiencies (by reducing cost) and access to care while ensuring safety (no adverse events). Patients were receptive and satisfied with these new models of care (with no patient complaints).

Conclusion: Implementation of these models of care highlighted critical success factors which need to be identified early, such as identified need, stakeholder engagement, effective use of data, if successful and sustainable change is to be achieved.

Justifying Disinvesting: Kinked Thresholds?

Susan Griffin², Matthew Taylor³, Bhash Naidoo¹

¹National Institute for Health And Care Excellence,²Centre for Health Economics, University of York,³York Health Economics Consortium Ltd

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To assess whether different cost-effectiveness thresholds should be used for making disinvestment decisions to investment decisions given social value arguments.

Background: NICE assesses whether an NHS intervention is cost-effective by using the cost per quality adjusted life year (cost/QALY), and if the value is equal to or less than £20,000/QALY it is judged to an efficient use of public money, where £20,000/QALY represents the opportunity cost of forgone healthcare.

For interventions deemed cost-effective, but which reduce QALYs for individuals (with lower costs), it has been suggested that differential preferences for health losses compared to health gains could be addressed with the adoption of a ’kinked’ decision threshold, where a higher threshold would be adopted when recommending to disinvest in interventions.

Methods: A literature review was undertaken to explore the theoretical, empirical and social value arguments for and against of the use of a different threshold for disinvestment and investment decisions.

Results: The opportunity cost for public sector budgets are not necessarily equivalent to opportunity cost in terms of individual income. The appropriateness of using willingness to pay and willingness to accept to inform resource allocation depends on the decision context, and health is one context where they may be inappropriate. There is a lack of evidence to support the need for or extent of any loss aversion or status quo bias in societal level decisions about health.

Conclusions: NICE's approach of applying a single threshold to all decisions is consistent with maximising health gains from limited available resources.

Factors affecting implementation of patient identification and matching guidelines in the ambulatory care setting

Ms Dung (Louise) Tran

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim/Background: Incorrect patient identification and mismatching to clinical interventions have led to adverse events and poorer patient outcomes. Ambulatory care settings are at particular risk for patient identification and matching errors owing to a high turnover of patients and a high number of clinical interventions undertaken. Following anecdotal evidence from our Local Health District ambulatory care units that international and national guidelines may not be being followed, we undertook to investigate current patient identification and matching practices in these settings.

Method: An observational implementation project using JBI Practical Application of Clinical Evidence System (PACES) and Getting Research into Practice (GRIP) audit and feedback tool was used to identify current patient identification and matching practices. A baseline audit was completed, and from the audit results patient identification bands were instigated. An interim audit was then undertaken followed by implementation of additional support strategies and a final audit completed.

Results/Conclusion: 4 ambulatory care units across three hospitals in a Sydney Local Health District were identified. In the baseline audit 39 patient procedures were observed involving 14 staff, and 41 patients were interviewed. The units varied in practices and compliance levels. Factors affecting implementation of the guidelines were identified, such as personal beliefs and time. After the interim audit there was resistance from some staff regarding the introduction of patient identification bands. This presentation will focus on the experiences and factors that affected implementation at all stages of the project with recommendations given for future similar projects.

The Role of Mobile Social Media for Evidence Dissemination: A Case of China

Zheng Zhu^1,2, Weijie Xing^1,2, Yingfeng Zhou^1,2, Yan Hu^1,2

¹School Of Nursing Fudan University,²Fudan University Centre for Evidence-based Nursing: A Joanna Briggs Institute Centre of Excellence

CS5 - Implementation and Impact, Riverbank Room 2, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To explore how evidence is disseminated on a mobile social media platform in China and to discuss the role of mobile social media for evidence dissemination.

Background: Current mobile information technologies fundamentally change evidence dissemination from the perspective of both evidence seekers and evidence providers. While mobile social media undeniably play a more and more important role in information dissemination, there is no related study exploring how evidence was disseminated on a mobile social media platform.

Methods: The authors released five posts on Wechat in March 2019 to promote the researches. Several indexes including number of new followers, reading number, number of forwards inside and outside application, and number of comments were obtained after the evidence posted. Communication effects of the posts were collected daily until the end of April 2019.

Results: The dissemination rate was 3.49% per post. The evidence's online activity peaked in the first day after released (2769 views per post). Online activity plateaued by 4 days after publication with 69 views daily per post. Among all post views, 63.47% were via followers’ direct reading on our mobile platform, followed by reading via Moment (24.07%) and reading via person-to-person or person-to-group forwarding (8.56%).

Conclusion: Our data suggest that mobile social media can increase evidence dissemination and have revolutionized the way evidence can be distributed compared to Web 2.0. We believe that organizations and individuals involved in development and application of evidence-based guidelines should consider using mobile social media to increase the dissemination of evidence.

Partnership working to add value to guidelines, pre-, during and post-production

Sarah Florida-James¹, Kay Cooper², Ms Pamela Kirkpatrick², Karen Graham⁶, Ms Anissa Tonberg³, Jay Shetty^4,5, Beatrice Cant¹

¹Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland,²Robert Gordon University, Scottish Centre for Evidence-based, Multiprofessional Practice: A Joanna Briggs Institute Centre of Excellence,³Epilepsy Scotland,⁴NHS Lothian,⁵University of Edinburgh,⁶Healthcare Improvement Scotland

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: Collaboration to achieve tailored objectives for SIGN guideline159: Epilepsies in Children.

To ensure:

1. Patient and Public Involvement (PPI), and improved documentation of this within the guideline
2. The guideline met user needs
3. The methodological approach addressed topics in the most appropriate way
4. Maximum proposed impact
5. Engagement and raise awareness of the guideline pre, during and post guideline development

Background: Collaboration is an inherent part of guideline development, and often centred on guideline production. This pilot focused on partnership working pre-, during and post-production.

Method: Partnership working with 12 partners.

Engaging and working with:

1. Charities to gather and incorporate patient/young person quotes and information-giving points within the guideline and gathering multi-stakeholder qualitative feedback on the value of this. Videos of young people were made to capture their stories and involvement.
2. The Scottish Paediatric Network (SPEN) for a national pre-consultation meeting around their vision and key topics
3. Robert Gordon University to incorporate a mixed methods approach, incorporating this within the guideline, and gathering multi-stakeholder qualitative feedback on the value of this
4. SPEN/charities to draft the implementation and audit strategy and quotes on the proposed impact
5. Partners for 5 public engagement activities (n = 128), plus an exhibition stand at Scottish Government and a Cross Party Group

Results/Conclusion: Consensus from multiple stakeholders that the use of PPI and mixed methods approach added value to the guideline. Feedback from stakeholders that pre-and post-consultation, awareness raising and collaboration around implementation and impact also added value.

Last library standing: the Australian Clinical Practice Guidelines Portal since 2015

Ms Alice Downing¹, Erica Crone¹, Geraint Duggan¹

¹National Health and Medical Research Council

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To assess the value of the Australian Clinical Practice Guidelines Portal (the Portal) and the impact of the 2015 inclusion criteria change. To compare the methods of NHMRC approved guidelines and other evidence-based guidelines on the Portal.

Background and Importance: The Portal was launched in 2011 as a single access point for Australian clinical practice guidelines. The Portal links to high quality NHMRC approved guidelines and evidence-based guidelines. In 2015 the Portal inclusion criteria changed. The change encouraged guideline developers to be more transparent about their methods and to produce evidence-based guidelines.

Methods: Portal data from 2014–2015 and 2015–2016 was compared to assess the effect of the 2015 inclusion criteria change. The guideline development methods of NHMRC approved guidelines and evidence-based guidelines included on the Portal were compared to identify differences.

Results: The Portal is the only Australian guideline library of it's kind. From 2014–15, 94 guidelines were added to the Portal (3 NHMRC approved). From 2015–16, 13 guidelines were added to the Portal (5 NHMRC approved). Since 2015, more guideline developers are seeking and attaining NHMRC approval and the number of evidence-based guidelines meeting the Portal inclusion criteria is declining. There are a number of factors that influenced this change.

Conclusion: The Portal provides a one-stop-shop for Australian evidence-based clinical practice guidelines. Less guidelines have been added to the Portal since the inclusion criteria changed in 2015, however all guidelines on the Portal are of a high quality and more guideline developers are seeking NHMRC approval.

Diamond Guidelines on a Zirconia Budget: Results of an innovative meeting approach used by G-I-N North America

Ms Mary Nix¹, Ms Pamela Genix³, Ms Brendalynn Ens²

¹Agency for Healthcare Research and Quality (AHRQ),²Canadian Agency for Drugs & Technologies in Health (CADTH),³Oncology Nursing Society

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: An innovative approach was used by G-I-N/North America in March 2019 to encourage members of the regional guideline community to learn, share successes and discuss their challenges in guideline development, adaptation, implementation and updating. This novel approach, “Open Space”, is a format known to stimulate discussion, debate and solutions across diverse participants. Our presentation will highlight the methods used, the emerging themes, evaluation results, and applicability to the greater international network.

Background/Importance: With many diverse guideline interests across members, planning an event to suit all needs is challenging. Open Space meeting design supports active expression of interests and finding common ground.

Methods/Process: Using Open Space methodological components, the day unfolded through expert facilitation, ignited participation, and priority topic generation. Participants selected topic sessions to attend, contributed to teaching/learning and note-taking, and ended with an activity synthesizing the day's work.

Results/Lessons Learnt: Sixty (60) members of the community attended the event; twenty four (24) topics were identified. Emerging themes included best practices for guideline development, patient and stakeholder engagement, implementation, and measuring the impact/outcomes of guidelines in patient care. Overall, the meeting format was well received with sixty seven percent (67%) of attendees reporting they found the event valuable. Additional evaluation results will be presented.

Conclusion: The format afforded individuals committed to learning to do so informally and safely. It promoted applied learning, brainstorming and collegiality. Our experience illustrated a valuable strategic approach that ensured “something for everyone” to meet diverse learning needs and promote future collaborations.

Guidelines for Guidelines: a new online handbook

Stephanie Goodrick¹, Kristal Jackson¹, Geraint Duggan¹, Davina Ghersi¹

¹National Health And Medical Research Council

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To develop a new handbook for guideline development.

Background: NHMRC has published methodological advice for Australian guideline developers for over 20 years. With the release of new NHMRC Standards for Guidelines in 2016, NHMRC sought feedback from guideline developers on priority topics and information that would be of most value to them. Priorities proposed in these initial consultations were among the first few modules to be developed: how to manage conflicts of interest; involving consumers; and whether to adopt, adapt or start guidelines from scratch.

Importance/Process: Leveraging national and international expertise, Guidelines for Guidelines is an online resource written by, and for, guideline developers. Applicable to clinical, public and environmental health guidelines, each module presents practical advice in a sequence of steps that reflects a guideline development pathway. A ‘living’ resource, the Guidelines for Guidelines relies on ongoing feedback from guideline developers through targeted and public consultation to keep content current and relevant.

Results: The first 18 modules of the Guidelines for Guidelines were launched in November 2018. Over 20 more modules are in progress. Guidelines for Guidelines already has a strong online presence in Australia with modules beginning to be cited in journal publications. Early feedback indicates that users find links to quality tools, guidelines and resources provided alongside contextual information one of the most useful features.

Conclusion: With modules relevant to public and environmental health guidelines currently being scoped, NHMRC now seeks to address the complex methodological issues that arise in these types of guidelines.

NICE's approach to maximising opportunity by working collaboratively

Emma McFarlane¹, Kay Nolan¹, Philip Alderson¹, Nichole Taske¹, Jenny Mills¹, Therese Docherty²

¹National Institute for Health And Care Excellence,²Cochrane UK

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To develop an approach for working collaboratively to expedite surveillance decisions and guideline updates.

Background: There are two main points in the lifecycle of NICE guidelines where Cochrane reviews are informative:

• During guideline development to inform recommendations
• During guideline surveillance to inform decisions about whether recommendations are current

We outline a pilot mechanism where Cochrane produces a review required by NICE, at a pre-determined cost, within an agreed timeframe. Overall, this scheme could avoid unnecessary duplication of effort.

Methods: Candidate topics are identified by the technical team working on a surveillance review. Additionally, when guidelines are being developed, questions may be identified where a Cochrane review would be helpful:

• A briefing paper is drafted
• Candidate topics are agreed to proceed by an oversight group
• The relevant Cochrane group will complete the briefing paper outlining a proposal to conduct the review including timescales and costs.

Results: To date, 10 Cochrane reviews have been commissioned, all updates. Two reviews have published and fed into surveillance decisions to update NICE guidelines on falls and intrapartum care. Aligning Cochrane review and guideline protocols from the outset is important as differences can make it difficult to use a Cochrane review directly.

Conclusion: A new or updated Cochrane review is useful for deciding whether recommendations remain current. This may avoid NICE commissioning a guideline update unnecessarily. For guideline updates, it is anticipated that the Cochrane review could be timed to feed into guideline development and make the process more efficient.

Can guidelines be successfully adapted to the needs of other countries and healthcare systems? Experience of contextualising NICE guidelines for New Zealand and Ireland.

Rachel Woodcraft¹, Steven Barnes¹, Nichole Taske¹

¹NICE

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: Having now successfully contextualised 4 NICE guidelines for other health care contexts we aim to share our experience of the benefits, challenges and lessons learnt during the process. With the aim of assisting other guideline producers who are interested in implementing similar programmes, or healthcare providers wanting to adopt NICE guidelines.

Background/Importance: Guideline production is resource intensive in terms of time and costs, and worldwide there are multiple different institutions developing guidelines on the same topics, using similar methods and processes. Re-purposing guidelines that have already been produced to the highest quality, and adapting them to the needs/context of the target healthcare provider through ‘contextualisation’, avoids duplication of work, reduces cost and development time, and still allows for local ownership and relevance.

Methods/Process: The NICE contextualisation process remains true to many of the principles of ADAPTE, however it provides a more pragmatic method of adaptation.

Results/Lessons learnt: Experience will be provided from 4 NICE guidelines that have been contextualised for use in New Zealand and Ireland. The most frequently used contextual rationale for adapting recommendations included differences in drug licensing/availability, the organisation and delivery of services and population differences.

Conclusion: The results of this work should provide some insight into the advantages of guideline contextualisation. We hope by this to encourage guideline sharing across the international health and care community, with benefits being seen in efficiencies through the de-duplication of work, while maintaining guideline production of the highest standards.

Co-creation to develop international guidelines for the management of kidney disease

David Tunnicliffe^1,2, Martin Howell^1,2, Michael Cheung³, Suetonia Palmer⁴, Giovanni Strippoli^1,2,5,6, Marcello Tonelli⁷, Allison Tong^1,2, Jonathan Craig^1,2,8

¹School of Public Health, The University Of Sydney,²Centre for Kidney Research, The Children's Hospital at Westmead,³Kidney Disease: Improving Global Outcomes,⁴Department of Medicine, University of Otago,⁵Divarum Academy,⁶Department of Emergency and Organ Transplantation, University of Bari,⁷Cumming School of Medicine, University of Calagary,⁸College of Medicine and Public Health, Flinders University

CS5 - Working Together in Guideline Development, Hall C - Plenary, November 1, 2019, 1:30 PM - 3:00 PM

Aim: To describe the collaboration between Cochrane Kidney and Transplant (CKT), Standardised Outcomes in Nephrology (SONG) Initiative, and Kidney Disease: Improving Global Outcomes (KDIGO) to develop international guidelines for the management of kidney disease.

Importance: Undertaking evidence review that adheres to international standards is resource intensive and methodologically challenging. Improved collaboration between stakeholders may improve efficiency in guideline development. KDIGO has engaged CKT as the evidence review team for two guideline updates (KDIGO Glomerulonephritis Guideline; KDIGO Management of Blood Pressure in Chronic Kidney Disease Guideline) and one new guideline (KDIGO Management of Diabetes in Chronic Kidney Disease) using MAGICapp.

Processes: CKT with KDIGO developed protocols mapping guideline PICO questions to existing Cochrane reviews. SONG core outcomes were identified as critically important outcomes in guideline development. The CKT Register of Studies was searched to update reviews with input from review authors as necessary. Summary of findings tables were created using MAGICapp to inform the development of guideline recommendations.

Lessons Learnt: This collaboration had led to 26 Cochrane reviews updated, and reduced duplication of effort across organisations. Collaboration has ensured that guidelines incorporate core outcomes in the evidence review process. The use of Cochrane systematic reviews and MAGICapp has improved the transparency in translation from evidence to recommendations and will allow for rapid updating of guidelines.

Conclusion: Developing collaborations in guideline development has improved efficiency for all organisations and developed new opportunities for guidelines to adhere to international best-practice and include patient-important outcomes, with the potential for rapid updating.

Consumer Involvement in the development of KHA-CARI clinical practice guidelines for chronic kidney disease

Talia Gutman^1,2, Nicole Scholes-Robertson^1,2, Martin Howell^1,2, Pamela Lopez-Vargas^1,2, Karine E Manera^1,2, Jonathan C Craig^2,4, Brydee Johnston^1,2, David J Tunnicliffe^1,2, Suetonia Palmer³, Allison Tong^1,2

¹Sydney School of Public Health, The University of Sydney,²Centre for Kidney Research, The Children's Hospital at Westmead,³Department of Medicine, University of Otago,⁴College of Medicine and Public Health

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To describe consumer involvement in the development of KHA-CARI clinical practice guidelines for kidney disease.

Importance: It is widely acknowledged that consumers should be involved the development of clinical practice guidelines to ensure that guideline recommendations reflect their values, preferences and priorities. The KHA-CARI guideline strategies and approaches for consumer involvement have developed using best-practice frameworks.

Processes:

KHA-CARI incorporates multiple approaches across the guideline development process to ensure a broad range of consumer perspectives are integrated into guideline recommendations. A consumer advisory board provides strategic input to the Steering Committee (including consumer members) and oversees development of the workplan and prioritization of resources. Consumers are involved as guideline writers and develop consumer versions of the guideline to aid dissemination and implementation. Workshops are conducted with consumers (e.g. for early chronic kidney disease, polycystic kidney disease, kidney biopsy, infection in haemodialysis units), in parallel with the guideline writing process, to determine guideline topics/questions, scope, and outcomes to ensure their relevance to consumers.

Lessons Learnt: Involving consumers in priority setting has ensured that constrained resources focus on areas of importance to patients and carers. Workshops with consumers in development of the last four guidelines have altered their scope to include topics and outcomes not identified by clinicians (e.g. psychosocial support). Developing consumer versions of the guidelines led by consumers has resulted in guidelines that are useful and practical to support self-management.

Conclusion: Partnership with consumers in guideline development has led to guidelines that address topics of importance to all stakeholders.

Consumer Involvement in clinical practice guideline development: a framework for reporting

Talia Gutman^1,2, Nicole Scholes-Robertson^1,2, Shilpanjali Jesudason^4,5, Jonathan C Craig³, Allison Tong^1,2

¹Sydney School of Public Health, The University of Sydney,²Centre for Kidney Research, The Children's Hospital at Westmead,³College of Medicine and Public Health, Flinders University,⁴Central and Northern Adelaide Renal and Transplantation Service and Royal Adelaide Hospital,⁵Faculty of Health and Medical Sciences, University of Adelaide

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To develop a checklist for reporting consumer involvement in guideline development.

Background: Consumer involvement in guidelines is recommended, yet there is no consensus on best practice. Recommendations and methods vary across guideline organisations, and reporting of consumer involvement in guideline development is scarce.

Methods: We conducted a systematic search to identify national and international guideline handbooks/manuals. All sections on consumer involvement were imported into HyperRESEARCH software. We extracted and coded domains and items related to consumer involvement. These were synthesized into a checklist for reporting consumer involvement in guideline development and implementation.

Results: Twenty handbooks/manuals were included. We identified four domains of consumer involvement in guidelines and synthezised them into an 18-item checklist for reporting: who was involved? (type of consumers, number of consumers, recruitment strategy); what stages were consumers involved? (scoping, planning the review, assessing the evidence, writing recommendations, reviewing, implementing, disseminating); how were consumers involved? (informing, consulting, involving, partnership, consumer-led); and what support was provided? (training, education, financial).

Conclusion: The reporting framework will enable guideline developers to report consumer involvement in their methods, and guideline users to clearly understand these methods. Improving the transparency of reporting may inform strategies to strengthen consumer involvement in the development and implementation of guidelines.

Exploration of facilitators and barriers to the implementation of a guideline to reduce HIV-related stigma and discrimination in Ethiopian healthcare settings: a descriptive qualitative study

Garumma Tolu Feyissa^1,2, Craig Lockwood², Mirkuzie Woldie³, Zachary Munn²

¹Jimma University,²The Joanna Briggs Institute,³Department of Global Health and Population, T.H. Chan Harvard School of Public Health

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Background: In-depth exploration of local factors is needed to integrate a guideline into routine practice. This project aimed to assess contextual barriers and facilitators to the implementation of a guideline developed to reduce HIV-related stigma and discrimination (SAD) in the Ethiopian healthcare setting.

Methods: A descriptive qualitative research was conducted using a semi-structured interview guide informed by the Registered Nurses Association of Ontario (RNAO) framework. The interviews were conducted among a purposive sample of seven key informants from Jimma University and Jimma Zone HIV Prevention and Control Office. The interviews were transcribed, coded and analysed.

Results: Guideline attributes, provider-related factors and the presence of other health-related goals that complement SAD reduction programs were identified as factors that can potentially affect the implementation of the guideline. The presence of expert patients were identified as agents for guideline implementation, whilst regular health education programs in addition to initiatives related to service quality improvement, were identified as suitable platforms to assist with the implementation of this guideline. Study participants recommended that the guideline should be disseminated through multidisciplinary team (MDT) meetings, gate keepers such as opinion leaders and unit heads, one-to-five networks and mentorship programs, as well as training, workshops and posters.

Conclusion: This study identified that guideline implementation can make use of existing structures and pathways such as MDT meetings, service quality improvement initiatives, one-to-five networks, training and workshops. Teamwork and partnership with stakeholders should be strengthened to strengthen facilitators and tackle barriers related to the implementation of the guideline.

A Canadian Model for Providing High Quality Evidence for Health Decision-Maker Needs (The Strategy for Patient-Oriented Research (SPOR) Evidence Alliance)

Andrea Tricco¹, Wasifa Zarin, Sonia Thomas, Sharon Straus, Ahmed Abou-Setta, Anik Giguère, Christina Godfrey, Ian Graham, Lisa Keeping-Burke, Chantelle Lachance, Colin Macarthur, Janet Martin, Suzanne Morin, Kelly Mrklas, Caroline Porr, Amanda Ross-White, Sandra Small, Roberta Woodgate, Janet Yamada

¹Li Ka Shing Knowledge Institute

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To provide an overview of the SPOR Evidence Alliance (hereafter Alliance), including our mandate, governance structure, processes, and activity.

Background/Importance: The Alliance is a national initiative designed to promote evidence-informed healthcare and policy that is grounded in the principles of inclusion (all relevant stakeholders), diversity, integrated knowledge translation, co-creation of research, and transparency. The Alliance was funded by the Canadian Institutes of Health Research to support decision-maker needs.

Method: Decision-makers are invited to submit requests for support based on their organizational priorities via a web-based form (https://sporevidencealliance.ca/get-involved/make-a-query-suggestion-to-the-alliance/). We match a research query from a decision-maker to a method – whether it is a knowledge synthesis (e.g., rapid review, scoping review, systematic review), knowledge translation, implementation or clinical practice guideline that is required. Research teams affiliated with the Alliance co-create the research projects with the decision-makers; who are invited to participate in every aspect of the research, from inception to dissemination. Results are submitted for publication and an active knowledge translation strategy is devised which is tailored to specific knowledge users’ needs.

Results: There are >200 members of the Alliance, consisting predominantly of researchers, research trainees, patient partners, policy-makers, healthcare managers, and healthcare providers. More than 25 queries have been received from decision-makers. Preliminary data on the ongoing process evaluation of this initiative will be presented.

Conclusion: The Alliance is a new pan-Canadian initiative with the potential to improve patient care through national coordination of knowledge synthesis, knowledge translation, implementation and clinical practice guidelines development.

Reaching a shared understanding of shared decision-making

Clare Wohlgemuth¹, Katrina Penman¹, Chris Carmona¹, Monica Desai¹, Kay Nolan¹, Nichole Taske¹

¹National Institute for Health and Care Excellence

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To discuss NICE's experience of scoping its shared decision making guideline and additional steps undertaken at the scope development stage for overarching guidelines with wide-ranging audiences.

Background: NICE is supportive of embedding shared decision making into routine healthcare practice. Its guideline on shared decision making is due to publish in April 2021.

Methods: NICE's usual process involved discussion with early recruited committee members and engagement with stakeholders, through a workshop and formal consultation. In addition to usual process, scope development was facilitated by the consideration of existing conceptual frameworks for shared decision making to aid refinement.

Results: The conceptual frameworks aided discussion with stakeholders and early committee members which in turn helped to reveal important, and sometimes divergent, viewpoints. For example, the desirability and achievability of shared decision making and a lack of a shared lexicon. Key questions and inclusion criteria, detailed in the scope, were shaped by suggestions of the need for cultural change; overlap with other national initiatives; and concern about managing patient expectations in terms of treatment options. The scope was further influenced by commentary on the difficulties of adequately measuring the success of shared decision making, the need to agree from whose perspective success is measured and the value of proxy measures.

Conclusion: The use of conceptual frameworks to inform discussions with stakeholders and early committee members during the scoping stage was essential for identification and clarification of key issues. Engagement at this stage is particularly important for overarching guidelines to ensure relevance.

Rapid review of approaches for presenting evidence-based recommendations across health and non-health sectors

Katrina Penman¹, Catrin Austin¹, Victoria Axe¹, Rachel Archer¹, Toni Tan¹, Louise Foster¹, Amy Finnegan¹, Elizabeth Barrett¹, Jenny Craven¹, Andrea Heath¹, Kay Nolan¹, Nichole Taske¹, NICE Strength of Recommendations Working Group¹, Clare Wohlgemuth¹

¹National Institute for Health and Care Excellence

CS6 - Evidence for Consumers and End users, Riverbank Room 2, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To identify and describe approaches for presenting evidence-based recommendations from international guidance developers or policy makers to better understand the strengths and limitations of different approaches.

Background: NICE aims to ensure its recommendations are implementable and accessible to all users, across multiple sectors, are up to date, allow rapid integration of new drugs and technologies into existing pathways and are presented via a platform that can facilitate the rapid flow and integration of emerging evidence across the evidence ecosystem.

Methods: A web-based search identified guidance developers, policy makers, international working groups and organisations that produce evidence-based recommendations. The results were assessed against pre-specified criteria and a descriptive summary was produced.

Results: 216 guidance or organisations were identified across clinical and public health, social care, education, criminal justice, environment and transport sectors. A total of 47 types of guidance or organisations met the criteria across the 6 sectors.

Conclusion: The review considered platforms for presenting recommendations, what was presented, if and how the strength of recommendations or quality of evidence was presented and what informed the strength of recommendations judgment. Across the 47 included examples, 23 presented the strength of the recommendations, 13 indicating strong/weak recommendations, 10 using letters and 1 narratively. These judgements were informed by up to 13 different factors. This review showed there are many ways of presenting recommendations across organisations and sectors. All approaches have strengths and limitations and it is important for organisations to consider these to ensure recommendations are clear and transparent.

Use of GRADE in Australian clinical practice guidelines

Timothy Barker¹, Zachary Munn¹, Mafalda Dias^1,2, Kylie Porritt¹, Catalin Tufanaru^1,3, Cindy Stern¹, Edoardo Aromataris¹, Rick Wiechula⁴, Sue Brennan⁵, Holger Schünemann⁶

¹Joanna Briggs Institute, The University of Adelaide,²School of Pharmacy and Medical Sciences, University of South Australia,³Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University,⁴Adelaide Nursing School, Faculty of Health and Medical Sciences, The University of Adelaide,⁵Cochrane Australia, School of Public Health & Preventive Medicine, Monash University,⁶Department of Health Research Methods, Evidence, and Impact, Cochrane Canada, and McMaster GRADE Centre, McMaster University

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Aim: This project aimed to determine whether and to what extent GRADE methodology has been and is currently being used in Australian clinical practice guidelines (CPGs).

Background: CPGs are designed to assist both practitioner and patient in healthcare decision making. Guidelines are methodically prepared and based on peer-reviewed evidence that is often systematically reviewed. In 2000, the GRADE (Grading of Recommendations, Assessment, Development and Evaluation) working group developed a framework to grade the quality of evidence and the strength of recommendations in healthcare. This framework helps to frame the question, select and rate the importance of outcomes and to evaluate the evidence whilst considering it together with the values and preferences of patients and society. Due to these transparent and rigorous standards, GRADE been officially adopted by the NHMRC for the purposes of rating the quality of evidence that underpin each recommendation. However, despite this adoption and strong recommendation of GRADE by the NHMRC, it is unclear to what extent Australian CPGs currently follow GRADE methods.

Process: Guidelines available through the NHMRC CPGs Portal (whether NHMRC approved or not) since 2011 (year of NHMRC adoption) were assessed to determine if they appropriately utilised the GRADE methods, in the forming of their recommendations.

Lessons Learnt: We present our findings as to the extent at which GRADE has been employed appropriately for Australian CPGs and suggest methods in which GRADE compliance can be further adopted and maintained for future Australian CPGs

Full GRADE Guidelines for decision-making: The Brazilian Experience

Jessica Yumi Matuoka¹, Mariana Michel Barbosa¹, Jorgiany Eibedala Emerick², Haliton Alves de Oliveira Junior¹

¹Hospital Alemao Oswaldo Cruz,²National Committee for Health Technology Incorporations, Ministry of Health

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To explore the impact of applying high quality standards and following GRADE recommendations in health technology assessment and decision-making.

Background: One step in the development of clinical guidelines involves the evaluation of new technology for a given condition. It is based on the best evidence and considers all the relevant clinical outcomes and the setting in which the technology should be included.

Methods: Our experience with the recent guideline for Niemann-Pick Type C Disease (NPC) will be reported below. All steps to guarantee the methodological quality and new approaches to develop a report oriented to patients’ needs but focused on optimal resource allocation were described.

Results: Once identified that miglustat was potentially beneficial for the treatment of NPC, we conducted a systematic review including studies that evaluated its efficacy and safety. Systematic reviews and randomized clinical trials were prioritized. Because NPC is a rare disease, case series were also considered. For each outcome, the quality of evidence was evaluated following the GRADE approach. Economic evaluations (cost-effectiveness analyses and budget impact analyses) were conducted from the Brazilian healthcare system perspective. Next, all evidence and aspects related to the implementation of miglustat were summarized in a GRADE Evidence-to-Decision table. In this case, the marginal benefits, the very low quality of evidence, and high costs led decision makers to decide against its coverage.

Conclusion: The development of high-quality and decision-oriented protocols allows better informed decisions that are aligned with the Brazilian healthcare system values, patients’ needs, but focused on the system's sustainability.

National project of the Czech Republic: Trustworthy Clinical Practice Guidelines

Miloslav Klugar^1,2, Jitka Klugarova^1,2, Andrea Pokorna^1,2, Radim Líčeník^1,2, Miroslav Ryska¹, Ladislav Dušek³

¹Czech Health Research Council,²The Czech Republic Centre for Evidence-Based Healthcare: The Joanna Briggs Institute Centre of Excellence, Czech National Centre for Evidence-Based Healthcare and Knowledge Translation, Institute of Biostatistics and Analyses, Faculty of Medicine, Masaryk University,³Institute of Health Information and Statistics of the Czech Republic

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Aim: The main aim is to present the first results and main methodological concepts of the ongoing Czech national project “ Trustworthy Clinical Practice Guidelines”.

Background: The national guidelines project is led by the Czech Health Research Council (CHRC), partners are Ministry of Health (MH) and Institute of Health Information and Statistics (IHIS) of the Czech Republic with methodological support from members of the Czech National Centre for Evidence-Based Healthcare and Knowledge Translations (CEBHC-KT), which is an umbrella for (Cochrane Czech Republic, Masaryk University Grade Centre and the Czech JBI Centre of Excellence).

Method: Project started January 2018 and will last 5 years. During this time at least 40 trustworthy CPGs will be developed. Czech national guidelines methodology was developed by members of CEBHC-KT and is based on the GRADE approach. The methodology was tested on the first 5 pilot CPGs, which were completed by end of December 2018.

Results: First steps were to establish managing authorities including a Guarantee committee (GC) and an Appraisal (Methodological) committee (AMC). Members of GC are key stakeholders and policy makers in the Czech Republic. First 5 pilot guidelines were developed in cooperation with Czech Medical Professional Associations (CMPA) and were already successfully reviewed by CMPA, AMC, GC and currently is ongoing public review.

Conclusion: During the first year of the Czech National Guidelines project the following were developed:

• a national methodology was established,
• managing authorities for guidelines were established, and
• processes to generate topics and approval of guidelines.

Cooperation of all relevant stakeholders and involvement of CMPA is important.

Are guideline recommendations on diagnostic tests supported with evidence about the downstream consequences?

Mariska Tuut^1,2, Jako Burgers^1,3, Trudy van der Weijden¹, Miranda Langendam⁴

¹CAPHRI School, Maastricht University,²PROVA,³Dutch College of General Practitioners,⁴Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam UMC/University of Amsterdam

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To investigate the evidence-base of diagnostic recommendations, including the evidence beyond test accuracy, using the GRADE for diagnosis framework.

Background: Diagnostic recommendations in clinical practice guidelines (CPGs) should not only be based on evidence on diagnostic accuracy but also on test burden and consequences of test results for patient outcomes. GRADE for diagnosis provides a framework to evaluate these different types of evidence.

Method: We analyzed a purposeful sample of CPGs in English, German or Dutch with recommendations concerning two tests: CRP to diagnose pneumonia and colonoscopy to diagnose colorectal cancer. We assessed the bodies of evidence of diagnostic accuracy, test burden, natural course, disease management and link between test result and management.

Results: We included eight CPGs on CRP and eight on colonoscopy. Twelve CPGs evaluated diagnostic accuracy, of which six by a systematic review, three of these also determined the quality of the evidence. Few CPGs assessed test burden (n = 3), natural course (n = 2), disease management (n = 3) and linked evidence (n = 2). This was not done systematically and the quality of the evidence was not assessed, except for two CPGs evaluating the evidence quality of disease management. No CPG assessed all steps of the test-treatment pathway systematically. Ten CPGs stated to use the GRADE approach.

Conclusion: Our review suggests that CPGs rarely assess the quality of the evidence of test-treatment strategies systematically. Test consequences are generally ignored despite using the GRADE approach. Further research is needed to reduce the gap between the recommended approach and practice of CPG development.

Sickle cell guidelines by the American Society of Hematology: Methodology, Challenges and Innovations

Reem Mustafa¹, Elie Akl¹⁰, Eddy Lang¹¹, Jeorg Meerrpohl¹², Robert I. Liem⁵, Sophie M. Lanzkron⁷, Michael R. DeBaun³, John F. Tisdale⁴, Amanda M. Brandow⁶, Stella T. Chou⁸, Star Webb, M. Hassan Murad²

¹University of Kansas Medical Center,²Evidence-based Practice Center, Mayo Clinic,³Vanderbilt-Meharry Center for Excellence in Sickle Cell Disease,⁴National Heart, Lung, and Blood Institute,⁵Ann & Robert H. Lurie Children's Hospital of Chicago,⁶Medical College of Wisconsin,⁷The Johns Hopkins Hospital,⁸The Children's Hospital of Philadelphia,⁹The American Society of Hematology,¹⁰American University of Beirut,¹¹University of Calgary,¹²University Medical Center Freiburg

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Background: The American Society of Hematology is developing Sickle cell disease (SCD) clinical practice guidelines. We describe several unique challenges faced by the guideline panels and the specific innovations to address them; including working with indirect and non-comparative evidence to assess screening questions.

Process: To allow for consistent decisions among the different screening questions, we developed a framework that supported developing recommendations based on association studies. The framework was informed by the GRADE diagnosis Evidence to Decision framework, the WHO and the USPSTF criteria and a manual for screening tests.

Results: Our framework to support decisions when screening is justified includes the following criteria: The condition should be an important health problem (either sufficiently prevalent or it has severe consequences); Individuals with a positive screening test would get a different management than those with a negative test; The condition being screened for should have a natural history that is understood and that can be recognized latent or at an early symptomatic stage; There should be an effective management for the condition available that improves outcomes if administered earlier than when the condition is clinically apparent; Improvement in outcomes based on management according to screening results outweigh harms of screening; High or moderate certainty evidence for a sufficient accuracy of the test; screening should be cost effective, acceptable to patients and feasible.

Conclusion: Using an explicit framework when direct evidence is lacking is critical for transparency and to show guideline users how decisions were made.

Adoption of the GRADE Approach by Guideline Developers in the United States: An Assessment of Published Guidance Documents

Philipp Dahm^1,2, Shahnaz Sultan¹, Reem Mustafa³, Rebecca Morgan⁴, Hassan Murad⁵, Ygnve Falck-Ytter⁶.

¹Minneapolis VAMC,²St. John's Hospital,³University of Kansas, Department of Internal Medicine, Division of Nephrology and Hypertension,⁴McMaster University, Department of Health Research Methods, Evidence and Impact,⁵Mayo Clinic, Evidence Based Practice Center,⁶Case Western Reserve University, Division of Gastroenterology and Liver Disease

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Objective:

Aim: We performed this study to assess whether clinical practice guidelines (CPG) published by organizations based in the USA comply with published criteria for the use of the GRADE approach.

Method: We performed a cross-sectional study of all CPG that indicated the use of GRADE, were published between 2011 and 2018 and listed in the National Guidelines Clearinghouse. Two independent members screened and abstracted data independently and in duplicate using a pilot-tested form based on published GRADE criteria.

Results: We included 67 CPG documents from 44 of 135 (32.6%) United States-based organizations that indicated the use of the GRADE approach. The majority (89.6%; 60/67) of guidelines defined the certainty of evidence consistent with GRADE but only approximately one in ten (10.4%; 7/67) explicitly reported consideration of all eight criteria to assess the certainty in the evidence for rating down and up. All but one guideline (98.5%; 66/67) described the overall certainty of evidence using a 3 or 4-tiered system. A majority of guidelines (53.7%; 36/67) provided a summary of the evidence, described explicit consideration of all four central domains (53.7%; 36/67) and rated the strength of recommendation consistent with GRADE (53.7%; 36/67). When comparing CPG published in 2015–2018 versus 2011–2014, rates of appropriate use were higher in more recent CPG for nearly all criteria.

Conclusion: Adherence to published criteria is inconsistent although improving over time. As uptake of the GRADE approach increases in the US, continued efforts to train guideline methodologists and panel members are important.

Piloting a new approach to assessing GRADE imprecision in a guideline development context

Katrina Penman¹, Joshua Pink¹, Kathryn Hopkins¹, Marie Harrisingh¹, Yolanda Martinez¹, Toni Tan¹, Nichole Taske¹, Clare Wohlgemuth¹

¹National Institute for Health and Care Excellence

CS6 - Using GRADE, Hall C - Plenary, November 1, 2019, 3:30 PM - 5:00 PM

Aim: To describe NICE's experience of exploring and piloting an alternative approach to assessing minimally important differences (MIDs) and imprecision in guidelines.

Background: When judging imprecision GRADE recommends developers should consider whether confidence intervals cross a ‘clinical decision threshold’. Developers often use MIDs to define clinical decision thresholds, however there are practical and conceptual difficulties associated with using MIDs to assess imprecision. Currently, default MID values are often used to rate imprecision in NICE guidelines, but these are poorly evidenced and unlikely to be valid across a range of disease areas and outcomes.

Methods: A pilot was undertaken in the depression in children guideline update to investigate the impact of not including a rating of imprecision in the GRADE profile, but instead including a discussion of the precision of effect estimates and the impact on decision making in the committee discussion section of the evidence review.

Results: The pilot showed the alternative approach was easy to understand for both guideline developers and the guideline committee. The guideline committee, with support from the developer, considered imprecision when discussing the evidence and writing recommendations, leading to a transparent discussion section within the guideline.

Conclusion: This pilot showed there is a viable alternative approach to using default MID values to assess imprecision in GRADE, NICE is considering exploring the application of this approach in further guidelines.

Supporting users: NICE antimicrobial prescribing guidelines

Fiona Glen¹, Johanna Hulme¹, Ruth Garnett¹

¹National Institute for Health and Care Excellence

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To describe a novel approach for presenting national antimicrobial prescribing guidance.

Background/Importance: To support the appropriate use of antimicrobials, NICE with Public Health England, produces antimicrobial prescribing guidelines (APG) for managing specific common infections. These guidelines provide recommendations for all people (except those under 72 hours of age) in all care settings and replace information previously developed by Public Health England (PHE).

Following stakeholder feedback, different innovative approaches have been used to present the guidelines using a layered approach. This includes a visual summary, an online platform that houses all national guidance on antimicrobial prescribing information and a longer evidence review. Users can therefore navigate to one place to access guidance and have the option to drill down for further information.

Methods/Processes: With input from stakeholders, NICE worked with PHE to design one platform that provides a summary of all national guidance on antimicrobial prescribing for common infections.

Results/Lessons learnt: We will present findings on the uptake and use of the platform plus any challenges in implementing the guidance at a local level.

Conclusion: Our approach allows users to access the information they need, in their preferred format, quickly and easily to support shared decision-making with patients.

Expanding nurses scope of practice with the support of nursing clinical guidelines: The national development process in the Republic of Kazakhstan

MSc, RN Eveliina Kivinen¹, Zaure Baigozhina², Johanna Heikkilä³, Ayagoz Umbetzhanova⁴

¹Lahti University of Applied Sciences,²Republican Center for Health Development,³JAMK University of Applied Sciences,⁴Astana Medical University

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: In the Republic of Kazakhstan (RK), the aim of nursing clinical guideline development is to adapt and implement clinical guidelines in order to ensure safe expanding of nurse's scope of practice by assisting nurses to make informed, reasoned and evidence-based decisions in clinical practice.

Background: Globally there is a need to facilitate evidence-based practice in nursing to enhance the contribution of nurses in health care. In RK, this is identified in the national comprehensive nursing development plan. The guideline development was launched with the support of Finnish experts of nursing from Universities of Applied Sciences.

Process: In 2018–2020 15 clinical guidelines will be adapted by using a systematic approach described in manual providing instructions on the technical and procedural aspects. A trained Research Committee is responsible for ensuring that guidelines address important health questions and are relevant in local environment. In addition to adaptation, the work includes the preparation of implementation material and trainer education.

Results: Development work started in 2018, and significant results have been achieved. Four guidelines have progressed to approval and five are in review process. Implementation materials for two guidelines have been developed and the first trainer education has been organized.

Conclusion: The value of guidelines assisting nurses to make informed, reasoned and evidence-based decisions is important. National nursing clinical guidelines are developed to promote best practice and improve the quality and safety of nursing care across the Kazakhstani healthcare system.

Mapping clinical barriers and facilitators to implementation in low-to-middle income countries (LMICs)

Tania Marin¹, Ashley Whitehorn¹, Aye Aye Gyi¹

¹Joanna Briggs Institute

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: Compare and contrast barriers identified in implementation projects from low-to-middle income countries (LMICs), using the Donabedian SPO framework and EPOC classification for interventions for implementation of practice change.

Background/Importance: Mapping barriers and strategies to implementation of evidence-based healthcare, then adapting these to the localized political, cultural and organizational contexts, facilitates appropriate implementation. Developing an understanding of the barriers in LMICs is especially important as many of the experiences are likely to be shared across similar settings.

Methods/Process: Published implementation reports that used clinical audit methods in LMICs were included. Data were extracted against the Donabedian framework and implementation strategies mapped to the Effective Practice and Organisation of Care (EPOC) taxonomy.

Results/Lessons Learnt: Sixty implementation reports from Asia (65.0%), Africa (30%), and South America (5.0%) were identified. Projects involved both multi- and single-disciplinary teams in acute, outpatient and community settings. A total of 279 individual barriers were identified; 42.3% structural, 55.9% process and 1.84% outcome. Barriers included lack of knowledge (22.9%), lack of resources (24%), increased workload (8.2%). Implementation strategies (n = 567) mapped to 11 EPOC categories and 42 EPOC sub-categories.

Conclusion: This study provides a summary of the most common barriers and implementation strategies that may be useful in the planning of future implementation projects in LMICs. Data showed a very high number of common barriers when implementing evidence based healthcare.

The EBPracticeNet Africa: Promoting healthcare in rural settings through the use of locally contextualized and foreign clinical practice guidelines

Loveline Lum Niba¹, Mbah Okwen¹, Vander Stichele³, Olabisi Oduwole², Emmanuel Kamga¹, Euphrasia Ebai-Atuh¹, Chia Olivette¹, Uwambaza Clementine⁴, Tiwa Serge Valentin^1,5

¹Effective Basic Services, Africa, Cameroon,²Cochrane Nigeria,³University of Ghent,⁴Effective Basic Services, Rwanda,⁵Ministry of Public Health

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To develop and/or contextualize guidelines for some of the poverty related diseases to guide point of care decision making for clinicians in rural settings for Middle Africa.

Background/Importance: In Africa evidence-based practice in primary care services by clinicians is a major challenge due to lack of access to research evidence needed to inform policy and practice. This has impacted negatively on the quality of care and health systems performance.

Methods/Processes: We have successfully set up a web-based platform http://www.ebmafrica.net which provides national, WHO and foreign guidelines and JBI evidence-based criteria with adaptations for Nigeria, Rwanda and Cameroon targeting poverty related diseases (malaria, TB, HIV, soil transmitted helminthes and hypertension). We worked with G-I-N Africa to collaborate with clinicians using WhatsApp platforms within the network. We reviewed evidence recommendations for diagnosis, treatment and prevention and developed and/or contextualized guidelines by country. Doctors were provided accounts to freely access evidence within the portal to help improve their practice. We partnered with Cochrane Nigeria, Cochrane Belgium, University of Ghent, iScientia Belgium and a consumer referee organisation in developing these guidelines.

Results: We have used 149 systematic reviews from Cochrane and JBI networks to develop 10 guidelines which have been contextualized for diseases with the greatest burden in African underserved communities. We are currently working with 10 clinicians and 5 nurses across 4 countries (Cameroon, Rwanda, Nigeria and Ghana) as guidelines editors in Africa.

Conclusion: Development of trustworthy evidence-based guidance and support will significantly improve health outcomes especially for rural communities in Africa.

Contextualising guidelines to support large scale health and social care system redesign

Joanne McCormack¹, Jane Moore¹, Gillian Leng¹

¹National Institute for Health and Care Excellence

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To develop themed resource packages of guidelines, to support priorities of integrated service transformation.

Background: The integration of health and care involves streamlining services, improving health and maintaining sustainable finances. Traditionally, the NICE guidelines portfolio has focussed on single conditions and not tended to focus on transformation and integration agendas. NICE's implementation strategy supported development of resource packs aligned to system redesigns so that new emerging systems would be evidence-based, effective and work for people using them.

Method: A thematic review of priorities from integrated service re-design plans was completed and ranked. A suite of web-based resources for 10 topics was developed containing relevant Guidelines, tools and resources extracted from NICE's portfolio addressing these priorities. Reference was made to other accredited guidelines. The web-based resources and accompanying presentations, were based on field team intelligence, utilised editorial input and were externally quality-assured.

Results: The topic specific resources are now freely available on the NICE website and promoted by the NICE Field Team. Website hits are monitored together with examples of use. Having a link on another public body's website significantly increased hits, and feedback has been very positive to date.

Conclusions: These packages support systems with the implementation element of the evidence ecosystem, underpinning discussions on pathway redesign and assisting decision-making. By listening to the system and user's needs, repackaging guidelines in ways helpful to new emerging systems, maximised applicability and user engagement. By using a common language, services can be united around a shared aim, ensuring guidelines are fit for purpose.

Modelling Treatment Pathways: a solution to NICE's guidance jigsaw?

Bhash Naidoo¹, Gabriel Rogers¹, Ross Maconachie¹

¹National Institute for Health and Care Excellence

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To describe the technical, procedural and structural difficulties of integrating NICE Technology Appraisals and Clinical Guidelines into single clinical pathways.

Background: There is an aspiration to present all NICE guidance in an integrated and coherent fashion through a clinical pathway approach, particularly with respect to drug/treatment sequencing. This applies to both relatively short-term clinical choices, such as in what order should a range of treatments be tried for a given goal, or long-term care pathways, such as which modes of management are most appropriate as a disease progresses.

Process: We will outline an idealised technical solution for modelling of treatment pathways, and then describe how technical, procedural and structural issues make the implementation of such a solution challenging.

Lessons Learnt: The benefits of modelling whole clinical pathways to find the optimal sequence of treatments, in terms of both costs and benefits, has been previously demonstrated by Tappenden et al. Although there are non-trivial technical difficulties in undertaking such analyses, these are not insurmountable.

However, practical, structural and statutory issues may present more of a barrier to progress. Issues that make NICE Technology Appraisals and Clinical Guidelines different and may make them harder to align are:

• Timeliness and timelines
• Ownership of technical work
• Contestability
• Confidentiality, particularly regarding price
• NHS funding direction

Conclusions: Differences in NICE processes and status within the NHS between NICE Technology Appraisals and Clinical Guidelines may present more of a barrier to integrating guidance through single clinical pathways than the technical challenges.

Prognostic Risk Tools: How to Assess Usefulness for Clinical Decision Making and Guideline Development

Jacqui Real¹

¹National Guideline Centre, Royal College of Physicians

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To investigate the usefulness of prognostic risk tool reviews and commonly reported risk tool statistics, in the context of clinical decision-making and guideline development.

Background/Importance: Commonly reported statistical measures of discrimination for prognostic risk tools may not be appropriate for clinical decision-making. Reviews of such tools may be of limited value in guideline development.

Methods/Process: The NICE chronic heart failure guideline update included a review of validated mortality risk tools, to identify those that may support palliative care decisions. The statistical measures reported, and the overall value of the review, were considered.

Results/Lessons Learnt: The review identified a substantial literature on heart failure prognostic risk tools. Sensitivity, specificity, and negative and positive predictive values at clinically relevant risk thresholds were rarely reported. When sensitivity and specificity were reported, even tools with reasonable discrimination according to area under the curve performed poorly. The guideline committee recommended that prognostic risk tools not be used to inform referral to palliative care services. The review did, however, highlight the unpredictable trajectory of heart failure and the poor predictive power of perceived risk factors, and show the importance of clinical judgement in assessing palliative care needs.

Conclusion: There is a gap between the statistics needed to assess the utility of prognostic risk tools, and the statistics typically reported in the literature. The likely value of conducting a prognostic risk tool review for guideline development should be carefully considered given the lag between the methods and theory, and the available evidence base.

Identification and use of a broader range of evidence to inform scoping of NICE guidelines

Clare Wohlgemuth¹, Katrina Penman¹, Louise Foster¹, Catrin Austin¹, Jean Bennie¹, Toni Tan¹, Nichole Taske¹

¹National Institute for Health and Care Excellence

CS7 - Guideline Contextualisation and Use, Riverbank Room 2, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To describe NICE's experience of using a broader range of evidence (sometimes called real world evidence) to inform scope development.

Background: Previously scopes were informed by more traditional evidence sources such as systematic reviews and RCTs. A broader range of evidence is now routinely used to develop scopes for NICE guidelines, in both new guidelines and guideline updates.

Methods: NICE has conducted pilots to consider the benefits and limitations of considering a broader range of evidence in the development of guideline scopes. This evidence can include audits, statistics, patient registries, and information from patient experience and support forums. Following successful pilots, a broader evidence search is now routinely undertaken to inform scope development.

Results: A broad range of evidence can be valuable in setting out the context and need for guidelines, as well as refining the populations, settings and the key issues that will be covered. Furthermore, this type of evidence can aid the identification of potential equality issues that will need considering during the guideline development process. Use of a broader range of evidence may also allow for new questions to be included that may have been previously deprioritised due to a lack of traditional evidence, or for refining of included questions.

Conclusion: The use of a broader range of evidence can be beneficial and should be considered at the scoping stage, as well as during guideline development, where appropriate, to ensure high quality guidelines.

Assessment and management of women with Heavy Menstrual Periods At Kenyatta National Hospital-A best practice implementation project.

Lydia Akumu¹, Mrs. Lucylynn Lizarondo², Philomena Owende¹, Stanley Ratemo¹

¹The University of Nairobi/Kenyatta National Hospital,²The Joanna Briggs Institute

CS7 - Rapid Presentation Session, Riverbank Room 3, November 2, 2019, 9:00 AM - 10:30 AM

Objective: Promoting evidence-based practice in the assessment and management of women with HMB attending the gynaecology outpatient clinic at Kenyatta National Hospital and thereby improve patient outcomes and resource utilization.

Introduction: Heavy menstrual bleeding (HMB) is a significant public health problem that has a notable impact on a woman's physical, psychological and social functioning, leading to a reduction in their quality of life. Women in developing countries largely suffer in silence due to cultural barriers.

Methods: This evidence implementation project was carried out at the gynaecology outpatient clinic of the largest referral facility in Kenya. The Joanna Briggs Institute (JBI) Practical Application of Clinical Evidence System and Getting Research into Practice audit and feedback tool were used. A baseline audit was conducted using a sample of 40 patient files, this was followed by interventions such as trainings, developing of visual tools and updating of protocols. A follow up audit was then conducted four months post intervention.

Results: Baseline audit results demonstrated varying results in each of the 9 criteria selected. Four of the criteria recorded less than 51% compliance, with two of these recording 0% compliance. Post intervention results showed a remarkable increase in most of the audit criteria selected, with 8 of these having compliance levels of 90% or more.

Conclusion: The implementation project served the dual purpose of enlightening health care workers on best practices and educating patients on what they need to know about their condition. This led to an overall improvement in the management of HMB.

Global trends in the approach to evidence-based healthcare: a scoping review

Samia Alhabib¹, Yasser Amer, Craig Lockwood

¹King Abdullah Bin Abdulaziz University Hospital

CS7 - Rapid Presentation Session, Riverbank Room 3, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To map global trends in Evidence-based healthcare (EBHC) approaches in order to provide a roadmap informing the future of national EBHC strategies in Saudi Arabia and to facilitate collaboration with international bodies

Background: Understanding world-wide EBHCs approaches and trends in pursuing centres would inform and guide local strategies when establishing EBHC centre at a national level.

Methods: This scoping review used the PRISMA Extension for Scoping Reviews Checklist. We included only Centres of EBHC. We searched websites of all identified EBHC centres and grey literature sources. Two reviewers independently screened search results for inclusion. Identified data were extracted and thematically analysed based on the four EBHC approaches from the JBI model: evidence generation (EG), synthesis (ES), transfer (ET), and implementation (EI) and the translational research levels (T1–4).

Results: Preliminary results from twenty EBHC centres revealed that 18(90%) were from high-income countries, 25% were affiliated to Universities, 55% were independent bodies. All centres were not related to specific clinical disciplines, provide ET and translational research level T3, and have specific EBHC deliverables (systematic reviews, evidence-based clinical guidelines, training and education etc.). Majority of the included centres (65%) have an international EBHC scope of services, 40% had a formal EBHC model, and only 20% provide public EBHC services and/or resources free of charge.

Conclusion: Mapping the ecosystem of EBHC in terms of evidence generation, synthesis, transfer, implementation and translation approaches at a global level will guide EBHC end-users and novice EBHC centres and encourage efficient networking, collaborations and exchange of experience.

Sifting through the morass of literature reviews: disentangling the need for further qualitative evidence synthesis in a particular substantive area

Sandra Small¹, Catherine de Boer², Michelle Swab³

¹Faculty of Nursing, Memorial University of Newfoundland,²School of Social Work, Memorial University of Newfoundland,³Health Sciences Library, Memorial University of Newfoundland

CS7 - Rapid Presentation Session, Riverbank Room 3, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To examine the implications of the growing number of published literature reviews for determining the need for further evidence synthesis in a particular substantive area.

Background: To justify the need for a systematic review (SR), reviewers are required to indicate that a search of pertinent databases has been conducted to determine whether SRs already exist on the topic of interest and to indicate how the planned SR differs from any preexisting SRs. Making decisions about the relevance of SRs is difficult when such reviews only partially address the phenomenon or population of interest, are peripherally, rather than centrally important to the phenomenon of interest, lack robust methods, or have important limitations. Determining the need for a qualitative SR is especially challenging when a large number of possibly relevant SRs are available.

Methods: A qualitative SR about experience with disability policy and labor market engagement will be used to illustrate challenges encountered in sorting through a host of preexisting literature reviews.

Lessons Learnt: With the ever-increasing proliferation of SRs, reviewers need to take great care to determine what would be redundant, what would be unique, and what would be valuable when examining evidence to inform the derivation of their review questions.

Conclusion: Guidelines are warranted to assist reviewers in knowing what constitutes redundant versus the need for new synthesized evidence and in knowing how to address the issue of the quality of existing SRs. Guidelines are particularly important for novice reviewers and could help eliminate the over-production of unhelpful SRs.

Study on the relationship between the standard of diagnosis and treatment of common and frequent diseases in primary health institutions

Jianjian Wang^1,2,3, Yanfang Ma^1,2,3, Xufei Luo^1,2,3, Shuaixiang Zhao⁴, Meng Lv⁴, Xiaohui Wang⁴, Yaolong Chen^1,2,3

¹Evidence-Based Medicine Center, Lanzhou University,²Chinese GRADE Center,³WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,⁴School of Public Health, Lanzhou University

CS7 - Rapid Presentation Session, Riverbank Room 3, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To explore a method to identify diagnosis and treatment standards of common and frequent diseases in primary health institutions, and develop a virtual patient-oriented tool based on smart phones to evaluate the quality of service.

Background/Importance: There are few tools for evaluating the quality of primary health care services at present, mainly the Primary Care Assessment Tool (PCAT) of Johns Hopkins University.

Methods/Process: All literatures related to specific evaluations using standardized patients/simulated patients were included. Then refers to the WHO Handbook for Guideline Development, and uses the Delphi method to explore its specific processes in accordance with the principles of evidence-based medicine.

Results/Lessons Learnt: A total of 11 related literatures were included, and proposed five key steps: (1) set up a multidisciplinary team; (2) select and evaluate relevant literatures; (3) extract medical information to form a basic item pool; (4) product consensus of clinical experts on items; (5) pre-test to determine the final items.

Conclusion: Considering the complexity of clinical practice, it is recommended to use the Delphi method to conduct consensus on the target disease diagnosis and treatment standards, and classify all the items according to the importance, necessity and feasibility. What's more, set up a national primary medical assessment team, according to the actual conditions to further modify and improve the key items.

An analysis of extensions of reporting guidelines in the equator network

Nan Yang^1,2, Jingyi Zhang^1,2,3, Qiuyun Wang⁴, Xiaoqin Wang^1,2, Yaolong Chen^1,2, Kehu Yang^1,2

¹Evidence-based Medicine Center of Lanzhou University,²WHO Collaborating Centre for Guideline Implementation And Knowledge Translation,³School of Public Health, Lanzhou University,⁴Department of Hematology, the First People's Hospital of Taicang City

CS7 - Rapid Presentation Session, Riverbank Room 3, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To analyse the current situation of extensions of reporting guidelines.

Background: Some reporting guidelines need to be extended by items for reporting specific types of research, components of the study, and clinical fields. The EQUATOR group included several such extensions of reporting guidelines.

Methods: We searched the extensions of reporting guidelines collected in the EQUATOR Network. This database includes almost all reporting guidelines worldwide.

Result: We included 54 extensions of six reporting guidelines from EQUATOR. Twenty-five (46%) extensions were adapted from the Consolidated Standards of Reporting Trials (CONSORT) statement and seventeen (31%) were adapted from the Strengthening the Reporting of Observational studies in Epidemiology (STROBE). The extensions of reporting guidelines were published from 2004 to 2015; Eleven (20%) were published in 2017. Eleven (20%) extensions added new items on top of the checklist items from the primary reporting guidelines without modifing any existing items. Ten (19%) extensions modified the existing items without adding any new items. The remaining 33 (61%) extensions both modified the existing items and added new items. There was an average of 13 items including sub-items, and six items including sub-items in each extension. Fourteen (26%) extensions focused on specific components of the study, such as abstracts, outcomes, harms, or discussions. Twenty-four (44%) extensions were developed for specific clinical areas, such as complementary and alternative medicine, infectious diseases, or surgery.

Conclusion: There were growing number of extensions of reporting guidelines. The more adaptive strategy is needed for extending existing guidelines.

How many systematic reviews that fail to meet the reporting requirements of PRISMA are being published?

Shelley De Kock¹, Lisa Stirk¹, Steven Duffy¹, Kate Misso¹, Janine Ross¹, Caro Noake¹, Jos Kleijnen^1,2

¹Kleijnen Systematic Reviews Ltd,²Maastricht University

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To analyse a sample of systematic reviews (SRs) to assess compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) #7 (Information sources) and #8 (Search) checklist.

Background: PRISMA was developed to improve transparency of SR methods to ensure certainty of SR findings. Previous research indicates that SRs are not always complying with #7 and #8 of the PRISMA 2009 checklist.

Method: A sample (n = 505) of SRs published in 2016, were selected from KSR Evidence, a database of SRs with a subset of ROBIS-assessed SRs, and analysed to assess:

• How many SRs claiming PRISMA compliance meet #7 and #8?
• How many SRs published in journals requiring PRISMA meet #7 and #8?
• How many SRs in total comply with PRISMA #7 and #8?
• Is there a correlation between how well-reported an SR is and its risk of bias in Domain 2 (D2) (Identification and Selection of Studies) of the ROBIS tool?

Results:

• 10% of SRs in our sample were low risk of bias for D2 and compliant with PRISMA #7 and #8
• 71% of sampled SRs failed to comply with #7 and #8
• 70% of SRs claiming to follow PRISMA were non-compliant in #7 and #8
• 76% of SRs failed to meet #7 and #8 were published in journals which require PRISMA

Conclusion: PRISMA standards are not being routinely met. To ensure SRs meet PRISMA standards, SRs should be peer-reviewed by information specialists experienced in SR searching methods.

Which bibliographic databases provide the best return for your time spent screening citations?

Skye Newton¹, Klara Salinger¹, Tracy Merlin¹

¹Adelaide Health Technology Assessment, University of Adelaide

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To describe the optimal databases for use in systematic reviews.

Background: Pressure to make the systematic review process more rapid, without reducing the rigour of findings, is meaning that the process of performing the review needs to be more efficient. One of the most time consuming elements of performing systematic reviews is the screening of citations. One influential factor determining the volume of citations (and consequently the time required to cull), is the number of bibliographic databases searched.

Methods: The value of different databases (PubMed, Embase, the Cochrane library, Web of Science, Scopus, Cinahl, PsycLit), was retrospectively determined from a dataset of completed systematic reviews. Value was defined by the recall (no. of citations retrieved from each database), precision (the proportion of included studies the database identified), number needed to read (to find one included study), and the number of unique citations. These measures per database were assessed for seven systematic reviews performed on a variety of topics including genetic tests, therapeutic interventions as well as an aetiology question.

Results: The combination of Embase and PubMed identified the vast majority of included studies. Little additional value was seen from searching the Cochrane Library, Current Contents, Web of Science or Scopus. For an aetiology question on alcohol and mental health, Cinahl and PsycLit both provided unique citations.

Conclusion: A rigorous systematic review can be performed using a limited number of databases, allowing more rapid systematic reviews, although for topics other than health interventions, additional databases should still be considered.

Predatory journals and publishers: Validation of a published checklist for evaluating research papers for inclusion in scoping reviews.

Michelle Block^1,2, Marsha Bennett^3,4, Mr. Adam Cooper^5,6, Robin Christian^7,8, Christina Godfrey^9,10, Patrick Palmieri^11,12,13

¹Purdue University Northwest, College of Nursing,²Indiana Center for Evidence Based Practice: A Joanna Briggs Institute Affiliated Group,³Louisiana State University Health, School of Nursing,⁴Louisiana Center for Promotion of Optimal Health Outcomes: A Joanna Briggs Institute Center of Excellence,⁵University of California San Francisco Health,⁶University of California San Francisco Centre for Evidence Synthesis & Implementation: A Joanna Briggs Institute Center of Excellence,⁷University of Mississippi, School of Nursing,⁸Mississippi Centre of Evidence-Based Practice: A Joanna Briggs Institute Centre of Excellence,⁹Queen's University, School of Nursing,¹⁰Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence,¹¹Universidad Maria Auxiliadora, Center for Health Sciences Research,¹²Universidad Norbert Wiener, School of Nursing,¹³South America EBHC: A Joanna Briggs Institute Affiliated Group

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: This presentation will: 1) identify the prevalence of predatory journals in 10 scoping reviews published in the Joanna Briggs Database (JBD); 2) explore agreement between the results of a published journal evaluation tool (JET) with Beall's updated predatory list of publishers and journals (BPL); 3) provide lessons learned about the review with the tool and list; and 4) recommend an evaluation process for the scoping review method.

Background: Increasing prevalence of predatory publishers and journals results in more studies published in these sources that can adversely impact data utilization and the scientific record. While systematic reviews include an appraisal to screen for quality and rigor, scoping reviews lack an appraisal process as they aim to examine the breadth of the literature on a given concept, map out key concepts, draw conclusions about conceptual definitions, and/or identify gaps in the literature.

Methods: Using the JET developed at Loyola Marymount University, the authors evaluated 204 journals from studies included in 10 scoping reviews published in the JBD. In addition, journals were cross referenced BPL.

Results/Lessons: Through the review process predatory journals were identified. The review evidences the JET lacks sensitivity and specificity. There were also deficiencies in the journal citations and abbreviations, problems with publisher website clarity, quality assessments that are not contextualized, and confusion between indexing and abstracting.

Conclusion: There needs to be a robust method to evaluate journals, including establishing methodological criteria for predatory evaluation for scoping reviews, and the development of an evidence-based standardized tool for evaluating journals.

The Rapid Review: delineating potential risks to trustworthy evidence

Caroline Porr¹

¹Memorial University of Newfoundland

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To enhance awareness of the potential risks to the trustworthiness of evidence when adopting a rapid review approach to knowledge synthesis.

Importance: Rapid reviews are becoming increasingly popular among decision makers within the health sector who must respond quickly to changes in legislation, new policies, medical advances, and forces affecting population health and well-being. However, are rapid reviews sufficient? Is the process robust enough to ensure trustworthy evidence? Who really are the arbiters of the quality of the evidence? The reviewers? The knowledge users?

Process: The approach, methods, and processes of the rapid review and the systematic review will be delineated paying special attention to bias, rigor and other elements of trustworthiness.

Lessons Learnt: Based on the results of a recent rapid review with knowledge users in a Canadian health care context, there are important measures that must be undertaken to mitigate risk to the trustworthiness of the evidence. Moreover, steps must be put in place from the outset to clarify expectations regarding the deliverables and accurate translation of the results.

Conclusion: Reviewers must anticipate potential risks to trustworthy evidence when undertaking a rapid review.

How to complete a full systematic review in under 2 weeks: processes, barriers and facilitators

Justin Clark¹, Paul Glasziou¹, Chris Del Mar¹, Alexandra Bannach-Brown¹, Paulina Stehlik¹, Anna Scott¹

¹CREBP, Bond University

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To identify barriers and enablers to decrease the time required to complete a full systematic review to under 2 weeks.

Background: Systematic reviews are time- and resource-intensive, requiring 40–50 weeks from protocol registration to submission.

Method: We completed a systematic review of evidence for an intervention question, by integrating 3 key components: 4 experienced systematic reviewers (two researcher clinicians, an information specialist, an epidemiologist), Systematic Review Automation (SRA) tools, and time was blocked off solely for this project. We recorded the time required to complete each step of a systematic review and identified barriers and facilitators. The final outcome was a completed systematic review of sufficient quality to circulate externally for feedback.

Results: 64 person-hours across 8 ½ workdays (11 calendar days) were required to complete a moderately sized systematic review (1,381 articles screened, 8 included studies, 4 meta-analyses). Barriers included: website downtime, deviating from normal processes, lack of clinical knowledge by screeners, environmental disruptions. Key facilitators included: expertise in systematic review methods, pre-existing familiarity with the Systematic Review Automation tools and their capabilities, blocked off time to focus on this project specifically, real-time availability of clinical and methodological expertise, rapid response time from authors of included studies.

Conclusions: A small and experienced systematic review team using Systematic Review Automation tools and time dedicated solely to the project can complete a moderately-sized full systematic review in under 2 work weeks.

The PRISMA Extension to Scoping Reviews (PRISMA-ScR)

Andrea Tricco¹, Marsha Bennett, Christina Godfrey, Hanan Khalil, Patricia McInerney, Zachary Munn, Micah Peters

¹Li Ka Shing Knowledge Institute

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To provide an overview of the PRISMA-ScR, including how it was developed and how it differs from PRISMA.

Background: Scoping reviews are frequently requested by decision-makers to provide a snapshot of the literature, identify research gaps, and identify areas for future research. In 2018, the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement was extended to scoping reviews (PRISMA-ScR) and published in the Annals of Internal Medicine. Several JBI members were involved with the PRISMA-ScR, which was intentionally written to be consistent with the JBI methodological guide for scoping reviews.

Method: The PRISMA-ScR was developed according to published guidance by the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network. A modified Delphi approach was used, with three rounds of ranking and six working groups, and interaction at either an in-person meeting or online.

Results: The final PRISMA-ScR checklist contains 20 items considered to be essential components of a scoping review article, plus two optional items. A rationale for each item, along with an example of good reporting, is provided. Consistencies with the JBI guidance for the conduct of scoping reviews are noted.

Conclusion: This presentation will assist reviewers to understand the PRISMA-ScR statement and improve use of reporting of JBI scoping reviews. The PRISMA-ScR is consistent with the JBI guidance and is intended to guide the reporting of scoping reviews; improving transparency and completeness of reporting; facilitating the appraisal of results and increasing the relevance of scoping reviews for decision-making.

AMSTAR 2 tool using for selecting studies in rapid evidence-based products

Maria Teresa Vallejo-Ortega¹, Giovanna Andrea Gómez¹, Doctor John Feliciano-Alfonso¹

¹Instituto Nacional De Cancerologia

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Objective: To determine a cut-off point based on the AMSTAR 2 tool for including high-quality SR in F-EBP.

Background: Fast evidence-based products (F-EBP) development requires availability of tools that could help to get high-quality evidence in a short amount of time. In this regard, developers have been able to include only good-quality systematic reviews (SR) based on their AMSTAR score. Recently, the AMSTAR 2 tool was released; consequently, it is not clear how it can be used to establish a cut-off point for including evidence in this context.

Method: We performed a diagnostic test assessment approach using the SRs considered for eligibility in a specific EBP. As the test reference, we used AMSTAR score with a cut-off point of 6/11 for inclusion in the EBP product. AMSTAR 2 was the Index test. Three researchers performed independently the evaluations. We assessed the point-biserial coefficient and calculated sensitivity, specificity, likelihood ratios (LR), and ROC curves.

Results: We included 63 SRs in the present analysis and 33 of them were incorporated in the EBP product. The point-biserial coefficient was 0.6597 (P value < 0.0001). The best cut-off point was an AMSTAR 2 score of 8/16 [Area under curve (AUC) 0.89, IC95% 0.81 to 0.97, sensitivity 81.8%, specificity 86.7% LR+ 6.1 LR- 0.21). In an exploratory analysis, AMSTAR 2 critical domains evaluation showed an inferior performance.

Conclusion: In F-EBP, it is suggested to consider an AMSTAR 2 score of 8 or superior as eligibility criteria in circumstances where a plethora of SRs are available.

Reasons for Non-Cochrane Review Retraction: A Cross-sectional Study

Qi Zhou^1,2,3, Ruizhen Hou¹, Qianling Shi¹, Meng Lv⁴, Kefeng Liu⁵, Yongjie Yang⁵, Yaolong Chen^1,2,3, Kehu Yang^1,2,3

¹The First Clinical Medical School, Lanzhou University,²Evidence-Based Medicine Center of Lanzhou University,³WHO Collaborating Centre for Guideline Implementation and Knowledge Translation,⁴School of Public Health of Lanzhou University,⁵The First Affiliated Hospital of Zhengzhou University

CS7 - Systematic Review Advances, Hall C - Plenary, November 2, 2019, 9:00 AM - 10:30 AM

Aim: To investigate the reasons for retraction of the Non-Cochrane Review (NCR).

Background: Systematic reviews (SRs) is the cornerstone of guideline development. The retracted SR has certain reference significance for SRs makers and journal editors. By understanding the retracted SRs, SRs authors can circumvent common problems, journal editors can effectively review SRs, and ultimately publish high-quality SRs.

Methods: The PubMed and Embase databases were retrieved from their inception until March 31, 2019. We searched “Retracted, Retraction, Withdrawal, Systematic Review, Meta-analysis” words in the form of keywords combined with free words. Two researchers independently screened the records and extracted data, disagreements resulted through discussion or by a third party. Data processing was performed by Stata statistical software 14 to present descriptive results.

Results: A total of 1024 studies were retrieved and eventually included 101 NCRs. In terms of retracting parties, 12.9% of the retracted articles were retracted by authors, 22.8% were retracted by the editors, 54.5% were decided by both the publishers and the authors, and 8.9% were showed unclear. In terms of reasons for retraction, impaired peer review process(51.5%) and academic misconduct (10.9% plagiarism, 3.0% for one manuscript, 2.0% for falsified data, 4.0% for author attribution) were the main reason for retraction, and data errors (misquotation 7.9%, calculation error 4.0%, chart error 2.0%) and research overlap with others (5.9%) were the secondary reasons.

Conclusions: Journals should strengthen peer review of SRs, and producers of SRs should strengthen their academic ethics.