Risk of bias assessment in JBI systematic reviews: a new series of articles in JBI Evidence Synthesis

A defining feature of many reviews of existing research, and certainly any review following current JBI methodologies that can lay claim to being systematic, is critical appraisal of the included studies. The process is focused principally on assessing the quality of conduct (its validity) and the possibility or potential that bias has crept into the design, conduct, or analysis of the study. Identifiable risk of bias will impinge on the trustworthiness of the results and any subsequent conclusions presented. Assessment of risk of bias is a critical consideration in the analysis and interpretation of any synthesis and, accordingly, any assessment of certainty in the findings presented in the review following the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach.¹ Checklists, scales, and domain-based instruments are commonly used to aid systematic reviewers to complete and present an assessment that is both consistent and transparent in its conduct as well as, ideally, its reporting. The existing suite of JBI checklists to facilitate critical appraisal has been available for over two decades. These tools are frequently used by authors and are valued for their ease of use and comprehensiveness; the latter facilitating authors’ ability to assess study validity across a range of common study designs relying on a single source.²

This issue of JBI Evidence Synthesis presents the launch of an exciting new series focused on risk of bias in JBI systematic reviews of quantitative evidence, which reflects the current and ongoing work of the JBI Effectiveness Methodology Group.³ The articles in this issue present foundational concepts that underpin considerations of risk of bias. The first 2 articles in the series present both the vision and the basis for the planned reconsideration of the JBI tools aligned with international developments in the field⁴ and the evolution of terminology in the field, respectively.⁵ Conceptual papers such as these will accompany the revised tools designed for practical use throughout the series as it grows. An additional article articulates the methods that have been applied by the JBI Effectiveness Methodology Group^3,6 and will be applied to the remainder of the tools involved.⁷

Furthermore, to headline the practical changes for JBI systematic reviews, the revised JBI tool for assessing risk of bias in randomized controlled trials, one of the most frequently used by review authors, is also presented in this issue.⁸ Signaling questions in the revised tool have not been modified from the currently available version; however, the existing questions have now been reorganized and aligned to diverse and recognizable domains of bias. This alignment will help reviewers in their consideration of the impact of potential bias in the results of their analyses. The update also sees the use of this tool prompt users to move beyond assessment of risk of bias at the study level, and to consider the response to a subset of the questions for individual outcomes, as well as for a further subset of questions, considering the results reported.⁸ These changes illustrate the first stage in the planned revision process for the JBI tools and, while maintaining the ease of use that has made them so popular, it marks the beginning of a much-needed evolution toward demanding more sophisticated consideration of risk of bias in JBI systematic reviews.⁴

The advent of this first revised tool, and the promise of more to follow in this JBI Evidence Synthesis series on risk of bias, has broader impacts for JBI society members and systematic reviewers. Most importantly, as the new revised tools are introduced, authors who submit manuscripts to JBI Evidence Synthesis, as well as peer reviewers and editors of the journal, will be required to embrace these new tools and the nuances of their use as we move forward, working together to present high-quality evidence informed by the most current methods in synthesis to the community. Over the next 12 months, updates to journal guidelines and templates will be made available, along with a transition period for authors to adopt the new tools during the conduct of their review. Similarly, the content of the JBI education program for synthesis and accredited international trainers, and the forthcoming development of JBI SUMARI—JBI’s software to facilitate and support the conduct of systematic reviews—must now also embrace and integrate these changes and evolve accordingly.^3,9,10

In line with the methodological theme for this month, this issue presents a scoping review providing a comprehensive overview of the use of statistical shape modeling of the hip joint,¹¹ and also a range of review protocols, all linked to investigation of methods of research, its dissemination, or conduct of synthesis. A further noteworthy addition to the content of this issue dominated by consideration of methodological conduct of reviews is the latest guidance from the JBI Scoping Review Methodology Group delving further into extraction, analysis and presentation of the results of scoping reviews.¹² Scoping reviews are popular review undertakings, evidenced by the increasing appearance of this type of review in the table of contents of JBI Evidence Synthesis over the past 2 to 3 years. This additional guidance is essential reading for any scoping review author, providing additional clarity and useful examples across these important steps in the review process.