Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

The Sensitivity of Review Results to Methods Used to Appraise and Incorporate Trial Quality Into Data Synthesis

van der Velde, Gabrielle, DC*†‡; van Tulder, Maurits, PhD§∥; Côté, Pierre, DC, PhD*¶**††; Hogg-Johnson, Sheilah, PhD*†¶; Aker, Peter, DC§§; Cassidy, J David, PhD, DrMedSc†¶**†† Members of the Scientific Secretariat of The Bone and Joint Decade 2000–2010 Task Force on Neck Pain and Its Associated Disorders

doi: 10.1097/01.brs.0000258903.67718.d5
Health Services Research
Buy
SDC

Study Design. Systematic review.

Objective. To determine whether results and conclusions on the effectiveness of exercise for workers with neck pain vary with the Cochrane Back Review Group Guidelines and best-evidence synthesis review methods. To identify methodologic weaknesses associated with these review methods that may impact on the validity of their results.

Summary of Background Data. The Cochrane Back Review Group Guidelines and best-evidence synthesis have different approaches to appraising trial quality and incorporating quality into data synthesis. The impact of different review methods on the reproducibility and validity of review results is unknown.

Methods and Results. Systematic search of Medline, Embase, CINAHL, and Cochrane databases, without language restrictions. Twelve trials were selected. Two review methods were used to appraise trial quality and to incorporate quality into data synthesis. As recommended by the Cochrane Back Review Group Guidelines, trials were assigned quality scores using a scale. Results of all 12 trials were stratified into levels of evidence according to their scores. Based on these results, no treatment recommendation could be formulated. Best-evidence synthesis critically appraised methodology; trials were accepted on the strength of their scientific merit or rejected due to risk of bias. According to the 4 trials accepted for best-evidence synthesis, workers should be activated with exercise given its beneficial effect on patient-perceived recovery. Both the Cochrane Back Review Group Guidelines and best-evidence synthesis reviews were found to have weaknesses associated with their methods.

Conclusions. Review results and conclusions are sensitive to methods for appraising trial quality and incorporating quality into data synthesis when the evidence consists largely of low-quality trials. Both the Cochrane Back Review Group Guidelines and best-evidence synthesis methods were found to have strengths and methodologic weaknesses that healthcare decision-makers should be aware of when interpreting systematic reviews.

A Cochrane Back Review Group Guidelines review and a best-evidence synthesis addressing the same question were performed in parallel to determine the impact of different review methods on the reproducibility and validity of review results. Conclusions on the effectiveness of exercise for workers with neck pain varied substantively across methods.

From the *Institute for Work and Health, Toronto, Ontario, Canada; Departments of †Health Policy, Management, and Evaluation, and ¶Public Health Sciences, University of Toronto, Toronto, Ontario, Canada; ‡Canadian Memorial Chiropractic College, Toronto, Ontario, Canada; §Institute for Research in Extramural Medicine (EMGO), VU University Medical Center, Amsterdam; and ∥Institute for Health Sciences, VU University, Amsterdam, The Netherlands; §§Private Practice, Tweed, Ontario, Canada; **Division of Outcomes and Population Health, Toronto Western Research Institute, University Health Network, Toronto, Ontario, Canada; and ††University Health Network Rehabilitation Solutions, Toronto Western Hospital, Toronto, Ontario, Canada. ‡‡Eugene Carragee, Linda Carroll, Jaime Guzman, Scott Haldeman, Lena Holm, Eric Hurwitz, Margareta Nordin, and Paul Peloso.

Acknowledgment date: February 3, 2006. First revision date: July 9, 2006. Acceptance date: August 1, 2006. Supported by financial and material support provided by the Institute for Work & Health and the Bone and Joint Decade 2000–2010 Task Force on Neck Pain and Its Associated Disorders. Supported also by the Canadian Institutes for Health Research through a Fellowship Award and by the Canadian Memorial Chiropractic College (to G.v.d.V.) and the Canadian Institutes for Health Research through a New Investigator Award (to P.C.).

The manuscript submitted does not contain information about medical device(s)/drug(s).

No funds were received in support of this work. No benefits in any form have been or will be received from a commercial party related directly or indirectly to the subject of this manuscript.

Address correspondence and reprint requests to Gabrielle van der Velde, DC, Institute for Work & Health, 481 University Avenue, Suite 800, Toronto, Ontario M5G 2E9, Canada; E-mail: gvdvelde@iwh.on.ca

© 2007 Lippincott Williams & Wilkins, Inc.