Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Performance of Between-study Heterogeneity Measures in the Cochrane Library

Ma, Xiaoyuea; Lin, Lifengb; Qu, Zhiyongc; Zhu, Motaod; Chu, Haitaoe

doi: 10.1097/EDE.0000000000000857

The growth in comparative effectiveness research and evidence-based medicine has increased attention to systematic reviews and meta-analyses. Meta-analysis synthesizes and contrasts evidence from multiple independent studies to improve statistical efficiency and reduce bias. Assessing heterogeneity is critical for performing a meta-analysis and interpreting results. As a widely used heterogeneity measure, the I2 statistic quantifies the proportion of total variation across studies that is caused by real differences in effect size. The presence of outlying studies can seriously exaggerate the I2 statistic. Two alternative heterogeneity measures, the


have been recently proposed to reduce the impact of outlying studies. To evaluate these measures’ performance empirically, we applied them to 20,599 meta-analyses in the Cochrane Library. We found that the


have strong agreement with the I2, while they are more robust than the I2 when outlying studies appear.

From the aDepartment of Anesthesiology, Columbia University College of Physicians and Surgeons, New York, NY

bDepartment of Statistics, Florida State University, Tallahassee, FL

cSchool of Social Development and Public Policy, Beijing Normal University, Beijing, China

dCenter for Injury Research and Policy, The Research Institute at Nationwide Children’s Hospital, Department of Pediatrics and Division of Epidemiology, Ohio State University, Columbus, OH

eDivision of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN.

Submitted September 6, 2017; accepted May 22, 2018.

The data and code are available for replication upon request.

The authors report no conflicts of interest.

Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article (

Ma and Lin have contributed equally to this article.

Correspondence: Haitao Chu, Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455. E-mail:

Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved.