The Journal Impact Factor as a Predictor of Trial Quality and Outcomes

Cohort Study of Hepatobiliary Randomized Clinical Trials

Abstract

OBJECTIVES 

To examine the association between the impact factor and characteristics of hepatobiliary randomized clinical trials.

METHODS 

A cohort study of 530 hepatobiliary randomized clinical trials was performed. The journal impact factor was extracted from Science Citation Index. For each trial, we extracted the sample size, the quality of randomization and blinding methods, and the statistical significance of the primary outcome measure.

RESULTS 

The median sample size was 45 participants (interquartile range 25–88). The allocation sequence generation was adequate in 273 trials (52%). Allocation concealment was adequate in 178 trials (34%). The primary outcome measure was statistically significant in 374 (71%) trials. Nonparametric analyses for trend indicated that the impact factor was significantly associated with the sample size (p < 0.01) and the proportion of trials with adequate allocation sequence generation (p < 0.01) or allocation concealment (p = 0.02). The impact factor was not significantly associated with the study outcome (p = 0.28).

CONCLUSIONS 

The present study supports the use of the impact factor as a rough quality indicator. However, even trials in high impact journals may be small or may have inadequate quality. Critical appraisal of individual trials is always necessary, irrespective of the place of publication.