False Alarms and Pseudo-Epidemics

The Limitations of Observational Epidemiology

Abstract

Most reported associations in observational clinical research are false, and the minority of associations that are true are often exaggerated. This credibility problem has many causes, including the failure of authors, reviewers, and editors to recognize the inherent limitations of these studies. This issue is especially problematic for weak associations, variably defined as relative risks (RRs) or odds ratios (ORs) less than 4. Such associations, commonly reported in the medical literature, are more likely to be attributable to bias than to causal association. All observational research has bias (which can include selection, information, and confounding bias). Hence, detection of small associations falls below the discriminatory ability of observational studies. In general, unless RRs in cohort studies exceed 2 to 3 or ORs in case-control studies exceed 3 or 4, associations in observational research findings should not be considered credible. However, these guidelines are not foolproof: strong (yet spurious) associations can result when large amounts of bias are present. Only in a properly performed randomized controlled trial, free of bias, should small associations merit attention. Better training and more circumspection on the part of investigators, tougher editorial standards on the part of journals, and hefty skepticism on the part of referees and readers are necessary to avoid the dangers of false alarms, pseudo-epidemics, and their unfortunate consequences.

In Brief

Observational studies are not able to detect small associations, a limitation routinely overlooked in clinical research reports and tort law.