Institutional members access full text with Ovid®

Share this article on:

False Alarms and Pseudo-Epidemics: The Limitations of Observational Epidemiology

Grimes, David A. MD; Schulz, Kenneth F. PhD, MBA

doi: 10.1097/AOG.0b013e31826af61a
Current Commentary

Most reported associations in observational clinical research are false, and the minority of associations that are true are often exaggerated. This credibility problem has many causes, including the failure of authors, reviewers, and editors to recognize the inherent limitations of these studies. This issue is especially problematic for weak associations, variably defined as relative risks (RRs) or odds ratios (ORs) less than 4. Such associations, commonly reported in the medical literature, are more likely to be attributable to bias than to causal association. All observational research has bias (which can include selection, information, and confounding bias). Hence, detection of small associations falls below the discriminatory ability of observational studies. In general, unless RRs in cohort studies exceed 2 to 3 or ORs in case-control studies exceed 3 or 4, associations in observational research findings should not be considered credible. However, these guidelines are not foolproof: strong (yet spurious) associations can result when large amounts of bias are present. Only in a properly performed randomized controlled trial, free of bias, should small associations merit attention. Better training and more circumspection on the part of investigators, tougher editorial standards on the part of journals, and hefty skepticism on the part of referees and readers are necessary to avoid the dangers of false alarms, pseudo-epidemics, and their unfortunate consequences.

Observational studies are not able to detect small associations, a limitation routinely overlooked in clinical research reports and tort law.

From the Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, and FHI 360, Research Triangle Park, North Carolina.

Corresponding author: Dr. David A. Grimes, Department of Obstetrics and Gynecology, CB #7570, UNC School of Medicine, Chapel Hill, NC 27599-7570; e-mail: david_grimes@med.unc.edu.

Financial Disclosure Dr. Grimes serves as a consultant to Bayer and Merck. The other author did not report any potential conflicts of interest.

© 2012 The American College of Obstetricians and Gynecologists