Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Zen and the Art of Multiple Comparisons

Lindquist, Martin A. PhD; Mejia, Amanda MS

doi: 10.1097/PSY.0000000000000148

Objective The need for appropriate multiple comparisons correction when performing statistical inference is not a new problem. However, it has come to the forefront in many new modern data-intensive disciplines. For example, researchers in areas such as imaging and genetics are routinely required to simultaneously perform thousands of statistical tests. Ignoring this multiplicity can cause severe problems with false positives, thereby introducing nonreproducible results into the literature.

Methods This article serves as an introduction to hypothesis testing and multiple comparisons for practical research applications, with a particular focus on its use in the analysis of functional magnetic resonance imaging data.

Results We discuss hypothesis testing and a variety of principled techniques for correcting for multiple tests. We also illustrate potential pitfalls problems that can occur if the multiple comparisons issue is not dealt with properly. We conclude, by discussing effect size estimation, an issue often linked with the multiple comparisons problem.

Conclusions Failure to properly account for multiple comparisons will ultimately lead to heightened risks for false positives and erroneous conclusions.

From the Department of Biostatistics, Johns Hopkins University, Baltimore, Maryland.

Address correspondence and reprint requests to Martin A. Lindquist, PhD, Department of Biostatistics Johns Hopkins University, 615 N Wolfe St, E3634, Baltimore, MD 21205. E-mail:

Received for publication May 15, 2014; revision received September 26, 2014.

Copyright © 2015 by American Psychosomatic Society
You currently do not have access to this article

To access this article:

Note: If your society membership provides full-access, you may need to login on your society website