Secondary Logo

Journal Logo

Letters to the Editor: Letters & Announcements

P Values

A Guide to Uncertainty but Not Truth

Myles, Paul S., MPH, FANZCA; Forbes, Andrew, PhD

Author Information
doi: 10.1213/ANE.0b013e3181f2c02f
  • Free

To the Editor

We take issue with the comments of Cappiello et al.,1 who in responding to a query regarding P values in a recent paper interpreted the difference between a P value of 0.07 vs. 0.03 to mean that these represent the probability of the difference not (italics our emphasis) being observed by chance alone being 93% vs. 97%. In other words, they state, the odds still favor the intervention albeit with less certainty than the standard threshold for “statistical significance.” This is a common but wrong interpretation of the P value.24

The P value is the probability that a difference as large or larger as that observed arises when the null hypothesis is true. In the absence of bias, the mechanism by which such differences occur is chance, and therefore the P value is often stated as the probability that the difference as large or larger as that observed occur by chance alone. Note, critically, that this is not the probability that the null hypothesis is true, and so 1 − P value is not the probability of the difference not being due to chance; such a statement infers a probability of truth, something a P value does not describe. P is calculated on the assumption that the null hypothesis is true; it is not the probability of that assumption being true. For completeness, the correct interpretation of 1 − P value (93% or 97%) is the probability that a difference of the size observed, or smaller, occurs under the null hypothesis, and this clearly differs markedly from the interpretation of Drs. Cappiello et al.1 As Steven Goodman pointed out in his critique of the P value,4 the error in inferring statements about the probability of the null hypothesis from the P value reinforces the mistaken notion that the data alone can tell us the probability of truth.

Paul S. Myles, MPH, FANZCA

Andrew Forbes, PhD

Alfred Hospital and Monash University

Melbourne, Australia

p.myles@alfred.org.au

REFERENCES

1. Cappiello E, O'Rourke N, Segal S, Tsen LC. Are the conclusions supported by the statistics? Anesth Analg 2010;110:969
2. Browner WS, Newman TB. Are all significant p values created equal? The analogy between diagnostic tests and clinical research. JAMA 1987;257:2459–63
3. Sterne JAC, Davey Smith G. Sifting the evidence: what's wrong with significance tests? BMJ 2001;322:26–31
4. Goodman S. Towards evidence-based medical statistics: 1. The P value fallacy. Ann Intern Med 1999;130:995–1004
© 2011 International Anesthesia Research Society