Journal Logo

Letter to the Editor

Letter to the Editor

Effect Size Is Just as Important as P-Value

Emergency Medicine News: May 2020 - Volume 42 - Issue 5 - p 9
doi: 10.1097/01.EEM.0000666356.69367.9d

    Letters to the Editor

    Emergency Medicine News welcomes letters to the editor about any subject related to emergency medicine. Please limit your letter to 250 words, and include your full name, credentials, and city and state of residence or practice.

    Letters may be edited for content, length, and grammar. Submission of a letter constitutes the author's permission to publish on all media, including print, online, and social media, but does not guarantee publication. Letters express the views of the authors and do not necessarily reflect those of Emergency Medicine News and Wolters Kluwer.

    Letters to the editor may be sent to emn@lww.com.

    Editor:

    I compliment Rory Spiegel, MD, for his choice of articles to review, and his commentary regarding the possible existence of a subgroup that may benefit from these therapies is reasonable. I wanted to share with readers some important statistical issues in his article, “CRASH-3: TXA Has Nominal Benefit for TBI.” (EMN. 2020;42[2]:1; http://bit.ly/2UimLTH.)

    I ask readers this: Did you know that if a study enrolls enough subjects, a statistically significant difference between groups can be demonstrated for any groups compared? Of course, that difference may be very small.

    A recent review summarized four reasons scientists and doctors should not consider p-value the be-all and end-all. (Biol Lett. 2019;15[5]:20190174; http://bit.ly/3aEzPHW.) Considering a p-value in isolation, without concomitant consideration of effect size, represents an incomplete analysis.

    Effect size also matters. Take, for example, the WOMAN trial, which enrolled women with significant postpartum hemorrhage. (Lancet. 2017;389[10084]:2105; http://bit.ly/2skflDp.) This trial failed to demonstrate a difference in mortality after enrolling more than 15,000 women. After 20,060 patients were enrolled, a small mortality benefit was suggested for TXA. Looking at all causes of death, however, 227 of the 10,036 women given TXA died (2.3%), while 256 of the 9985 women not given TXA died (2.6%). This suggests a number needed to treat (NNT) of more than 330 women for TXA, NNT=(1/[.026-.023]).

    The 95% confidence interval for deaths, however, was a relative risk of 0.74-1.05. In other words, even after enrolling more than 20,000 patients, the 95% confidence interval extrapolated from this trial included the possibility that the true population relative risk of death may have been greater with TXA.

    I hope EMN readers look not only at the p-value but also at the effect size, which is just as important, when viewing clinical trial data. Interventions with small effect sizes should be viewed with skepticism despite a “significant” p-value, especially when the treatment is expensive. (Are you listening, cardiology colleagues?)

    Gary Gaddis, MD, PhD

    St. Louis, MO

    Dr. Spiegel responds: Great point! This is a really important concept to consider whenever you are appraising the literature. Thanks for your insightful comments.

    Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.