Secondary Logo

Journal Logo

Web Site and R Package for Computing E-values

Mathur, Maya B.; Ding, Peng; Riddell, Corinne A.; VanderWeele, Tyler J.

doi: 10.1097/EDE.0000000000000864
Letters
Free

Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, MA

Quantitative Sciences Unit, Stanford University, Palo Alto, CA, mmathur@stanford.edu

Department of Statistics, University of California at Berkeley, Berkeley, CA

Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montréal, Quebec, CA

Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, MA

Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, MA

Reproducibility: No data analyses were conducted. All code for the R package and website is publicly available (https://github.com/mayamathur/evalue).

M.M. was supported by National Defense Science and Engineering Graduate Fellowship 32 CFR 168a. P.D. was supported by IES Grant R305D150040 from the Institute for Education Science and DMS grant 1713152 from the National Science Foundation. C.A.R. received salary support from McGill University’s Department of Epidemiology, Biostatistics, and Occupational Health. T.V.W. was supported by NIH grant ES017876. The funders had no role in the design, conduct, or reporting of this research.

The authors declare that they have no conflicts of interest.

Back to Top | Article Outline

To the Editor:

Observational studies often attempt to address questions related to causation. However, even with statistical adjustment for a number of measured confounders, residual unmeasured confounding may still compromise causal conclusions. New methods help quantify evidence strength for causality in the possible presence of unmeasured confounding through a new measure called the E-value.1,2 The E-value is defined as the minimum strength of association on the risk ratio scale that an unmeasured confounder would need to have with both the exposure and the outcome, conditional on the measured covariates, to fully explain away a specific exposure–outcome association. As discussed below, the E-value makes no assumptions on whether the unmeasured confounders are binary, continuous, or categorical, on how they are distributed, or on the number of confounders, and it can be applied to several common outcome types in observational research. To facilitate these sensitivity analyses, we provide an R package (“EValue”3) and also an online E-value calculator (https://mmathur.shinyapps.io/evalue/) that compute E-values for a variety of outcome measures, including risk ratios, odds ratios, rate ratios, risk differences, hazard ratios, and standardized mean differences.2

Suppose we have an observational study with a binary exposure

, a binary outcome

, and a possible binary unmeasured confounder

(though note that as discussed below, the E-value applies more generally). Two sensitivity parameters jointly determine the maximum bias that could result from unmeasured confounding in the estimated relative risk of the exposure on the outcome. First, to characterize the strength of association between the unmeasured confounder and the outcome, let

be the relative risk of the outcome comparing subjects with versus without the unmeasured confounder (

vs.

) and taken as the maximum over unexposed (

) or exposed (

) subjects. Second, to characterize the extent to which the prevalence of the unmeasured confounder is unbalanced between the exposed and the unexposed, let

be the relative risk of

versus

, comparing the exposed to the unexposed group and again conditional on any measured confounders. Then, if the 2 sensitivity parameters

and

are taken to be equal, the E-value is the minimum value for both associations that would be capable of attenuating the observed association to the null.1 The E-value can be calculated for an observed risk ratio (denoted

) by E-value

. If the original risk ratio is below 1, then one first takes the inverse before applying the E-value formula. This formula can also be used for hazard ratios or odds ratios with outcomes that are rare at the end of follow-up. For hazards or odds ratio with a common outcome at the end of follow-up, or with continuous outcomes, approximate E-values can still be obtained through various transformations.1

For example, with an observed risk ratio of

, we can calculate an E-value of

. This E-value indicates that if there were an unmeasured confounder that (1) doubled the risk of the outcome among either the unexposed or the exposed (

) and (2) that were also twice as prevalent among the exposed than among the unexposed (

), this amount of confounding could suffice to completely “explain away” the observed association, but weaker confounding could not. Although this interpretation of the 2 sensitivity parameters is given in the context of a binary unmeasured confounder, the E-value applies without modification to multiple, potentially categorical, confounders by considering the maximum risk ratio comparing any 2 categories of the unmeasured confounder(s). With a continuous confounder, the interpretations of the parameters

and

are slightly different, but the mathematical form of the E-value is unchanged.2

Ideally, we believe, E-values would be reported routinely for observational studies to better characterize evidence strength for causality above and beyond the presence of a “statistically significant,” but potentially spurious, association.1,2,4 The E-value could be reported for both the point estimate and the corresponding confidence interval limit that is closer to the null; these E-values represent the minimum confounding strength, respectively, capable of attenuating the point estimate to the null and capable of attenuating the confidence interval such that it includes the null.1 Last, it is easy to calculate E-values for values of a true effect other than the null of

to assess how much confounding would be needed to move the estimate to any other value. For example, as part of a holistic assessment of the scientific importance of the true causal effect in an observational study, one could choose an effect size threshold below which a causal effect might be considered too weak to be meaningful, as informed by the specific scientific context. Then, one could assess the E-value capable of attenuating the observed association to this small, non-null effect size threshold, or alternatively, to increase a near-null result to one that is of meaningful size in the given scientific context.2

In addition to calculating E-values, the R package we provide also produces plots visualizing the maximum possible bias in the observed association as a function of

and

. In contrast to existing code,2 the present R package handles more outcome types and can characterize the minimum confounding strength capable of attenuating the observed association to a non-null threshold of scientific importance. Additionally, we provide a freely available web site (https://mmathur.shinyapps.io/evalue/) to easily compute E-values without requiring coding or familiarity with R.

Back to Top | Article Outline

ACKNOWLEDGMENTS

We thank Jaffer Zaidi for serving as a pilot tester.

Maya B. Mathur

Department of Biostatistics

Harvard T. H. Chan School of Public Health

Boston, MA

Quantitative Sciences Unit

Stanford University

Palo Alto, CA

mmathur@stanford.edu

Peng Ding

Department of Statistics

University of California at Berkeley

Berkeley, CA

Corinne A. Riddell

Department of Epidemiology

Biostatistics, and Occupational Health

McGill University, Montréal, Quebec, CA

Tyler J. VanderWeele

Department of Biostatistics

Harvard T. H. Chan School of Public Health

Boston, MA

Department of Epidemiology

Harvard T. H. Chan School of Public Health

Boston, MA

Back to Top | Article Outline

REFERENCES

1. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med. 2017;167:268–274.Am Coll Physicians.
2. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiol. 2016;27:Wolters Kluwer Health; 368.
3. Mathur MB, Ding P, VanderWeele TJ. Package ‘EValue’, version 1.0.0. 2017.
4. Localio AR, Stack CB, Griswold ME. Sensitivity analysis for unmeasured confounding: E-values for observational studies. Ann Intern Med. 2017;167:285–286.
Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved.