To the Editor:
Casey et al.1 examined associations between unconventional natural gas development in Pennsylvania and several birth outcomes, including preterm birth and high-risk pregnancy, concluding that “This study adds to limited evidence that unconventional natural gas development adversely affects birth outcomes.” But does it? There are two important impediments to this interpretation.
First, “unconventional natural gas development adversely affects birth outcomes” is an unwarranted causal interpretation of associational results: in general, “the associational or regression approach to inferring causal relations—on the basis of adjustment with observable confounders—is unreliable in many settings.”2 Without explicit causal analyses (e.g., Granger causality tests, causal graph models), claiming that associations provide evidence for a causal conclusion is unjustified.
Second, the associational analyses also have several limitations. Exposure metrics were not validated and estimation errors were ignored. The authors constructed surrogate exposure metrics using well depth and distance, ignoring relevant geology and whether homes are up- or down-gradient from the well. No monitoring data were used. In addition, an unknown number of wells used imputed data for exposure characterization. It is uncertain whether the surrogate metrics accurately reflect true exposures. Yet, errors-in-variables describing such uncertainties are omitted from model specifications. This alone could invalidate reported estimates, confidence intervals, and conclusions. No model validation or diagnostics were conducted to determine whether assumed models appropriately describe the data; thus, conclusions may simply reflect unjustified modeling assumptions. Appropriate statistical methods for spatial and longitudinal data, such as kriging or panel data analyses, were not used. It is unclear what more formal and thorough spatial and time series analysis would have revealed, but quantities that vary with location or time are often spuriously associated with each other in the absence of any causal relation.3 Negative controls, clarifying whether unconventional wells appear more harmful than randomly selected locations, were not discussed. Finally, the investigators ignored model uncertainty. They treat selected (but not validated) regression model forms as if they were known to be correct for purposes of calculating associations and confidence intervals, without quantifying model uncertainty. This risks creating artificially narrow uncertainty intervals, biases, and false positives.4
In light of these limitations, positive associations observed in selected models between an unvalidated exposure index and two (of several) birth outcomes examined do not provide valid evidence of a causal relation between unconventional natural gas development and adverse health impacts in newborns.
Louis Anthony (Tony) Cox, Jr.
Cox Associates and University of Colorado
1. Casey JA, Savitz DA, Rasmussen SG, et al. Unconventional natural gas development and birth outcomes in Pennsylvania, USA. Epidemiology. 2015.
2. Dominici F, Greenstone M, Sunstein CR. Science and regulation. Particulate matter matters. Science. 2014;344:257–259.
3. Cox LA Jr, Popken DA, Berman DW. Causal versus spurious spatial exposure-response associations in health risk analysis. Crit Rev Toxicol. 2013;43(suppl 1):26–38.
4. Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: a tutorial. Stat Sci. 1999;14:382–417.