Rejoinder: Understanding Uncertainties in a Change Versus Change Study

Fitz-Simon, Nicola; Fletcher, Tony; Armstrong, Ben

doi: 10.1097/EDE.0b013e3182961926
Author Information

From the Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine, London, United Kingdom.

Funded by The C8 Class Action Settlement Agreement.

The authors report no conflicts of interest.

Correspondence: Nicola Fitz-Simon, Neonatal Medicine, Imperial College London, 4th Floor, Chelsea and Westminster Hospital, 369 Fulham Road, London SW10 9NH, United Kingdom. E-mail:

Article Outline

“I was in great perplexity,” begins Franz Kafka’s “A Country Doctor” as he sets out into a blizzard1; and as we read Dr. Burstyn’s commentary2 we share the doctor’s feeling. Our perplexity arises from both Dr. Burstyn’s apparent misunderstanding of the analyses we have done, and the fact that he attributes to us, and comments on, analyses we did not do, and claims we did not make.3

For instance, Burstyn appears to believe that we have done an ecological analysis and that we attribute an overall change in low density lipoprotein (LDL) cholesterol to the average change in exposure to perfluoralkyl acids (PFAAs). We did not do an ecological analysis because the difference in mean LDL between 2005/2006 and 2010 may be affected by a number of factors, and the resulting parameter estimates would thus be particularly vulnerable to bias: the population is ageing, there may have been shifts in dietary, exercise, and other determinants of lipids, and there may have been a drift in laboratory measurements. This we clearly stated in our article. However, as we also showed in the Statistical Analyses section, the relevant slope estimate from the individual change versus change model is not affected by factors such as laboratory drift that cause lipid changes in the whole population. This and the opportunity to eliminate time-invariant confounding by design were our reasons for choosing the change versus change model. We are disappointed that Burstyn does not acknowledge that this model reduces some biases that may arise in a cross-sectional analysis.

We agree with Burstyn that change versus change models are not immune to all biases due to measurement error, but we never claimed they were. As Burstyn notes and Liker has shown in more detail, to the extent that random errors in repeat measures of PFAA were not perfectly correlated, there will be classical error in the PFAA difference.4 However, as Liker shows, this would bias regression slopes toward the null and thus would not explain an observed non-null association. The same would apply to random error introduced if, as Burstyn hypothesizes, the timing of the biologically effective dose was imperfectly picked up by the serum PFAA. Thus while it is always the case that confidence intervals reflect only one source of uncertainty, bias due to random measurement error is overwhelmingly likely to be toward the null, so broadening confidence intervals should reflect this uncertainty.

With respect to possible confounding by time-varying risk factors, Burstyn mentions US trends in serum lipids, citing findings from the analysis of three cross-sectional data sets from the National Health and Nutrition Examination Surveys.5 Overall LDL has decreased between 1988 and 2010. This may be due in part to a decrease in the consumption of trans-fatty acids, but the main reason is the increasing use of lipid-lowering drugs. For this very reason, we were careful to exclude all participants who reported that they had been prescribed lipid-lowering drugs during the study period. Furthermore, we did a very conservative sensitivity analysis in which we assumed that all participants who reported their lipid-lowering drug use before their follow-up blood tests were actually prescribed lipid-lowering drugs in the brief intervening period. Furthermore, time-varying risk factors would confound our change versus change estimate only if changes in the risk factors were correlated with the changes in PFAAs, for which there seems no strong plausibility. Although the relative robustness to confounding of change versus change estimates is, we believe, clear, we agree that such estimates will be subject to some residual confounding and biases.4 However, none of the specific sources of bias Burstyn describes are very convincing beyond that due to random error—which is much more likely be toward the null. Although regression to the mean can produce biases in many contexts, both heuristic arguments and simulations indicate that this is not, as Burstyn claims without argument, one of those contexts (unless via random error in PFAAs, which would bias estimates toward the null as noted above). The choice, advocated by Burstyn, to prefer our Model 3 perfluoroctanoic acid (PFOA) estimate adjusted for perfluoroctanesulfonic acid (PFOS) is arguable if one assumes that the association with PFOS is causal, but Burstyn does not. Furthermore, even if we were to accept Burstyn’s various ad hoc adjustments to the 95% confidence interval to make it a full uncertainty interval, this would change interpretation if only one makes a fetish of whether the 95% confidence interval includes the null.

We acknowledged in our article that the predicted effects of changing PFOA and PFOS were small. The implications are at the population level rather than an individual level. We did not call on country doctors to worry about reducing environmental exposures to lessen the risk of heart disease in their patients. We did find a positive association between LDL and PFOA that is corroborated by other studies. We consider it important that this finding is made public so that anyone from a simple doctor to a clever epidemiologist to a person living in an area with public water supplies contaminated with industrial pollutants may be aware of the evidence.

Back to Top | Article Outline


1. Kafka F. Transl: Willa, Edwin Muir. The Penal Colony: Stories and Short Pieces. 1948 New York Schocken Books
2. Burstyn I. Country doctor versus epidemiologist: how uncertainty analysis can help see what is in plain view. Epidemiology. 2013;24:577–579
3. Fitz-Simon N, Fletcher T, Luster MI, et al. Reductions in serum lipids in relation to a 4-year decrease in serum perfluorooctanoic acid and perfluoroctane sulfonic acid. Epidemiology. 2013;24:569–576
4. Liker JK, Augustyniak S, Duncan GJ. Panel data and models of change: a comparison of first difference and conventional two wave models. Soc Sci Res. 1985;14:80–101
5. Carroll MD, Kit BK, Lacher DA, Shero ST, Mussolino ME. Trends in lipids and lipoproteins in US adults, 1988-2010. JAMA. 2012;308:1545–1554
© 2013 by Lippincott Williams & Wilkins, Inc