Secondary Logo

Journal Logo

Causal Inference in Environmental Epidemiology

Old and New Approaches

Pearce, Neila; Vandenbroucke, Jan P.a,b,c; Lawlor, Deborah A.a,d,e

doi: 10.1097/EDE.0000000000000987
ISEE Commentary
Free
SDC
ISEE

aLondon School of Hygiene and Tropical Medicine, London, United Kingdom

bDepartment of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

cDepartment of Clinical Epidemiology, Aarhus University, Denmark

dMRC Integrative Epidemiology Unit, University of Bristol, Bristol, United Kingdom

ePopulation Health Science, Bristol Medical School, Bristol, United Kingdom.

Editor’s Note: A related commentary appears on p. 317.

The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013)/ERC grant agreement no 668954 and ERC grant agreement no 669545 and from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No 733206.

This commentary was sponsored by the International Society of Environmental Epidemiology (ISEE). The contents are the sole responsibility of the author(s) and do not necessarily reflect the official views of the ISEE.

Correspondence: Neil Pearce, Department of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom. E-mail: neil.pearce@lshtm.ac.uk.

It has been argued that epidemiology is currently going through a methodologic revolution involving the “causal inference” movement.1,2 This proposes that observational studies should mimic key aspects of randomized trials, because this allows them to be rooted in counterfactual reasoning, which is said to formalize the natural way that humans think about causality.3–5 These new methods have many merits, particularly for conducting studies of interventions; they have also led to technical analytic innovations.6–9

However, we and others have argued that causal inference needs integration of a wider range of methods to answer the complex questions needed to improve population health.6–12 Causal inference almost never hinges on a single method or a single study but rather involves considering a wide variety of evidence.13 Thus, we consider it unfortunate that the term “causal inference” is being used to denote a specific set of newly developed methods rather than taking a pluralistic approach that encompasses both the older traditional methods that we continue to use and the newer ones that have become available9 (we use quotation marks to denote this randomized controlled trial (RCT)–mimicking set of “causal inference” methods, in contrast to the broader field of causal inference of which it is a part).

Environmental epidemiologists have always attempted to make inferences about causality from imperfect data and have discovered many major environmental causes of disease (e.g., contaminated water and cholera,14 air pollution and respiratory disease,15 Balkan nephropathy,16 and many more17), using “traditional” methods, i.e., those existing before the new “counterfactual based” methods. These traditional methods reflect the nature of population-level exposures that are fundamental to environmental epidemiology. The purpose of this commentary is to describe the challenges of making causal inferences in environmental epidemiology and to describe complementary causal inference methods (both old and new). In particular, we describe how several methods can be integrated in a triangulation framework to improve causal inference in this field.

Back to Top | Article Outline

CHALLENGES TO CAUSAL INFERENCE IN ENVIRONMENTAL EPIDEMIOLOGY

The term “environmental exposure” is sometimes used loosely to mean any exposure that is not genetic. However, the field of environmental epidemiology is typically restricted to “physical, chemical, and (noninfectious) biological factors in our everyday environment”,18 although some approaches may also include the global eco-environment19 and the local social environment; many environmental exposures (e.g., pesticides) can also occur in the occupational environment, so the two fields overlap considerably. On the other hand, it does not usually consider individual behavioral factors. For example, environmental tobacco smoke exposure would be considered as an environmental epidemiology problem, whereas individual smoking behavior typically would not.

Environmental epidemiology has some relatively unique characteristics that have often made causal inference difficult, because it is inherently focused on exposures which occur in dynamic and evolving populations, with their particular societal characteristics. This is typified by issues such as climate change, urban design, public transportation, air pollution, and water and soil contamination, all of which usually affect individuals across entire communities. The implication of this is that it is often difficult to mimic an RCT, with specific well-defined interventions, and (conditional) exchangeability of exposure groups. An extreme but increasingly urgent example is to determine the effects of climate change on health: mimicking an RCT would require the existence and availability of similar societies that could be (cluster) randomized; this would require at least two planets for a study to be conducted successfully.20,21

A related issue is that confounders will also often affect entire communities. For example, the association of population-level exposure to contaminated water with health outcomes is likely to be confounded by other population-level factors such as the level of economic development, poor housing, and indoor air pollution. Some sources of confounding so closely co-occur with the specific toxicants or pollutants that are the exposures of interest, such that methods dealing with collinearity and identifiability need to be considered.22 This has been a particular issue in air pollution studies where it has been difficult to validly estimate the effects of individual components of PM2.5 pollution.23

These methodologic difficulties mean that some environmental epidemiology questions cannot be answered simply by doing “better” studies that more closely mimic RCTs. A pluralistic approach is required, with the integration of evidence provided by a variety of study designs and approaches. Therefore, we briefly describe different approaches to causal inference that we feel have value in environmental epidemiology and discuss the possibility of integrating findings in a triangulation framework. We group these methods into three general categories: (1) “traditional” methods, (2) extensions of these traditional approaches, and (3) triangulation of evidence.

The Table summarizes methods that we consider have specific value for causal inference in environmental epidemiology.

TABLE

TABLE

Back to Top | Article Outline

‘TRADITIONAL’ METHODS

As noted above, environmental epidemiology is often concerned with population-level exposures. Thus, time trends and geographic differences, often disparaged as implying a lower level of causal evidence (“old-fashioned” descriptive epidemiology), may be particularly useful, both in generating new ideas and as a check on existing explanations.32 For example, global asthma prevalence comparisons have provided strong evidence that “established” asthma risk factors such as allergen exposure, air pollution, and environmental tobacco smoke do not explain the population patterns and are likely to be secondary rather than primary causes of asthma itself.33 Ecologic studies have played a key role in identifying that arsenic in drinking water is a cause of cancer.34 Similarly, international comparisons of the prevalence of chronic kidney disease of unknown cause are playing a crucial role in the search for the causes of this major public health problem.35

Furthermore, findings from environmental epidemiology can be more convincing if they are replicated in different populations with different underlying patterns of confounding (an approach known as cross-context comparisons).12 For example, exposure to air pollution from truck traffic primarily occurs in poor people in high-income countries, whereas it is often more common in rich urban-dwellers in low- and middle-income countries; thus, it is reassuring that findings for air pollution from truck traffic and asthma symptoms are similar in high-income countries and low- and middle-income countries.25 The effects of environmental exposures can also be investigated in specific occupational populations where exposures are often higher, and confounding is often minimal, because there are usually few socioeconomic and behavioral differences between different groups of workers.26 Thus, risks from low-level environmental exposures are rarely studied directly; rather, the effects of occupational exposures (which are higher and less subject to confounding) are studied, and the risks to exposed communities are estimated by extrapolation.

Back to Top | Article Outline

EXTENSIONS OF TRADITIONAL APPROACHES

In this section, we consider several extensions of traditional approaches, many of which have been used for decades in econometrics, but only applied to epidemiology more recently.

Instrumental variable (IV) analyses utilize variables that robustly relate to the exposure of interest in a way that they can be seen as good as randomizing the exposure. Such variables, like any technique used for proper randomization, should not be related directly to the outcome, nor to potential confounders (i.e., other risk factors for the outcome). If such a variable is found, it has the potential to improve causal inference.36 For example, one study, using wind speed and height of the planetary boundary layer as IVs that determine air pollution (and are not direct causes of mortality, nor likely to be associated with other risk factors for mortality), found evidence for an effect of local air pollution (at levels below the US standards) on daily death rates.27 In another study, differences in the order that piped water was supplied to houses and the water company providing water, in Yemen, were used as IVs to test the effect of piped water supply on childhood diarrhea.37,38 The results suggested that piped water increased childhood diarrheal diseases owing to water rationing or broken pipes resulting in its contamination.

Mendelian randomization, the use of genetic variants as IVs, is increasingly used to explore causal effects in epidemiology.39,40 Although genetic IVs may be less prone than nongenetic IVs to violations of the assumptions of IV analyses,40 they do not reflect the population-level exposures that are the focus of this commentary. However, an extension of Mendelian randomization that uses gene–environment interactions to explore causality could have value in establishing underlying mechanisms in environmental epidemiology. The assumption is that genetic variants that are known to influence the metabolism of, for example, pollutants would only be associated with the relevant health outcomes in populations exposed to that pollutant. For example, trichloroethylene (TCE) has been found to be associated with renal cancer risk in workers with at least one intact GSTT1 allele (odds ratio = 1.88), but not among workers with two deleted alleles (odds ratio = 0.93).28 Similarly, active GSTT1 genotype was associated with renal cancer risk in those exposed to TCE, but not in those unexposed to TCE. Such analyses are also particularly relevant to studies which explore mechanisms through which population-level exposures might act.41

There are two types of “negative control” studies: outcome and exposure negative controls. Negative control outcome studies use associations between the exposure of interest and a condition thought to be unaffected by the exposure to highlight potential residual or uncontrolled confounding.12 These studies are widely used in pharmacoepidemiology, where the control outcomes are known as prespecified falsification outcomes.42 We have found few examples of this approach in environmental studies of physical or chemical exposures, but we found one example used in social environmental epidemiology. Numerous studies have shown associations between social networks (i.e., where persons with social ties are more likely to have a similar outcome than two random people from the same population) and the spread of complex health-related outcomes (e.g., smoking, obesity, and depression). The assumed causal mechanism here is that social networks influence behavior, such that (for example) people of a healthy weight who change their social networks towards groups who are overweight or obese, may increase their own risk of becoming overweight or obese (because of moderating their ideas of what constitutes a healthy weight and changing behaviors to those of the new social network which are more obesogenic). However, a prespecified falsification/negative outcome control study suggested that such hypothesized mechanisms were unlikely, because they found similar associations with outcomes that the authors a priori assumed could not be explained by these mechanisms (acne, height, and headaches).29 On the other hand, negative control exposure studies have been widely used in studies of the developmental origins of disease, typically by using the association between paternal exposures (negative control) and outcomes to highlight potential uncontrolled confounding (see, e.g., the study by Santos Ferreira et al43). Population-level exposures in environmental epidemiology make negative control exposure studies less plausible in environmental epidemiology, but we would encourage the greater use of negative control outcome studies. For example, exposure to pesticides from aerial spraying often affects whole districts, and a number of different health outcomes may be affected by these pesticides, but showing associations with one or more outcomes where a confounded association is likely but a causal effect not plausible (e.g., deaths from violence) would raise questions as to whether the observed associations for other outcomes might also be owing to confounding.

Regression discontinuity designs44–46 can be applied when exposure is assigned at a threshold, as is often the case in medicine, particularly if the threshold is a continuously measured variable. The assumption is that people just above or just below the threshold will be assigned different exposures, but that these people are in fact very much alike, given the likely random errors in measuring the variable used for the assignment. An example is the assignment of antiretroviral therapy according to CD4 count, where the idea is that the persons just below or above the threshold may differ little; another is the study of the effect of mailing of a warning letter by a health authority to general practitioners who prescribed an inordinate amount of a particular drug (say, a painkiller or sleeping drug) where the idea is that the general practitioners just above and just below the threshold for mailing the letter might be similar. The design has been applied in a variety of other contexts, including a study of ozone, smog warnings, and asthma hospitalizations.30

Difference in differences analyses require that the outcome is measured repeatedly over time. They compare the mean change in outcome over time between exposed and unexposed groups (or between different levels of exposure). In all categories of exposure there must be at least one measure of the outcome before, and at least one measure after, exposure occurred. The assumption is that baseline differences in outcome (i.e., prior to exposure) reflect differences in confounders and that rates of change in outcome are similar until the exposure occurs (parallel slope assumption). Under this assumption, the differences in outcome between those exposed and those unexposed, “before” versus “after”, reflects the causal effect of exposure. In one example, this method was used to explore the impact of greening vacant urban spaces (in comparison with urban spaces which were not greened), finding some evidence of benefits on criminal behavior but limited effects on health outcomes.31

Back to Top | Article Outline

TRIANGULATION OF EVIDENCE

The idea of triangulating evidence from different methods and data sources has been proposed and used implicitly for decades, often without explicitly describing it as triangulation.10,12,47 In fact, the term “triangulation” has been used in at least two different ways in health research: (1) to refer to multiple lines of evidence from different research approaches, including integrating epidemiologic findings with other forms of evidence and (2) to refer within the field of epidemiology to different analytical approaches/populations that have been chosen because they have differing key sources of bias (ideally in different directions).12

The first type of triangulation is routinely used in assessing environmental health research, e.g., by the International Agency for Research on Cancer (IARC) Monographs Programme, which integrates epidemiologic, animal, and mechanistic evidence to infer causality for various potential carcinogens, including environmental carcinogens. One application was the assessment of the health effects of environmental tetrachlorodibenzo-p-dioxin (TCDD; dioxin) exposure. The main health effects are likely to occur owing to exposure to low levels that are near-ubiquitous across populations, but these were difficult, if not impossible, to elucidate. However, by integrating evidence from different study designs and methods (occupational studies in a number of different countries, animal studies, and mechanistic studies showing that TCDD increases the risk of cancer through its action at the aryl hydrocarbon [Ah] receptor), IARC has concluded that there is sufficient evidence in human (i.e., epidemiologic) studies that dioxin is a cause of cancer.48 A similar example is that of Balkan Endemic Nephropathy (BEN),16 for which a wide variety of evidence (epidemiologic, genetic, and toxicologic) was required before it was established that the likely cause was chronic dietary exposure to aristolochic acid, a contaminant of wheat in the endemic regions. These can be regarded as examples of triangulation in that different methods were brought to bear on the issue, with studies being conducted in a number of different populations; however, the term “triangulation” was not used in either.

The second type of triangulation refers to triangulation of different types of evidence within epidemiology, which might be called “epidemiologic triangulation”. We have had difficulty in finding examples of the latter approach within environmental epidemiology, and we propose that this approach be used more systematically in this field to improve causal inference and understanding in human populations. Criteria for its use in causal inference in epidemiology have been proposed recently, and these specify that results from at least two (but ideally more) methods that have differing key sources of unrelated bias be compared.12 If evidence from such different epidemiologic approaches all point to the same conclusion, this strengthens confidence that is the correct causal conclusion, particularly when the key sources of bias of some of the approaches would predict that the findings would point in opposite directions.

The difference between epidemiologic triangulation and the systematic review approach of trials or epidemiologic studies is that a systematic review seeks similar studies, which are expected to yield similar findings, and hence can be grouped in a meta-analysis to obtain a more precise estimate of an exposure. Epidemiologic triangulation, in contrast, looks for different types of studies, which might be expected to yield different findings, because they involve different potential biases, or biases in different directions; this allows one to assess the likely existence or absence of the biases that one might be concerned about in one particular type of study.

Back to Top | Article Outline

CONCLUSIONS

Where does this leave us? It is opportune to write this commentary in Epidemiology, which has published many of the successes of the causal inference movement and which is also the official journal for the International Society of Environmental Epidemiology (ISEE). We are not arguing that “causal inference methods” that mimic RCTs are not useful; for example, they can improve individual studies with individual-level exposures that can be seen as interventions. Rather, we are arguing that they form only part of the larger set of causal inference methodologies. There have been older methods, as well as other developments in methodology, which are complementary to, and in some instances superior to causal inference methods, at least for some risk factors or in some contexts. All methods have assumptions that are often not possible to (fully) test. We believe that all valid methods should be part of the (environmental) epidemiology toolkit and that integrating the resulting evidence in a framework that acknowledges the key sources of bias of each will provide for better causal inference.

Back to Top | Article Outline

ABOUT THE AUTHORS

NEIL PEARCE is Professor of Epidemiology and Biostatistics at the London School of Hygiene and Tropical Medicine. His current research interests focus on epidemiological and biostatistical methods, and their application to studies of non-communicable diseases (NCDs), including occupational and environmental health, asthma, kidney disease, and neurological disease. He has a particular interest in global epidemiological studies and was President of the International Epidemiological Association during 2008–2011.

JAN P. VANDENBROUCKE is Emeritus Professor of Clinical Epidemiology, Leiden University Medical Center, the Netherlands, Professor of Clinical Epidemiology at Aarhus University, Denmark, and Honorary Professor at the London School of Hygiene and Tropical Medicine, London. His main interests are translating techniques of observational epidemiology to the research interests of physicians, and the issues of causal inference that play a role in elucidating etiology and pathogenesis of diseases.

DEBORAH A. LAWLOR is Professor of Epidemiology at the MRC Integrative Epidemiology Unit at the University of Bristol. Her current research interests focus on developing and applying causal epidemiological and biostatistical methods, to studies of reproductive, perinatal and cardiometabolic health and the links between those. She has a particular interest in pregnancy/birth cohorts and supporting data sharing and team science.

Back to Top | Article Outline

REFERENCES

1. Porta M, Bolúmar F. Caution: work in progress: while the methodological “revolution” deserves in-depth study, clinical researchers and senior epidemiologists should not be disenfranchised. Eur J Epidemiol. 2016;31:535–539.
2. Porta M, Vineis P, Bolúmar F. The current deconstruction of paradoxes: one sign of the ongoing methodological “revolution”. Eur J Epidemiol. 2015;30:1079–1087.
3. Hernán MA. The C-word: scientific euphemisms do not improve causal inference from observational data. Am J Public Health. 2018;108:616–619.
4. Hernán MA. Invited commentary: hypothetical interventions to define causal effects–afterthought or prerequisite? Am J Epidemiol. 2005;162:618–620; discussion 621.
5. Hernán MA, Taubman SL. Does obesity shorten life? The importance of well-defined interventions to answer causal questions. Int J Obes (Lond). 2008;32(3 suppl):S8–S14.
6. Broadbent A, Vandenbroucke JP, Pearce N. Response: formalism or pluralism? A reply to commentaries on ‘Causality and causal inference in epidemiology’. Int J Epidemiol. 2016;45:1841–1851.
7. Pearce N, Lawlor DA. Causal inference-so much more than statistics. Int J Epidemiol. 2016;45:1895–1903.
8. Pearce N, Vandenbroucke JP. Causation, mediation and explanation. Int J Epidemiol. 2016;45:1915–1922.
9. Vandenbroucke JP, Broadbent A, Pearce N. Causality and causal inference in epidemiology: the need for a pluralistic approach. Int J Epidemiol. 2016;45:1776–1786.
10. Krieger N, Davey Smith G. The tale wagged by the DAG: broadening the scope of causal inference and explanation for epidemiology. Int J Epidemiol. 2016;45:1787–1808.
11. Krieger N, Davey Smith G. Response: FACEing reality: productive tensions between our epidemiological questions, methods and mission. Int J Epidemiol. 2016;45:1852–1865.
12. Lawlor DA, Tilling K, Davey Smith G. Triangulation in aetiological epidemiology. Int J Epidemiol. 2016;45:1866–1886.
13. Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965;58:295–300.
14. Snow J. On the Mode of Communication of Cholera. 1855.London: John Churchill.
15. Brunekreef B, Holgate ST. Air pollution and health. Lancet. 2002;360:1233–1242.
16. Stiborová M, Arlt VM, Schmeiser HH. Balkan endemic nephropathy: an update on its aetiology. Arch Toxicol. 2016;90:2595–2615.
17. Hertz-Picciotto I. Rothman KJ, Greenland S, Lash TL. Environmental epidemiology. In: Modern Epidemiology. 2008:Philadelphia, PA: Lippincott Williams & Wilkins; 598–619.
18. Brunekreef B, Baker D, Nieuwenhuijsen M. Baker D, Nieuwenhuijsen M. What is environmental epidemiology? In: Environmental Epidemiology. 1999:Oxford: Oxford University Press; 1–14.
19. McMichael AJ. Plantetary Overload. 1993.Cambridge, UK: Cambridge University Press.
20. Pearce N. Traditional epidemiology, modern epidemiology, and public health. Am J Public Health. 1996;86:678–683.
21. Pearce N. Epidemiology as a population science. Int J Epidemiol. 1999;28:S1015–S1018.
22. Greenland S, Daniel R, Pearce N. Outcome modelling strategies in epidemiology: traditional methods and basic alternatives. Int J Epidemiol. 2016;45:565–575.
23. Kelly FJ, Fussell JC. Air pollution and airway disease. Clin Exp Allergy. 2011;41:1059–1071.
24. Brion MJ, Lawlor DA, Matijasevich A, et al. What are the causal effects of breastfeeding on IQ, obesity and blood pressure? Evidence from comparing high-income with middle-income cohorts. Int J Epidemiol. 2011;40:670–680.
25. Brunekreef B, Stewart AW, Anderson HR, Lai CK, Strachan DP, Pearce N; ISAAC Phase 3 Study Group. Self-reported truck traffic on the street of residence and symptoms of asthma and allergic disease: a global relationship in ISAAC phase 3. Environ Health Perspect. 2009;117:1791–1798.
26. Siemiatycki J, Wacholder S, Dewar R, et al. Smoking and degree of occupational exposure: are internal analyses in cohort studies likely to be confounded by smoking status? Am J Ind Med. 1988;13:59–69.
27. Schwartz J, Bind MA, Koutrakis P. Estimating causal effects of local air pollution on daily deaths: effect of low levels. Environ Health Perspect. 2017;125:23–29.
28. Moore LE, Boffetta P, Karami S, et al. Occupational trichloroethylene exposure and renal carcinoma risk: evidence of genetic susceptibility by reductive metabolism gene variants. Cancer Res. 2010;70:6527–6536.
29. Cohen-Cole E, Fletcher JM. Detecting implausible social network effects in acne, height, and headaches: longitudinal analysis. BMJ. 2008;337:a2533.
30. Neidell M. Information, avoidance behavior, and health the effect of ozone on asthma hospitalizations. J Hu Resour. 2009;44:450–478.
31. Branas CC, Cheney RA, MacDonald JM, Tam VW, Jackson TD, Ten Have TR. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011;174:1296–1306.
32. Pearce N. Global epidemiology: the importance of international comparisons and collaborations. Open Access Epidemiology 2013;1:15.
33. Douwes J, Pearce N. Asthma and the westernization ‘package’. Int J Epidemiol. 2002;31:1098–1102.
34. Straif K, Benbrahim-Tallaa L, Baan R, et al; WHO International Agency for Research on Cancer Monograph Working Group. A review of human carcinogens–Part C: metals, arsenic, dusts, and fibres. Lancet Oncol. 2009;10:453–454.
35. Caplin B, Jakobsson K, Glaser J, et al. International collaboration for the epidemiology of eGFR in low and middle income populations - rationale and core protocol for the Disadvantaged Populations eGFR Epidemiology Study (DEGREE). BMC Nephrol. 2017;18:1.
36. Glymour MM. Oakes JM, Kaufman JS. Natural experiments and instrumental variable analyses in social epidemiology. In: Methods in Social Epidemiology. 2006:New York: Wiley; 429–460.
37. Klasen S, Lechtenfeld T, Meier K, Rieckmann J. Benefits trickling away: the health impact of extending access to piped water and sanitation in urban Yemen. J Dev Effect. 2012;4:537–565.
38. Lechtenfeld T. Why does piped water not reduce diarrhea for children? Evidence from urban Yemen. Courant Research Centre: Poverty, Equity and Growth - Discussion Papers 119, Courant Research Centre PEG. Available at: https://ideas.repec.org/s/got/gotcrc.html
39. Lawlor DA. Commentary: two-sample Mendelian randomization: opportunities and challenges. Int J Epidemiol. 2016;45:908–915.
40. Lawlor DA, Harbord RM, Sterne JA, Timpson N, Davey Smith G. Mendelian randomization: using genes as instruments for making causal inferences in epidemiology. Stat Med. 2008;27:1133–1163.
41. Vineis P. From John Snow to omics: the long journey of environmental epidemiology. Eur J Epidemiol. 2018;33:355–363.
42. Prasad V, Jena AB. Prespecified falsification end points: can they validate true observational associations? JAMA. 2013;309:241–242.
43. Santos Ferreira DL, Williams DM, Kangas AJ, et al. Association of pre-pregnancy body mass index with offspring metabolic profile: analyses of 3 European prospective birth cohorts. PLoS Med. 2017;14:e1002376.
44. Bor J, Moscoe E, Mutevedzi P, Newell ML, Bärnighausen T. Regression discontinuity designs in epidemiology: causal inference without randomized trials. Epidemiology. 2014;25:729–737.
45. Moscoe E, Bor J, Bärnighausen T. Regression discontinuity designs are underutilized in medicine, epidemiology, and public health: a review of current and best practice. J Clin Epidemiol. 2015;68:122–133.
46. Vandenbroucke JP, le Cessie S. Commentary: regression discontinuity design: let’s give it a try to evaluate medical and public health interventions. Epidemiology. 2014;25:738–741.
47. UN AIDS. An Introduction to Triangulation. 2010.Geneva: UN AIDS.
48. Steenland K, Pier B, Baccarelli A, Kogevinas M. Dioxin revisited: developments since the 1997 IARC classification of dioxin as a human carcinogen. Environ Health Perspec. 2004;112:1265–1268.
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.