The inherent limitations of ecologic studies as the basis for assessing causal relationships are well-known,1 starting with the “ecologic fallacy”—the potential lack of correspondence between associations identified in aggregate population groups and on the individual level. Ecologic studies are often labeled as “hypothesis-generating” studies, a questionable phrase if taken literally: If the study really generated the hypothesis, then why were the data assembled to conduct the study in the first place? The term is intended to deflect criticism, asking the reader to set the bar lower and expect less because it is just an ecologic study. A more honest label would be “susceptible to erroneous or misleading results.” Ecologic studies often serve as a useful step in the evolution of research, providing preliminary information that requires more definitive research designs to make progress. The hypothesis exists, perhaps based only on intuition or speculation, and ecologic studies provide an efficient, preliminary, imperfect way to address it. The key question (as for any type of study) is to know when the approach is useful and when it is not, and there are some simple determinants of the contribution that can be expected from ecologic studies.
First, the baseline level of knowledge against which the ecologic study is being measured has to be sufficiently limited for the contribution to be meaningful. The time- and cost-efficiency of using existing data for an ecologic study, relative to studies of individuals, means that some topics that are not deserving of more demanding research designs will be worthy of ecologic study. Prior to the report by Wheeler and colleagues on radon and skin cancer,2 the published literature consisted of a single, less sophisticated ecologic study3 and theoretical calculations of possible risk,4 setting the stage for a well-designed, carefully analyzed ecologic study to provide a substantial increment in knowledge. Nonmelanoma skin cancer has been studied far less extensively than other types of cancer, largely because it is rarely included in cancer registries. Therefore, the availability of comprehensive, high-quality data on nonmelanoma skin cancer across a large geographic area in which baseline risk is elevated provides a unique opportunity that would not be applicable to other cancer types that have been more widely registered and subjected to more intensive study. In contrast, such an approach would add little to our understanding of radon and lung cancer, for example, which has been rigorously examined in many individual-level studies.5
Second, the exposure needs to have marked variation across geographic units, sufficiently dramatic to overcome the inevitable blurring that results from aggregation. The ideal, of course, is for all the variation in exposure to be between units and none within, at which point the ecologic approach is as effective in assigning exposure as a study of individuals would be. To the extent that location is a direct determinant of exposure, in the case of radon arising as a function of local geology, the balance of within- and between-area variation is likely to be more favorable. When exposure variation arises as an indirect result of geographic tendencies that are less dominant than geology (eg, residential selection or location of industry), ecologic studies hold less promise.
Third, because confounding is notoriously difficult to control statistically in ecologic studies,1 research settings should be identified in which confounding is unlikely or modest in magnitude. Because variation in radon is a function of geology, and there is no direct correlate of geology that affects the desirability of housing or socioeconomic conditions, vulnerability to ecologic confounding is limited. In contrast, the relationship between proximity to hazardous-waste sites, for example, and health outcomes is far more vulnerable to ecologic confounding by social determinants of health.
Fourth, the opportunity to include health conditions that can serve as positive and negative controls strengthens ecologic evidence for the health outcome of interest. Even with all the other favorable features in place, ecologic studies may be too blunt an instrument to detect associations, making null findings particularly weak as evidence against an association being present. To the extent that there are known effects of the exposure and the study is able to confirm those expected effects, null results for other outcomes are more persuasive. Likewise, if there are health outcomes known not to be affected by the exposure and if one can document that the ecologic approach confirms those null results, evidence for a positive association with the outcome of interest is more convincing. Health outcomes that serve as positive and negative controls basically confirm the sensitivity and specificity of the design. With the consideration of the 3 types of skin cancer—melanoma, basal cell, and squamous cell—the investigators were able to make informative contrasts. The finding that radon was associated with rates of squamous cell skin cancer (predicted to be the most vulnerable given the depth of radon penetration) and not with basal cell skin cancer or melanoma enhances the credibility of the results.
Looking beyond radon and skin cancer to the merits of ecologic studies in environmental epidemiology more generally, there are some potential advantages of ecologic studies over studies of individuals, in addition to the shortcomings. Ecologic designs circumvent concerns with nonresponse and subjectively biased reporting of exposure or disease information. Furthermore, the geographic basis for differences in exposure may be more accurately identified and free of bias than the individual determinants within geographic areas. For example, exposure to drinking water disinfection by-products is a function of location (the water source and its treatment for disinfection) and behavior (water consumption, bathing and showering, etc),6 but the spatial determinant is much more efficiently identified and likely to be more accurate than variation due to individual behaviors, which are difficult to recall in sufficient detail. Individual measurements of biomarkers are integrative across exposure sources, a distinct strength, yet are also susceptible to distortion by the disease process and a reflection of general metabolic characteristics that may also determine internal dose of other contaminants and result in confounding.7,8 However, even when ecologic (spatial) exposure measures are useful, combining those with individual assessment of disease and covariates would always be preferable to a fully ecologic study design.
Convergent evidence from ecologic and individual studies, with their distinctive strengths and limitations, can be beneficial if the circumstances for ecologic studies happen to be favorable. Beyond simply making the best possible use of the data in hand, contributory ecologic studies have to begin with the identification of favorable conditions based on current level of knowledge, data resources, and within- and between-area variability of exposure.
1. Morgenstern H. Ecologic studies. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2008:511–531.
2. Wheeler BW, Allen J, Depledge MH, Curnow A. Radon and skin cancer in south-west England: an ecologic study. Epidemiology. 2012;23:44–52.
3. Etherington DJ, Pheby DF, Bray FI. An ecologic study of cancer incidence and radon levels in South West England. Eur J Cancer. 1996;32A:1189–1197.
4. Henshaw DL, Eatough JP. The theoretical risk of non-melanoma skin cancer from environmental radon exposure. J Radiol Prot. 1995;15:45–51.
5. Krewski D, Lubin J, Zielinski JM. Residential radon and risk of lung cancer: a combined analysis of 7 North American case-control studies. Epidemiology. 2005;16:137–145.
6. Nieuwenhuijsen MJ, Toledano MP, Elliott P. Update of chlorination disinfection by-products: a review and a discussion of its implications for exposure assessment in epidemiological studies. J Expos Anal Environ Epidemiol. 2000;10:586–599.
7. Longnecker MP. Pharmacokinetic variability and the miracle of modern analytical chemistry (commentary). Epidemiology. 2006;17:350–351.
8. Wolff MS, Anderson HA, Britton JA, Rothman N. Pharmacokinetic variability and modern epidemiology—the example of Dichlorodiphenyltrichloroethane, body mass index, and birth cohort. Cancer Epidemiol Biomarkers Prev. 2007;16:1925–1930.