Secondary Logo

Journal Logo


Optimal Referent Selection Strategies in Case-Crossover Studies: A Settled Issue

Mittleman, Murray A.

Author Information
doi: 10.1097/01.ede.0000183170.92955.25
  • Free

In this issue of Epidemiology, Janes et al1 present a comprehensive review of referent selection strategies in case-crossover analyses of air pollution data. This review is timely, given the increasing popularity of the case-crossover approach, and makes an important contribution to the field. As for other substantive areas of epidemiology, concerns over the validity of estimates of the short-term health effects of air pollution are paramount. Thus, study design decisions and analytic approaches should be engineered to maximize efficiency to the extent possible without introducing inordinate bias. The authors did an excellent job of summarizing insights regarding the importance of careful selection of referent periods to minimize potential biases, including those that arise from time trends in exposure, truncation of the air pollution data at the start and end of the exposure time series, and violations of the assumptions inherent in the conditional logistic likelihood that result in “overlap bias.”

The arguments put forth by Janes and colleagues convincingly show that, although other approaches may be valid under certain circumstances, the time-stratified selection strategy is more generally valid than any of the alternatives thus far proposed. Thus, this strategy should be considered the de facto standard approach to the analysis of data arising in studies of the short-term effect of air pollution and weather.

The caution against “model shopping” is an extremely important one, but it must be distinguished from the concept of sensitivity analysis. Conducting several analyses and “cherry picking” results (ie, presenting only the subset of results that look the most “interesting” to the investigator) clearly is problematic and is likely to lead to the presentation of spuriously extreme effect estimates.

However, assuming one considers only approaches that have been shown to be generally valid, it is reasonable for investigators to conduct sensitivity analyses. For example, when conducting a time-stratified case-crossover analysis, the length of the time strata should be chosen subject to the best judgment of the analyst. In studies of short-term health effects of air pollution, investigators have most commonly chosen strata defined by calendar month.2–6 This is clearly a convenient choice that is easy to implement. However, an implicit assumption is that time-dependent confounders, such as season, or respiratory illness outbreaks are relatively constant within a calendar month. It is possible that strata of 2 or 3 weeks duration may better control for such confounding or that longer periods of up to 6 weeks might control confounding as well, but be more efficient. Sensitivity analyses may be helpful in understanding the tradeoffs involved in making this decision.

Although the importance of valid and efficient referent selection strategies as shown in this review should not be minimized, in empirical evaluations the degree of bias is often small. For example, a recent paper by Sullivan et al5 showed that, compared with the time-stratified approach, use of a referent selection strategy known to be susceptible to overlap bias did not materially alter the empirical results of a large study of the short-term effect of particulate air pollution on myocardial infarction incidence in the King County, Washington. Nonetheless, given the potential for bias now known to exist with other approaches, the time-stratified approach to referent selection is clearly the approach of choice.

It also is important to note that there are other sources of bias operating in studies of short-term effects of air pollution that are often of even greater importance. Issues of exposure misclassification, misclassification of the timing of outcome events, and case definitions with low sensitivity or specificity are common in the large studies utilizing administrative data that continue to be common in the field. Studies where case events are assigned to the calendar date on which the patient presented to hospital or died, and exposure is based on the daily level of particulate matter, do not take into account that the event might have begun any time between 0:00 and 23:59 hours. At best, these studies are misclassifying exposure by a half-day on average. Furthermore, if there is delay between the onset of the pathophysiologic process and the clinical presentation of the outcome event, then the relative timing of the exposure and the event will be even more problematic. If one assumes that there is an average of a 4- to 6-hour delay between coronary plaque rupture and admission to the hospital for myocardial infarction, then approximately 17% to 25% of cases will be assumed to have occurred on the incorrect calendar day.

As elegantly summarized by Janes et al, the issue of how to sample referent periods in case-crossover studies of air pollution is clearly answered and a standard approach is available that can easily be implemented using standard software tools. Now it is time for the field to move on to minimizing other, potentially much larger sources of bias in studies of the short-term effects of air pollution. To accomplish this task, studies with more complete case characterization, more precise data on the timing of symptom onset, and more appropriate exposure assessment are needed.


MURRAY MITTLEMAN is director of the Cardiovascular Epidemiology Research Unit at the Beth Israel Deaconess Medical Center, Associate Professor of Medicine at the Harvard School of Medicine, and Associate Professor of Epidemiology at the Harvard School of Public Health. His applied work is primarily in the areas of cardiovascular disease and injury epidemiology. He has worked on methodological issues pertaining to the case-crossover study design since its inception.


1. Janes H, Sheppard L, Lumley T. Case-crossover analyses of air pollution exposure data: Referent selection strategies and their implications for bias. Epidemiology. 2005;16:717–726.
2. D'Ippoliti D, Forastiere F, Ancona C, et al. Air pollution and myocardial infarction in Rome: a case-crossover analysis. Epidemiology. 2003;14:528–535.
3. Levy D, Sheppard L, Checkoway H, et al. A case-crossover analysis of particulate matter air pollution and out-of-hospital primary cardiac arrest. Epidemiology. 2001;12:193–199.
4. Schwartz J. The effects of particulate air pollution on daily deaths: a multi-city case crossover analysis. Occup Environ Med. 2004;61:956–961.
5. Sullivan J, Sheppard L, Schreuder A, et al. Relation between short-term fine-particulate matter exposure and onset of myocardial infarction. Epidemiology. 2005;16:41–48.
6. Sunyer J, Basagana X, Belmonte J, et al. Effect of nitrogen dioxide and ozone on the risk of dying in patients with severe asthma. Thorax. 2002;57:687–93.
© 2005 Lippincott Williams & Wilkins, Inc.