Presents Can Come in Black Boxes, too

Weiss, Noel S.

doi: 10.1097/01.ede.0000135175.11460.27

From the Department of Epidemiology, University of Washington, Seattle, Washington.

Correspondence: Noel S. Weiss, Department of Epidemiology, University of Washington, Box 357236, Seattle, WA 98195-7236. E-mail:

Article Outline

The value of nonrandomized studies of disease etiology conducted among individuals within a given population—“risk factor” epidemiology—no doubt has been questioned for as long as there have been such studies. However, since 1988, when Alvan Feinstein1 claimed that epidemiologic methods currently in use “need substantial improvement to produce trustworthy scientific evidence,” questioning of this sort seems to have intensified.2–4 In his article5 elsewhere in this issue, Sander Greenland has described 2 bases for the concern with “risk factor” epidemiology: 1) in many studies, there are no strong prior hypotheses for associations that are observed (ie, a “black box” is present), and a considerable number of these associations will in fact not be genuine; and 2) the enumeration of exposure–disease associations can generate concern among many members of the public, and ultimately as much or more harm than good could ensue. However, Greenland points out—as was done in the very first issue of Epidemiology, by persons responding to Feinstein's critique6,7—that even in the absence of a compelling hypothesis, carefully made observations in epidemiologic studies can play a role in the development of theories of etiologic pathways leading to the occurrence of diseases, and so such observations should be welcomed, not shunned. Indeed, the years since the publication of Feinstein's commentary have seen instances in which “black box” epidemiologic studies have identified associations that have led to the development of widely accepted causal hypotheses (eg, the etiologic role of the prone sleeping position in sudden infant death syndrome8) and resulting beneficial interventions (eg, parental education regarding infant sleeping position9).

All of us, whether we are proponents or opponents of “risk factor” epidemiology, share a concern for the excessive (from a scientific point of view) attention paid by the news media to the results of the most recently published or presented epidemiologic research. There seems to be too little attention to the quality of the work or to the aggregate results of all studies of the question at hand. Yet, it would be sad if such public attention to communication among epidemiologists in journals or at meetings discouraged dissemination of (in Greenland's words) “the facts that require explanation by theories.” Cognizant that “eavesdropping” by nonprofessionals does indeed take place, it behooves epidemiologists to follow Greenland's admonition to indeed emphasize the facts and the nature of the study that led to their generation. To the extent that, as a consequence of their work, epidemiologists would like to draw broader etiologic conclusions or suggest policy implications, I would add that these statements should be based on all relevant research, not just on the findings presented by the investigators in a particular report.

Greenland did not address the attack on “risk factor” epidemiology that has been directed, not toward its methods, but rather to the subject of its inquiries. Pearce,10 in advocating that greater attention be paid to potential etiologic factors affecting whole populations, has contended that contemporary epidemiology has been studying “more and more trivial issues, while the major population causes of disease are ignored.” Susser and Susser11 suggest that “in epidemiology today the dominant black box paradigm is of declining utility and is likely soon to be superseded.” What Greenland might have said in response (I trust he will correct me if I am wrong!) is that results that seem trivial today might not be so later, once an etiologic theory that accommodates these results has been formulated; and that seemingly repetitive investigations of previously studied relationships could be required to solidify the data needed to help formulate such a theory. Finally, if Greenland shares my view, he would argue that epidemiologic studies cannot succeed without variation in exposure status. If that variation is present for an entire population across periods of time, or across different populations, we should exploit it in population-level studies to help gauge the exposure's impact on health. If that variation is present among individuals within a population, then a “risk factor” study, even one that must take place within the confines of a “black box,” is in order. These approaches are complementary.

Back to Top | Article Outline


NOEL WEISS has been at the University of Washington and the Fred Hutchinson Cancer Research Center since 1973. He has investigated the epidemiology of gynecologic and other cancers, and he has worked in the areas of clinical epidemiology and epidemiologic methods. He is the author of Clinical Epidemiology: The Study of the Outcome of Illness and (with Thomas Koepsell) Epidemiologic Methods: Studying the Occurrence of Illness.

Back to Top | Article Outline


1. Feinstein A. Scientific standards in epidemiologic studies of the menace of daily life. Science. 1988;242:1257–1263.
2. Skrabanek P. The emptiness of the black box. Epidemiology. 1994;5:553–555.
3. Gary Taubes faces epidemiology. Epidemiology Monitor. 1996;17:2–14.
4. Davey Smith G, Ebrahim S. Data dredging, bias or confounding. BMJ. 2002;325:1437–1438.
5. Greenland S, Gago-Dominguez M, Castelao JE. The value of risk-factor (“black-box”) epidemiology. Epidemiology. 2004;15:529–535.
6. Savitz DA, Greenland S, Stolley PD, et al. Scientific standards of criticism: a reaction to ’Scientific standards in epidemiologic studies of the menace of daily life,’ by A. R. Feinstein. Epidemiology. 1990;1:78–83.
7. Weiss NS. Scientific standards in epidemiologic studies. Epidemiology. 1990;1:85–86.
8. Guntheroth WG, Spiers PS. Sleeping prone and the risk of sudden infant death syndrome. JAMA. 1992;267:2359–2362.
9. Dwyer T, Ponsonby A-L, Blizzard L, et al. The contribution of changes in the prevalence of prone sleeping position to the decline in sudden infant death syndrome in Tasmania. JAMA. 1995;273:783–789.
10. Pearce N. Traditional epidemiology, modern epidemiology, and public health. Am J Public Health. 1996;86:678–683.
11. Susser M, Susser E. Choosing a future for epidemiology: I. Eras and paradigms. Am J Public Health. 1996;86:668–677.
Back to Top | Article Outline

Walk on the Wild Side

“His long life in comparative student loneliness had made him reserved in manner to strangers; but with private friends he was always open, and of sweet companionship. With his increased popularity he became less reserved to strangers, and in the last years of his life he so far threw off restraint as to visit the opera occasionally.” - Benjamin Ward Richardson on John Snow, 1887

Cited By:

This article has been cited 2 time(s).

“Black-Box” Epidemiology
Mejía-Aranguré, JM; Fajardo-Gutiérrez, A; Ortega-Alvarez, M
Epidemiology, 16(3): 418.
PDF (528) | CrossRef
Authors' Response
Greenland, S; Gago-Dominguez, M; Esteban Castelao, J
Epidemiology, 15(5): 527-528.
PDF (91) | CrossRef
Back to Top | Article Outline
© 2004 Lippincott Williams & Wilkins, Inc.