Sure, you are not misreading and this is no printing error. Raj Bhopal, professor of epidemiology, chair of Public Health Sciences at Edinburgh, Scotland, and writer of a popular textbook on epidemiology, has proposed this very idea. He mentions 7 sins of epidemiology, and as one of the remedies he proposes a World Council for Causality (http://www.ete-online.com/content/6/1/6
) . Two things the Council might do, he suggests, would be to:
“....[provide] authoritative statements on epidemiological evidence and [make] recommendations on when and how epidemiological data on associations are ready for application”.
Such a Council
“... could hasten advances, and counter the onslaught of undigested associations that bewilder us and will be multiplying as computerised data mining, data linkage, genetic epidemiology, and grand-scale epidemiology on millions of study participants become commonplace.”
In sum, a Council is needed:
“to counter criticisms about false findings [...], advance epidemiology, and properly engage the public.”
The concerns that prompt Raj Bhopal are shared by many of us. On one hand, there is unwarranted criticism of epidemiology. How often have we heard 'this study is observational, so there is always a potential problem of bias and confounding'. Do people forget that all genetics and all research on infectious disease outbreaks is observational? On the other hand, we are dismayed by unwarranted credulity: 'researchers showed that men who continue leading active sex lives....'. The latter might be replaced by anything that you have to do regularly and that needs a healthy brain and/or body to continue doing it – like taking your statins regularly – which has led to a boom of papers about the associated health benefits of doing so. Papers about seemingly-nonsensical associations are a source of criticism from outside of epidemiology, but also clearly a worry to Raj Bhopal. He revisits the problem in the second edition of his textbook 'Concepts of Epidemiology' (Oxford University Press 2008).
Should we bring some order? Could we help people to arrive at balanced judgements? From a completely different angle, an attempt at structuring our thoughts was made by Paul Rosenbaum, whose name will be forever linked to the Propensity Score. In the first chapter of his recent book on the 'Design of Observational Studies' (Springer, 2010). Rosenbaum lists 7 – seven again! – basic ingredients of epidemiologic studies. Each ingredient is discussed in the context of three types of studies: a randomized experiment, a better observational study and a poorer observational study. For example, under the heading 'How were treatments assigned', Rosenbaum writes that in the better observational study,
"...circumstances for the study were chosen so that treatment seems haphazard, or at least not obviously related to the outcomes subjects would exhibit under treatment or under control”.
In contrast, the poorer observational study gives
“little attention … to the process that made some people into treated subjects and others into controls”.
A troubling point is that Bhopal’s and Rosenbaum’s 7 points do not overlap except for the question of whether treatment and control groups are comparable. Their outlooks are different, one writer being a public health person and the other a statistician writing about study design. By the way, checking the comparability of groups can become a recipe for disaster if applied to case-control studies: when persons not familiar with this design demand that cases and controls are comparable in 'all respects' save disease. That is impossible: even in randomized trials the patients that develop the study outcome (in either treatment arm) will be different in many respects from those who do not.
I was involved in drafting STROBE, the guidelines about reporting observational studies ( http://www.strobe-statement.org
). It has often crossed my mind whether we should have additional guidelines to help people to think in a structured way about the credibility of an epidemiologic finding. Guidelines for reporting like STROBE are, unfortunately, often believed to say something about the validity and therefore credibility of a study. If nothing else, having a separate set of guidelines to help people interpret
observational studies, might clarify the difference with guidelines for reporting. The greatest use of guidelines for interpretation might be by persons who are not professional epidemiologists and who are bewildered by all the arguments that can be used to either deride or bolster an epidemiologic finding. In my own work, I have come not much further than paraphrasing Rosenbaum’s treatment assignment rule, but I have brought in a component that neither Bhopal nor Rosenbaum clearly mentions: the independent existence of evidence about an hypothesis, possibly formalized as prior odds, strongly determines our belief in a finding. The strength of the prior may even explain why randomized trials look more credible than observational studies . So, in my teaching, I am stuck with a mere 2 rules.
Should we worry about criticisms of epidemiology that we publish too many associations? Is it possible to educate people in thinking about the credibility of observational studies? Or is any judgment totally ad hoc, depending as much on subject-matter knowledge as on formal methods? A Dutch cartoon shows two people staring at a computer and commenting: “The probability that almost all professors of statistics agree... is, of course, very small.”
If you like to comment, Email me directly at firstname.lastname@example.org or submt your comment via the journal which requires a password protected login.
 Vandenbroucke JP. Observational research, randomized trials and two views of medical science. PLoS Med. 2008 March; 5(3): e67
© Jan P Vandenbroucke, 2010