Reporting Instrumental Variable Analyses

To the Editor:

Swanson and Hernán¹ discuss the reporting of instrumental variable analyses and propose a step-by-step checklist. Two phases can be distinguished in their suggested steps. The first comprises discussion of the three main instrumental variable assumptions. The second phase concerns estimation of the effect; Swanson and Hernán propose that authors should first discuss whether in the population or the effect in the compliers is of interest, then estimate bounds for the effect, and finally, if appropriate, justify an additional assumption that allows estimation of a point estimate.

We suggest an intermediate reporting step between these two phases: a presentation of the distribution of the outcome across instrumental variable values. This amounts to a crude analysis of the effect of levels of the instrumental variable on the outcome. This step resembles the form of the usual epidemiologic study in which one or more groups are contrasted, at various levels of exposure frequency.

For a dichotomous instrument, the presentation of the outcome across values of the instrument is straightforward. The comparison of the outcome at the two values of the instrument gives an effect estimate that can be thought of as similar to an intention-to-treat effect in a randomized trial. Davies et al² provide a specific reporting suggestion for the situation in which instrument, treatment, and outcome are all dichotomous—namely tabulation of frequencies of all combinations of instrument treatment and outcome. An example of how our suggested step can be reported if the instrument is continuous is presented in a article by Stukel et al.³ They performed an instrumental variable analysis that used regional cardiac catheterization rates as an instrument to investigate the effect of cardiac catheterization (as a marker of intent to treat invasively) on long-term survival in acute myocardial infarction. They provide a table (Table 4 in their article) with baseline characteristics as well as the outcome (mortality) across quintiles of regional cardiac catheterization rate. Direct comparison of the outcome across instrument quintiles shows a decrease in mortality with increasing regional cardiac catheterization rate. The display of baseline characteristics across the same quintiles allows the reader to evaluate how comparable the patient characteristics in these quintiles are (third instrumental variable assumption: the instrument is independent of confounders²).

Such an additional step in the reporting of instrumental variable analyses provides a presentation of the data before a decision is made about whether to report bounds only or a point estimate. An intermediate analysis might be done on these data—for example, by doing a comparative analysis of a dichotomous instrumental variable, or by directly contrasting the lowest and highest categories of a continuous instrumental variable distribution. The validity of this comparison does, of course, depend on the three main instrumental variable assumptions, and violations of these assumptions will lead to bias. However, the bias amplification that can occur when using standard instrumental variable methods to obtain effect estimates⁴ will not affect the comparison of the outcome across strata of the instrument. Falsification tests of the third instrumental variable assumption are discussed by Swanson and Hernán¹ and by Davies et al.² Showing the distribution of patient characteristics across values of the instrument (in parallel with the distribution of the outcome across values of the instrument) may also aid in detecting potential violations of this assumption.

Anna G. C. Boef

Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, The Netherlands, [email protected]

Olaf M. Dekkers

Departments of Clinical Epidemiology and Endocrinology and Metabolic Diseases, Leiden University Medical Centre, Leiden, The Netherlands

Saskia le Cessie

Departments of Clinical Epidemiology and Medical Statistics and Bioinformatics, Leiden University Medical Centre, Leiden, The Netherlands

Jan P. Vandenbroucke

Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, The Netherlands