To the Editor:
Once a healthcare intervention is proven effective, we expect clinicians to use the effect estimates to appropriately inform their patients. To reliably support causation, the two comparison groups should have exchangeability, that is, they should have equal expected outcome means (conditional on covariates for observational studies) in the absence of treatment. In the traditional placebo-blind, randomized controlled trial (RCT), randomization eliminates shared causes of treatment and outcome. Furthermore, patient and assessor blinding minimize the probability that outcomes will differ due to nontreatment-mediated effects (eg, psychobiological and adherence effects secondary to treatment assignment; Figure). Exchangeability can occur through several blinding procedures. This letter suggests that each blinding method is associated with its own causal effect, and one should therefore match the blinding method to the clinical context of interest.
The most common clinical context is when patients are told they are being “prescribed a treatment that is effective in general, although not everyone may benefit from it.” This context is different from a traditional placebo-blind RCT, where participants are told they may or may not receive active treatment. Although ethical challenges exist, a “deception” RCT design can mimic the clinical context. Here, participants are told they will receive active treatment, but 50% of patients are randomized to inactive treatment (mimicking the context of ineffective treatment) and 50% to active treatment (mimicking the context of effective treatment). The only difference between the two RCTs is the method of patient blinding; all other causal inference requirements are the same.
In one study on minor pain, researchers randomized unknowing participants to either type of RCT,1 where the anti-inflammatory medication was replaced with a placebo for one dose. The intention-to-treat analysis treatment-effect estimate differed in these two RCTs,1 perhaps because the causal effect was truly different or perhaps because six of 24 patients approached for the traditional RCT declined to participate. In studies on smoking cessation2 and caffeine,3 where estimates of treatment effect were also dependent on blinding methods, nonparticipation could not have caused the differences because patients were randomized to the type of RCT only after agreeing to participate (deception RCTs of effective treatment vs. placebo and of ineffective treatment vs. placebo, and traditional placebo-blind RCT).
These observed differences are expected if there is an interaction between the active treatment and the participants’ baseline psychological state. Participants expecting caffeinated coffee may have anticipatory physiological changes (eg, increased heart rate) even if they receive decaffeinated coffee. If the effect of actual caffeine depends to some degree on anticipatory physiological changes, regardless of whether the outcome is objective or subjective, the causal effect will depend on the blinding context (eFigure, http://links.lww.com/EDE/A705).
All study designs have limitations and untestable assumptions. In idealized traditional placebo-blind RCTs,4 one assumes the obtained causal-effect estimate is the same as that from a blinding method matching the target clinical context. The deception RCT is unethical for most healthcare questions. An unblinded RCT mimicking the clinical context has the untestable assumption that the outcome is unaffected by psychobiological effects (but remains unbiased if one is interested in the total causal effect including psychobiological and behavioral effects). Well-conducted prospective observational studies matching the clinical context of interest have an untestable assumption of no unknown or unmeasured confounding.5 With untestable assumptions present in each study design, it is the relative probability and magnitude of violations of these assumptions that determine which study design provides the least biased estimate.6,7 Is it therefore justified to accord one study design as superior to another8 when the probability and magnitude of these biases is clearly study specific?
Centre for Clinical Epidemiology
Lady Davis Institute for Medical Research
Jewish General Hospital
1. Bergmann JF, Chassany O, Gandiol J, et al. A randomised clinical trial of the effect of informed consent on the analgesic activity of placebo and naproxen in cancer pain. Clin Trials Metaanal. 1994;29:41–47
2. Hughes JR, Gulliver SB, Amori G, Mireault GC, Fenwick JF. Effect of instructions and nicotine on smoking cessation, withdrawal symptoms and self-administration of nicotine gum. Psychopharmacology (Berl). 1989;99:486–491
3. Kirsch I, Rosadino MJ. Do double-blind studies with informed consent yield externally valid results? An empirical test. Psychopharmacology (Berl). 1993;110:437–442
4. Turner RM, Spiegelhalter DJ, Smith GC, Thompson SG. Bias modelling in evidence synthesis. J R Stat Soc Ser A Stat Soc. 2009;172:21–47
5. Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15:615–625
6. Shrier I, Boivin JF, Steele RJ, et al. Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of the underlying principles. Am J Epidemiol. 2007;166:1203–1209
7. Shrier I. Structural approach to bias in meta-analyses. Res Synth Methods. 2012;2:223–237
8. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011:1–6