The aim of etiologic research is often to determine the causal pathway between an exposure and a disease. For example, if a DNA characteristic (a haplotype, a single nucleotide polymorphism or a gene) is associated with a particular disease, it must act via biochemical pathways to lead to the development of disease. In the search for a mechanistic explanation, one would like to investigate to what extent the relationship between the DNA characteristic and the disease can be explained by various biochemical mediating variables. The classic approach to explore this relationship is to condition on the intermediate factor in the causal path as advocated by Susser1 and by Baron and Kenny.2 If a genetic effect is still apparent after adjustment for the intermediate factors, researchers may conclude that the effect of the gene is not completely mediated and there must be another pathway.
However, this approach may lead to incorrect conclusions. For example, if the intermediate is measured with error, a regression model with the intermediate as one of the independent variables yields regression coefficients that are not causally interpretable.3,4 Another situation leading to biased estimators of causal effects is the presence of unadjusted confounding of the relationship between the intermediate variable and the outcome.5 – 7
Several authors have described methods to disentangle direct and indirect effects7 – 15 and the effect of confounding between intermediate and outcome on the direct effect estimates. Our focus here is on bias caused by measurement error in the intermediate. This measurement error may be nondifferential or influenced by exposure or outcome. Subsequently, we estimate the magnitude of the bias in several situations. This leads to a number of easy-to-calculate formulas that enable the researcher who is confronted with the possibility of a direct causal effect to assess the extent to which these biases may affect the conclusions.
The example discussed is the Leiden Thrombophilia Study, a case-control study that has evaluated the effects of various genetic and behavioral factors on the occurrence of venous thrombosis. Cases were patients with a first thrombosis, and control subjects were age-and-sex matched. In a paper by Koster et al,16 the role of clotting factor VIII in the relationship between blood group (O versus non-O) and venous thrombosis was reported for 301 case and control subjects. Patients with non-O blood groups have an increased risk of thrombosis, a result found in the Leiden study16 and other studies.17,18 Blood group O is associated with lower levels of factor VIII than blood group non-O. Factor VIII is a coagulation factor and a strong risk factor for thrombosis. Biologically, it is plausible that the effect of blood group on thrombosis is partly or completely mediated through factor VIII.
Because of the case-control design of the Leiden study, factor VIII was measured for patients after the occurrence of thrombosis. The odds ratio for thrombosis for subjects with blood group non-O versus O was 2.0 (95% confidence interval [CI] = 1.4–2.8), consistent with other studies. To study the mediating effect of factor VIII, we fitted a logistic regression model with blood group, factor VIII and the matching variables age and sex as covariates. This yielded an adjusted odds ratio of 1.5 (1.0–2.1) for blood group. This remaining positive association suggests a direct effect of blood group on thrombosis via a different pathway. As shown in this paper, there are a number of reasons why this estimate of the direct-effect odds ratio of blood group on thrombosis could be biased, and we explore the impact of the various sources of potential bias due to different types of measurement error of the intermediary variable factor VIII.
POTENTIAL SOURCES OF BIAS IN ESTIMATES OF DIRECT AND INDIRECT EFFECT
We are interested in the causal effect of an exposure X on disease outcome Y, and the extent to which this is mediated by an intermediate variable M. This situation is depicted in a causal diagram in Figure 1A. The indirect effect of X is the effect that passes through M, while the direct effect is reflected by the arrow from X to Y. There could be known baseline confounders for the exposure-outcome relation or for the relation between mediator and outcome. In Figure 1B, we add the effect of the known confounders, depicted by C. It is possible that other factors influence M, but these factors are assumed to be independent of X and conditionally independent of Y given M; therefore, these are not depicted. In the Leiden Thrombophilia Study, X is blood group (non-O versus O), Y is the occurrence of venous thrombosis, and M is factor VIII level.
Figure 2 shows various situations where estimates of the direct and indirect effect of X on Y can be biased. (For ease of interpretation, we have left out the effect of the known confounders C in the figures.) Figures 2A–F explore various types of measurement error in the intermediate. In Figure 2G, the true value of the intermediate is known, but the relationship between mediator and outcome is confounded. We discuss each of these situations in detail in the next section.
We focus on a binary outcome variable, while the exposure variable may be either binary or continuous. In the causal framework, there are various definitions of direct and indirect effects, according to whether the direct effect is controlled or natural.11 VanderWeele and Vansteeland14 have extended the causal definitions of direct and indirect effect to the odds-ratio scale, which we will use in this paper. These authors show that, under certain assumptions, regression techniques can be used to estimate direct- and indirect-effect odds ratios. There should be no unmeasured confounding of the exposure-outcome relation and the mediator-outcome relation when estimating controlled direct-effect odds ratios. Furthermore, to identify natural direct and indirect effects, no unmeasured confounding of the exposure-mediator relation should be present, effects of the exposure should not confound the mediator-outcome relation, and the outcome Y should be rare.
In this paper, we assume a logistic model for the outcome Y without interaction between exposure and mediator:
and a linear regression model for the intermediate variable M:
where M is normally distributed with residual standard deviation σM, and C is a vector of baseline confounders. If these models are correct, the direct-effect odds ratio for a one-unit increase of X, given the set of confounders is then equal to exp(β1). This way of estimating direct effects is common in epidemiology and sociology and is often referred to as the “Baron-Kenny”2 approach to mediation.
Under these assumptions, it is possible to derive simple bias formulas under measurement error for the remaining direct effect. In the next subsections, we approximate the bias in the observed regression coefficient for X for the various types of measurement error depicted in Figures 2A–F; additionally, we explain the situation where there is unmeasured confounding between the mediator and the outcome (Fig. 2G).
Situation 1: Classic Measurement Error in M
One explanation for an estimated remaining effect of the exposure X is measurement error in M. Several types of measurement error are discussed by Hernán and Cole,5 using directed acyclic graphs (DAGS). Figure 2A shows the simplest situation, where measurement error is independent of exposure and outcome. This can be due, for example, to error in the laboratory measurement, depicted as U. Instead of observing M, we observe M*. The DAG of Figure 2A shows that conditioning on M* does not completely block the path between X and Y, meaning that X and Y are still indirectly connected via M. To estimate the size of bias in the observed direct association, we assume that M* = M + U, with U, a random variable, independent of M, X, and C, and normally distributed with mean zero and variance σu 2. This implies that the variance of M* given X and C equals
This type of error is called classic measurement error.3 Using M* instead of M in the logistic model yields:
The relationship between the coefficients of model 2 and model 1, with M, X, and C as covariates, can be approximated using the formulas for multiple linear regression with one covariate measured with error given in Carroll et al3 p. 52 because these regression calibration formulas are also approximately valid for logistic regression.3 p. 91 This yields that the following relationship approximately holds:
The factor λ, which attenuates the regression coefficient β2, measures the precision of M*. This ratio of the variances of M and M* is often called the reliability ratio. Note that the variances of M and M* are conditional on X and C. From this, it can be derived that the bias in the estimated direct effect is
The bias is positive if the exposure affects the intermediate in the same direction as the mediator affects the outcome. In case of opposite effects, the measurement error will result in a remaining association that is negatively biased.
This bias formula and its extensions in the next sections can be used to adjust the estimate of the log-odds ratio for measurement error. This requires an estimate of the reliability ratio. To obtain such an estimate is often not difficult: reliability ratios have been published for many laboratory measurements. An approximate CI for the adjusted estimate can be obtained by applying formula (6) to the lower and upper CI of the original estimate, but this does not take into account that the parameters in formula (6) are themselves being estimated from the data. Using bootstrapping is therefore a more accurate way to obtain CIs.
Situation 2: Nondifferential Intraindividual Variation Over Time
The intermediate may not be measured exactly at the moment that is relevant for occurrence of the event, but earlier or later. In this situation, the measured intermediate is affected by a combination of permanent factors and temporary factors that briefly affect the levels of the intermediate. In the Leiden Thrombophilia Study, blood group has a permanent effect on factor VIII while, for example, physical stress temporarily increases factor VIII. Figure 2B depicts this situation, with M as the true intermediate that determines the outcome Y. The measured intermediate M* is affected by the same time-invariant components, but by different temporary effects and measurement error. The node UM* reflects all temporary factors that determine the value of M*, whereas the node UM indicates the temporal effects influencing M.
To approximate the regression coefficients when M* is used instead of M, we assume that:
where M∼ represents the time-invariant part of M that depends on X and C. We assume that M∼, UM, and UM* are independent random variables, with M∼ ∼ N (α0 + α1X,+ α2 t C, σM 2), UM* ∼ N (0, σU 2), and UM ∼ N (0, σU 2).
This situation is a mixture of classic measurement error and Berkson error, described in Carroll et al.3 p. 51 There is classic measurement error because M* is measured instead of M∼, and Berkson error because of the extra component UM that influences M. In linear regression, Berkson error does not bias the regression coefficients, which also holds approximately for logistic regression when the disease probability is small.19 Therefore, the same approximations as in situation 1 can be used, and the log-odds ratio for X after adjusting for M* is approximately equal to:
Reformulating the results in term of the reliability ratio (5) gives:
The reliability ratio depends on the time between measurements, and decreases if the time interval is larger, resulting in a larger bias in the association between X and Y.
Situation 3: The Exposure Affects the Measurement of the Intermediate
In the first 2 situations, measurement error in the intermediate is independent of the exposure and the outcome. Figure 2C shows a more general situation where the measured intermediate is affected by the exposure. A situation like this could occur when factor VIII is measured by different instruments in subjects with blood group O and non-O, or when the biochemical measurement is influenced by blood group. The extra bias of systematic differences in measurements between exposed and unexposed subjects can easily be derived under the assumption that M* = M + γ0 + γ1X + U.
Here, unexposed subjects have a systematic bias in their measurements equal to γ0, whereas γ0 + γ1 is the bias in the measurements for exposed subjects. The component U reflects random measurement error. If we assume that U is normally distributed with mean 0 and variance σu 2 , it is straightforward to show (see eAppendix, http://links.lww.com/EDE/A582) that the bias in the regression coefficient for X is approximately
with λ the reliability ratio (5). The first part of this formula is the result of the systematic difference between exposed and unexposed subjects. A positive systematic difference will negatively bias the regression coefficient, whereas a negative difference produces positive bias. Note that α1 in (8) equals
the regression coefficient for X in the linear regression model
Situation 4: Differential Intraindividual Variation Over Time
Figure 2D shows a situation that is closely related to situation 3. Here, there is a systematic difference between the measured intermediate and the true intermediate because the intermediate is measured at a different time point, and the effect of the exposure on the intermediate varies over time. This could happen, for example, when the effect of an exposure cumulates over time or when the exposure increases the intermediate only for a short time period. The effect of this differential error is estimated in the same way as situation 3. Assuming that E[M* − M|X, C] = γ0 + γ1X, the bias in the regression coefficient for the exposure is also given by formula (8).
Situation 5: A Trigger That Interacts With X and Influences M
Again in Figure 2E, M* is measured at a different point in time. The causal intermediate M is affected by additional error U that is a result of both X and a temporal factor T (the “trigger”). An example of a trigger is air travel, which may temporarily increase factor VIII levels.20 If the exposure X has no effect on U, no association between X and Y via M remains after adjusting for M* because conditioning on M* still blocks the path between X and Y through M. However, if the effect of T differs between exposed and unexposed subjects, the path remains open. In our example, this could occur when air travel leads to a stronger rise in factor VIII levels in non-O subjects. In the eAppendix (http://links.lww.com/EDE/A582), we calculate the extra effect of a trigger for various distributions of T. If T and X are binary and M is increased by a level c in the presence of both T and X, and if the disease is rare, then the bias in the log-odds ratio for X after adjusting for M* is approximately:
If the trigger increases levels of M (c > 0) and the effect of M on Y is positive, the bias in the remaining association between X and Y is positive. For simplicity, we do not account here for possible measurement error or extra day-to day variation in M*. If needed, the extra bias due to random measurement error can be handled as in situation 1 to 4 (see eAppendix, http://links.lww.com/EDE/A582).
Situation 6: The Outcome Affects the Intermediate (Post hoc Phenomenon, Reverse Causation)
Measuring an intermediate after the event can lead to a “post hoc” phenomenon, if the disease itself influences the levels of the measured intermediate. For example, the presence of a persisting thrombus could lead to a procoagulant state with increased factor VIII levels after thrombosis. Figure 2F depicts this situation. Conditioning on M* will not completely block the path from X to Y via M. This leads to a biased estimate of the direct effect of X on Y. The size of this bias can be calculated assuming that:
with γ0 the average bias in the measured intermediate for control subjects, γ0 + γ1, the average bias for cases, and U random measurement error. White21 calculates the effect of differential measurement error on the observed odds ratio for one continuous covariate, assuming that U is normally distributed with common variance σu 2 for cases and control subjects. These results can be extended, and in the eAppendix (http://links.lww.com/EDE/A582), we show that approximately:
The first part of the expression for β1 * is the effect of the systematic difference between the measurements of cases and control subjects, whereas the second part is the effect of the random component. If the occurrence of an event increases M* (γ1 > 0) and X positively affects M (α1 > 0), this results in a negative effect on β1 * . The effect is positive if the disease lowers the value of the mediator.
Situation 7: A Confounder That Influences Both M and Y
Unmeasured confounding can also lead to bias in the estimated direct effect.5 – 7 Both the relation between exposure and outcome and the relation between mediator and outcome can be confounded. The effect of confounding in the mediator-outcome relation is often overlooked. Figure 2G shows a simple DAG to illustrate this phenomenon. Here, unmeasured variables, depicted by U, affect both the value of the intermediate M and the outcome Y. In Figure 2G, M is a collider, and conditioning on M opens a backdoor path from X via U to Y, implying that the remaining association between X and Y is estimated with bias. Bias formulas for sensitivity analysis for controlled and natural direct-effect and indirect-effect odds ratios have been derived by VanderWeele.13
APPLICATION TO DATA OF THE LEIDEN THROMBOPHLIA STUDY
We now return to the Leiden Thrombophilia Study,16 the results of which are summarized in the Table . In this study, blood group affected factor VIII, which in turn increased the risk of thrombosis. Factor VIII was measured in a laboratory-standardized way in international units per liter (IU/L), with values that ranged between 50 and 182 IU/L. The odds ratio (OR) of thrombosis for blood group non-O versus O, obtained from a logistic regression with adjustment for the matching variables sex and age, was 2.0 (95% CI = 1.4–2.9). To study the mediating effect of factor VIII, a second logistic regression model was fitted with factor VIII as extra covariate, yielding an adjusted odds ratio of 1.5 (1.0–2.1).
Before we conclude that blood group affects the risk on thrombosis through other pathways than factor VIII, we explore the effect of the mechanisms depicted in Figure 2. As the study is a case-control design, subjects are selected in the study based on the outcome, and factor VIII was measured for patients after the occurrence of the thrombosis. Because the odds ratio is invariant under change of sampling scheme, the formulas derived in the previous section still hold.22 The relation between the mean intermediate and the exposure and confounders cannot be estimated directly from the data, but in case of a rare disease, it can be approximated using only the control subjects.
We explore the effect of measurement error in the following way. We begin with situations (1) and (2), where the effect of measurement error is nondifferential. Then we consider the effect of the types of differential measurement error (situation 3–6). For each of these situations, we calculate adjusted odds ratios, assuming both differential error and random measurement error in factor VIII.
Situation 1 and 2: Classic Measurement Error and Nondifferential Variation Over Time
We first explore the effect of classic measurement error and day-to-day variation. The bias formulas (6) and (7) are applied to correct the obtained log-odds ratio, and bootstrapping is used to estimate the 95% CI. Figure 3A provides the corrected odds ratios as function of the reliability ratio λ between measurements. For reliability ratios below 0.75, the corrected odds ratio is substantially smaller. Measurements in our laboratory indicate that the intra-assay coefficient of variation of factor VIII is around 4.5%, corresponding to a reliability ratio >0.90. This is too small to substantially affect the remaining odds ratio.
External data from a study on the effect of air travel on thrombosis23 showed that factor VIII varies substantially over time. Levels of 71 healthy adults were measured at 3 consecutive settings at least 2 weeks apart. The reliability ratio of these measurements, which suffer from laboratory error and variation over time, was 0.58. Using this reliability ratio in formula (7), to correct the estimated odds ratio for bias due to variation over time, yielded a correction ORcor = 1.12 (95% CI = 0.75–1.68). In this study, measurements for the cases had been done at least 3 months after the thrombosis, and the reliability ratio could therefore be even <0.58. This suggests that intraindividual variation did severely inflate the observed remaining odds ratio. This estimate of the reliability ratio will be used in the next situations where we study the effect of combinations of random and systematic measurement error.
Situation 3 and 4: Exposure Affects Measurement of the Intermediate
Figure 3B shows the effect of a systematic difference in measurement error between exposed and unexposed subjects. If the measured intermediate is systematically 6 IU/L lower for subjects with blood group non-O, this would decrease the odds ratio by 10%, whereas a systematic increase of 6 IU/L increases the odds ratio by 10%. It seems unlikely that the measurement error in factor VIII would be influenced as much as this by blood group because procedures of measurement were exactly the same; it is just as unlikely that blood group influences the biochemical measurements. We also do not expect time-varying effects of blood group on factor VIII.
Situation 5: Trigger Interacting With Exposure and Influencing the Intermediate
The effect of a trigger that is present for a short period and which interacts with exposure, can be estimated with formula (9). Figure 3C shows the effect on the OR for blood group of an interacting trigger with a prevalence of 20%. Even when the trigger has a large effect (say, a systematic increase of factor VIII by 10 IU/dL, for blood group non-O), the odds ratio is increased by a factor of only 1.08. To substantially change the estimated direct effect, the trigger would have to be prevalent, with a large effect on the intermediate. We therefore expect that interacting triggers are not a major source of bias in the Leiden Thrombophilia Study.
Situation 6: Outcome Affects Measurement
Next, we studied the effect of the post hoc phenomenon: systematic measurement error because factor VIII is measured for cases after the thrombosis. Formula (10) can be used to estimate the size of the bias. Figure 3D shows the corrected odds ratio for blood group as a function of the mean increase in factor VIII measurements after a thrombosis, again with λ = 0.58. We expect that a post hoc effect, caused by a persisting thrombus, should lead to increased levels of factor VIII. A mean increase in factor VIII values of 5 IU/L after a thrombosis seems reasonable. This post hoc effect will cause negative bias in the estimated direct effect. Using formula (10) with γ1 = 5 and λ = 0.58 yields an estimated direct effect odds ratio for blood group of 1.44 (95% CI = 0.98–2.15).
Situation 7: Unmeasured Confounding
Because confounding of the relation between factor VIII and thrombosis could also be a source of bias, we also calculated the direct effect that remains after adjusting for several potential known confounders, body mass index, smoking, and alcohol use. All of the measured confounders operated on factor VIII and thrombosis in the same direction: smoking and body mass index were positively associated with both factor VIII and thrombosis, whereas alcohol use was associated with slightly lower factor VIII levels and a slightly lower risk of thrombosis. Adjusting for confounders yielded an adjusted odds ratio for blood group of 1.52 (95% CI = 1.04–2.22), which hardly differs from the unadjusted odds ratio of 1.47. We do not expect that unknown confounders have a substantially larger effect than these known confounders. Therefore, we conclude that confounding here is not a major cause of bias.
These analyses showed that measurement error and variation over time could substantially bias estimates of direct effects. Adjusting for random variation in factor VIII measurements over time substantially shrank the estimated odds ratio of the direct effect from 1.47 to 1.12. From the 4 different types of differential measurement error, we expect that only the post hoc effect could be substantial. Adjusting for both random error and a post hoc effect, using educated guesses for the size of both sources, yielded a corrected odds ratio of 1.44 (0.98–2.15). The wide CI (including the value 1.0) shows that no conclusive statements can be made regarding the existence of a direct effect of blood group on thrombosis.
We studied the performance of the approximations for the different types of measurement error by simulation. In each simulation study, case-control data were generated with 300 cases and 300 control subjects, under conditions mimicking the Leiden Thrombophilia Study. The observed effect of X after adjusting for the observed mediator was compared with the expected effects using the formulas of the previous section. Three scenarios were considered in which all the effect of X on Y passes through M:
- The situation of Figure 2A: nondifferential error in M*.
2.The situation of Figure 2E. Trigger effect.
- True model: logit(Pr(Y = 1|M, X)) = β0 + β2M.
- The true mediator M|X ∼ N (α0 + α1X, σM 2).
- The observed mediator M* = M + U, with U ∼ N (0, σu 2).
- In these simulations, β0 = logit (0.001), β2 = 0.036, α0 = 0, α1 = 22, σM = 30, and Pr(X = 1) = 0.5. Several values for σu 2 are considered.
3.The situation of Figure 2F. Post hoc phenomenon.
- True model: logit(Pr(Y = 1|M,X)) = β0 + β2M.
- The true mediator M|X ∼ N (α0 + α1X, σM 2).
- The observed value of the mediator M* = M + cTX, with T binary, pr(T = 1) = pT.
- In these simulations, β0 = logit(0.001), β2 = 0.0176, α0 = 0, α1 = 22, σM = 30, and Pr(X = 1) = 0.5. Several values for c and pT are considered.
- True model: logit(Pr(Y = 1|M, X)) = β0 + β2M.
- The true mediator M|X ∼ N (α0 + α1X, σM 2 ).
- The observed value of the mediator M* = M + γ1Y + U, with U ∼ N (0, σu 2 ).
- In these simulations β0 = logit(0.001), β2 = 0.0176, α0 = 0, α1 = 22, σM = 30, σu = 5 and Pr(X = 1) = 0.5. Several values for γ1 are considered.
For each of these situations, the amount of bias was calculated using the bias formulas derived in this paper. Then, by simulation, 1000 replications were generated. In each of the generated data sets, the actual difference between the estimated regression coefficient for X and the true value (which is zero) was calculated. In Figure 4, the actual observed bias is compared with the expected bias for the 3 situations. The circles in these plots represent the actual observed median bias (median of the 1000 calculated differences between the estimated coefficient and its true value). The lines show the expected bias obtained from the bias formulas. In all situations, the actual and the expected bias were close.
We have discussed a common situation in which a researcher has estimated a direct effect and wants to know whether this could be biased by measurement error of the intermediary variable. We studied several types of measurement error and provided approximations of the size of the bias due to these mechanisms. Simulations showed that the developed approximations perform well.
Application of these tools in the Leiden Thrombophilia Study showed that intraindividual variation over time (ie, day-to-day variation within individuals) and the post hoc phenomenon are the largest source of bias for the remaining odds ratio of 1.5 between blood group and thrombosis after adjusting for factor VIII. Other explanations, such as laboratory measurement error, unmeasured triggers, and confounding of the relationship between factor VIII and thrombosis, are unlikely to affect the odds ratio of 1.5 substantially.
We made some assumptions to approximate the remaining association between exposure and outcome. We assumed that the mediator was continuous and followed a normal distribution. This was motivated by the situation of DNA characteristics acting via biochemical pathways. Adjustments for measurement error are robust against deviations from normality (see Caroll3 p. 91). However, if the distribution of the mediator is highly skewed, we suggest performing a normalizing transformation.
Another assumption was that the relationship between the mediator and the outcome could be described by a logistic model, without interaction between exposure and intermediate. If this assumption is not valid, the direct-effect estimates can be biased due to misspecification of the model. Whether this is the case can be easily examined by adding higher-order terms of M and interaction terms into the model and checking whether the model fit improves and the remaining association changes. In the Leiden Thrombophilia Study, higher-order terms in factor VIII did not substantially improve the model, and the odds ratio for blood group on thrombosis did not change. Interaction between the effects of blood group and factor VIII on thrombosis was also small (data not shown). In case the logistic regression model does not fit well, a more extensive model with nonlinear terms and interaction terms could be fitted. In that case, our simple correction formulas for measurement error can no longer be used, and it is necessary to correct for the measurement error using more advanced methods such as the simulation extrapolation (SIMEX) method.24
Finally, we considered only the situation of a binary outcome. For continuous outcomes, similar bias formulas can be derived for the first 3 types of measurement error: classic measurement error, day-to-day variation, and measurement error affected by exposure.
The situations we considered are deliberately kept simple: only one factor varies at a time, and complete knowledge of the relationship between blood group, factor VIII and thrombosis is assumed. In practical applications, a spurious association between exposure and outcome after conditioning on the mediator will often be caused by a combination of sources of error. Still, these simple rules are useful because they are easy to calculate and give a rough assessment of the magnitude of bias from the various sources and how they may affect conclusions.
In summary, we have presented simple tools that can be used to correct the estimate of a direct association between exposure and outcome for various types of measurement errors in the intermediate variable. We have shown that the effect of measurement error can substantially bias direct-effect estimates. The influence of measurement error on the estimated direct effects should be routinely assessed before drawing causal conclusions.