Secondary Logo

Journal Logo

Methods: Original Article

Bias Formulas for Sensitivity Analysis of Unmeasured Confounding for General Outcomes, Treatments, and Confounders

VanderWeele, Tyler J.a,b; Arah, Onyebuchi A.c,d

Author Information
doi: 10.1097/EDE.0b013e3181f74493

Abstract

Unmeasured confounders in observational studies result in biased effect estimates. Several sensitivity-analysis and bias-modeling techniques have now been developed to handle uncontrolled confounding.1–22 Although the literature is large and it would be difficult to provide a comprehensive review, a number of the existing techniques are restricted to simple or very particular settings. There is also a literature on bounds for causal effects or “partial identification”23–27that makes fewer assumptions than sensitivity-analysis techniques, but that thereby effectively considers extreme scenarios which in some settings give bounds that include the null irrespective of the data.

In this paper, we extend the bias-modeling and sensitivity-analysis literature on uncontrolled confounding. Using the potential outcomes framework, we derive a general class of formulas for sensitivity analysis of uncontrolled confounding with outcomes, treatments, and measured and unmeasured confounders that may be categorical (2 or more categories) or continuous. The formulas generalize many previous results in the literature and give rise to very flexible sensitivity-analysis techniques that can be used in a wide range of applications. We also describe a particularly easy-to-use sensitivity-analysis technique that can be used under some simplifying assumptions and that follows from our results and is summarized in the discussion section.

NOTATION, DEFINITIONS, AND ASSUMPTIONS

We will use the potential outcomes or counterfactual framework.28–30 Let treatment A denote the treatment received by a particular individual. Let Y be the observed post-treatment outcome of that individual. Let Ya denote the potential outcome Y for an individual if the treatment A, perhaps contrary to fact, had been set to value a. Note that we assume that the potential outcome Ya for an individual does not depend on the treatments received by other individuals. This assumption is sometimes referred to as SUTVA, the stable unit treatment value assumption30 or as a no-interference assumption.31 Furthermore, we require the consistency assumption that YA= Y, ie, that the value of Y that would have been observed if A had been set to what it in fact was is equal to the value of Y which was actually observed. Therefore, the only potential outcome for an individual that we observe is the potential outcome YA, the value of Y that would have been observed if A were set to what it in fact was. We will use the notation

to denote that a variable S is independent of another variable T conditional on a third variable V. To simplify notation further, we will use E(S|t) to denote E(S|T = t) and P(s|t) to denote P(S = s|T = t).

Let X denote observed categorical or continuous covariates. Let U denote an unmeasured categorical or continuous confounding variable or variables. Suppose that Y is dichotomous, ordinal, or continuous and that A is categorical, ordinal, or continuous. For causal contrasts, we compare expected potential outcomes (ie, counterfactual outcomes) for any 2 treatment levels, a1 and a0, of A where a0 is taken as the reference.

The average causal effect in the total population and among those receiving treatment A = a1 or A = a0 are given respectively by

and

Suppose that the effect of A on Y is unconfounded given (U, X), where again U is unmeasured; ie, in counterfactual notation we assume that

We then have that the true causal effects are given by adjusting for both X and U:

If adjustment is made for X but not U, we would obtain the following expressions for the average outcome differences adjusted for X when the target population is the total group, or those exposed to a1 or a0, respectively:

The bias due to not controlling for the unmeasured confounder U is thus given by the difference between the observed average outcome differences, adjusted for X, and the true causal effect. Let da+, da1, and da0 denote the relevant bias when the target population is the total group, or those exposed to a1 or a0, respectively:

In the next section, we derive formulas for these biases that can be used in sensitivity analysis.

GENERAL BIAS FORMULAS FOR SENSITIVITY ANALYSIS FOR AVERAGE TREATMENT EFFECTS

Because we do not have data on U, we cannot obtain unbiased estimates of causal effects. We can only estimate the average outcome differences, adjusted for the observed covariates X. We can proceed, however, by using sensitivity analysis. The following theorem is a generalization of previous work.1,8,21 Complete proofs of all results are given in the Appendix and eAppendix (https://links.lww.com/EDE/A429). The result gives formulas for the bias da+, da1, and da0 in terms of the relationship between the unmeasured confounder(s) U and the outcome Y as well as the relationship between U and treatment group A.

Theorem 1. If

and if u′ is any chosen reference value for the unmeasured confounder U then

To use these bias formulas in sensitivity analysis for the overall causal effect

one would need to specify (i) the relation between U and Y, among those with treatment level A = a1 and A = a0, within each stratum of X, ie,

and

and (ii) how the distribution of the unmeasured confounder U among those with treatment level A = a1 and A = a0 compares with the overall distribution of U, within each stratum of X, ie,

and

One could then use the observed data to estimate outcome differences adjusted for X only,

one could use the sensitivity-analysis specification concerning (i) and (ii) to estimate the bias da+ and then one could produce an estimate of

by subtracting the bias da+ from the estimate adjusted for X only. As can be seen for the bias formulas for da1 and da0, if one wishes to estimate the causal effect only among those with A = a1 or A = a0 then fewer specifications need to be made in the sensitivity analysis.

The result applies to mean differences in binary, ordinal or continuous outcomes, for categorical or continuous treatments, and with categorical or continuous measured and unmeasured confounding variables. We note that if one or both of X and U are continuous rather than categorical, the sums in Theorem 1 can be replaced with appropriate integrals. Theorem 1 is quite general, and in the Appendix we discuss how it encompasses a number of more specific sensitivity-analysis techniques in the literature.1,6,8,11,21 Unlike many of the existing techniques, the current approach does not assume that the unmeasured confounders are independent of the measured confounders (see the section in the Appendix on the relation to the external adjustment literature for more detail). The Appendix also discusses other parameterizations for sensitivity analysis (see the section on the relationship to Rosenbaum and Rubin6). We discuss standard errors and confidence limits for these bias-adjusted estimates below.

Note that, although the result in Theorem 1 provides a very general approach to sensitivity analysis, it requires that a great deal of information be specified; namely, on (i) the relationship between the unmeasured variable U and the outcome Y across strata of treatment and the measured covariates and also (ii) for each stratum of the measured covariates, how the distribution of the unmeasured confounder U among those with various treatment levels compares with the overall distribution of U. Thus for each stratum of the measured covariates one would have to specify 4 pieces of information (ie, (i) and (ii) for the 2 different treatment levels); if there are many strata of the measured covariates, this will not be particularly feasible in practice. Below we show that under some simplifying assumptions, the formulas in Theorem 1 reduce to expressions that are easier to use in practice and that require the specification of fewer sensitivity-analysis parameters.

There is, in general, a tension between generality and complexity. A general approach makes fewer assumptions but requires one to specify a large number of sensitivity parameters; a simpler approach as described below is easier to use and requires the specification of fewer parameters, but it makes stronger simplifying assumptions. Depending on the strength of assumptions a researcher is willing to make concerning the unmeasured confounders, more or less complex approaches are possible in the application of Theorem 1. In the applications section below, we illustrate both a particularly simple sensitivity analysis and another which makes more use of the generality of the bias formulas in Theorem 1. We now turn to a discussion of the sensitivity-analysis approach that results under simplifying assumptions.

If one makes the simplifying assumption that the relationship between U and Y, ie,

does not vary between strata of A then the expression for da+ simplifies to

If, in addition, the relationship between U and A, namely

also does not vary between strata of X, then this expression further simplifies to:

If U is binary, it further reduces to

Thus a straightforward sensitivity-analysis technique under simplifying assumptions would consist of hypothesizing a binary confounding variable U with a constant prevalence difference,

comparing treatment levels a1 and a0 over strata of X, and with

constant over strata of A and X; the magnitude of the bias comparing the true causal effect

with the estimate adjusted only for X but not U,

is then simply the product δγ, ie,

This simple formula has been obtained previously11,32 but under much stronger assumptions (see Appendix for further discussion). Unlike many other results in the sensitivity-analysis literature, this simple sensitivity analysis (following from Theorem 1) does not presuppose that any particular method, model, or functional form assumption was used to obtain the initial estimate adjusted only for X and not U. Its generality is illustrated in the application section below.

Note that the relationship between U and A, ie,

may or may not allow for an interpretation as the causal effect of U on Y. If the effect of U on Y is unconfounded given X, then

may allow for an interpretation as a causal effect of U on Y—namely, as the direct effect of U on Y controlling for A. Note, however, the assumption that the effect of A on Y is unconfounded given (U, X) does not necessarily imply that the effect of U on Y is unconfounded given X (see the eAppendix [https://links.lww.com/EDE/A429] for the precise conditions for such an interpretation). We note, nevertheless, that the expression

does not itself need to have a causal interpretation to use the bias formulas in Theorem 1 for sensitivity analysis and external adjustment. Causal diagrams can clarify when an expression such as

can be interpreted causally.33 Further discussion of these issues is given in the eAppendix (https://links.lww.com/EDE/A429).

We note also that bias formulas analogous to those in Theorem 1 also hold for conditional causal effects. The conditional causal effect with the entire population, or treatment level a1 or a0 taken as the standard, are given by

and

respectively. If we let da+(x), da1(x), and da0(x) denote the difference between the true effects and the estimator

then bias formulas for da+(x), aa1(x), and da0(x) are just like those given in Theorem 1 except the summation of x and the final term in each expression P(x), P(x|a1), or P(x|a0) would be removed. For conditional effects, to use the simple bias formula δγ described above one would only need to assume binary U whose effect on the outcome does not vary with levels of treatment.

We now consider standard errors and confidence intervals for bias-adjusted estimates of the causal effect. First, consider the simple sensitivity-analysis technique with binary U discussed above. Once the sensitivity parameters, corresponding to the relationships between U and A ]ie,

[ and between U and Y [ie, γ = {E(Y|a,x,U = 1) − E(Y|a,x,U = 0)}]] are fixed, the standard error of the bias-corrected estimator,

is precisely the same as that of the original estimator. This feature is shared by the results of Lin et al11 discussed in the eAppendix, but not by certain other sensitivity-analysis methods.12,17

Because the standard errors of the original and bias-adjusted estimates are the same, the bias formulas can be applied not only to the estimate itself but also to both limits of a confidence interval of the original estimate. This useful feature of identical standard errors thus gives rise to a very straightforward method for obtaining confidence intervals for the bias-corrected estimate. These remarks also hold for the bias-corrected estimators of the conditional causal effects discussed above. However, the remarks do not apply to the most general results in Theorem 1 if the sensitivity parameters

or

vary with x; this is because the bias formula for example da+ would depend on the distribution of X, which would have to be estimated from the data. When {E(Y|a1,x,u) − E(Y|a1,x,u′)}, {E(Y|a0,x,u) − E(Y|a0,x,u′)}, or {P(u|a,x) − P(u|x)} vary with x, one possible approach would be to obtain bootstrapped samples of the data and to apply the bias formulas to each bootstrapped sample and use the, eg, 2.5th and 97.5th percentiles of the bias-corrected estimates as a 95% confidence interval for the bias-corrected estimate of the original data.22

OTHER MEASURES OF EFFECT

For binary Y, other measures of effect such as the risk ratio or odds ratio may be of interest. The conditional causal risk ratio in the total population or among those receiving treatment a1 or a0 are defined respectively by E(Ya1|x)/E(Ya0|x), E(Ya1|a1,x)/E(Ya0|a1,x), and E(Ya1|a0,x)/E(Ya0|a0,x). The conditional causal odds ratios in the total population or among those receiving treatment a1 or a0 can be defined similarly, eg, the conditional causal odds ratio in the total population is defined by:

We can define bias expressions dRRa+(x), dRRa1(x) and dRRa0(x) corresponding to the ratios between the risk ratios conditional on X and the true conditional causal risk ratios respectively in the total population or among those receiving treatment a1 or a0:

We can define the conditional odds-ratio bias term analogously. For the bias terms for the conditional causal risk ratio, we then have the following result, expressing the biases in terms of the relationship between the unmeasured confounder(s) U and the outcome Y and the relationship between U and treatment group A.

Theorem 2. If

and if u′ is any chosen reference value for the unmeasured confounder U then

If the outcome is rare in all strata of a, x, and u, then the bias formulas given in Theorem 2 will also hold approximately for bias formulas for the conditional causal odds ratio. Exact bias formulas for odds ratios are also given in the eAppendix (https://links.lww.com/EDE/A429) but are somewhat more cumbersome to use. In some cases, we may be interested in marginal or “standardized” causal risk ratios and odds ratios. The marginal or “standardized” causal risk ratio in the total population or among those receiving treatment a1 or a0 are defined, respectively, by E(Ya1)/E(Ya0), E(Ya1|a1)/E(Ya0|a1), and E(Ya1|a0)/E(Ya0|a0). We can then define the bias expressions dRRa+, dRRa1 and dRRa0 for the marginal causal risk ratios by the ratio between the marginalized risk ratios adjusted for X only and the true causal risk ratios:

The marginal or “standardized” causal odds ratio, along with bias term for these marginal causal odds ratios, can be defined similarly. Although the formulas and their derivations are somewhat more complicated, bias results for the marginal causal risk ratio can also be obtained and are given in the following theorem.

Theorem 3. If

and if u′ is any chosen reference value for the unmeasured confounder U then, and if x′ is any chosen reference value for X, we then have the following:

where,

and where, dRRa1(x), and dRRa0(s) are the bias formulas for the conditional risk ratios given in Theorem 2.

Note that in applying Theorem 3, the expressions of the form E(Y|a,x,u)/E(Y|a,x,u′) and P(u|a,x) would be specified in a sensitivity analysis; the expressions of the form E(Y|a,x)/E(Y|a,x′) can be estimated from the data. If the outcome is rare in all strata of a, x, and u, then the bias formulas given in Theorem 3 will also hold approximately for bias formulas for the marginal causal odds ratios. As discussed in the Appendix, these results for the conditional and marginal causal risk ratios and causal odds ratios generalize those for the risk ratios and odds ratios in the external adjustment literature.7,9,21

We close this section by noting that in the simple sensitivity-analysis setting, in which it is assumed that there is a binary unmeasured confounding variable U such that the relationship between U and Y, ie,

is the same for treatment levels a1 or a0 and takes value γ, then3,4:

If the outcome is rare, this simple formula will also hold for dORa+(x). As with the linear scale, the bias formulas can be applied immediately to the estimate and both limits of a confidence interval in the case of conditional causal risk ratios and odds ratios; however, a bootstrap approach would be necessary for the marginal causal risk ratio or odds ratio.

APPLICATIONS OF BIAS FORMULAS

We illustrate the application of Theorem 1, using 3 examples. The first example highlights the interpretative ease of using a linear scale for sensitivity-analysis parameters (as opposed to, for example, an odds ratio scale); the second and third examples illustrate the generality and flexibility of the sensitivity-analysis technique described above. In the first example, we consider an analysis of the data presented by Rosenbaum and Rubin6; in the Appendix we compare and contrast the sensitivity-analysis technique described in this paper to that described in their work. Whereas Rosenbaum and Rubin give the sensitivity-analysis parameters on an odds-ratio scale, the sensitivity-analysis technique described above allows us to specify sensitivity-analysis parameters on a linear scale, which are often easier to interpret.

Rosenbaum and Rubin consider a study comparing coronary artery bypass surgery (A = 1) to medical therapy (A = 0) in the treatment of coronary artery disease. Their outcome, Y, is dichotomous—symptomatic relief after 6 months. Data are available on 74 covariates (X) and they use a propensity-score approach34 to form 5 propensity-score subclasses. The overall adjusted relief rates are 0.67 for surgery and 0.36 for medical therapy, yielding an estimate of the causal effect of 0.31 (95% confidence interval 0.169 to 0.451). In this section, we will apply the formula for da+ in Theorem 1 under simplifying assumptions to assess the sensitivity of the estimate of the causal effect to unmeasured confounding. Using the simple sensitivity-analysis technique described above, we see that, if there were a binary unmeasured confounding variable U with a 0.6 higher prevalence among those receiving surgery as compared with medical therapy (in all propensity-score strata), and if the outcome difference comparing those with U = 1 and U = 0 were 5.17 for both treatment groups in all propensity-score strata, then we would obtain a bias term of da+ = (0.517)(0.6) = 0.31. Thus such an unmeasured confounding variable would reduce the estimate of the causal effect to 0 (95% CI = −0.141 to 0.141). If, on the other hand, the difference in the prevalence of the unmeasured confounding variable among those receiving surgery as compared with medical therapy were only 0.3 in all propensity score strata, then we would obtain a bias term of da+ = (0.517)(0.3) = 0.155 and our estimate of the causal effect would still be 0.31−0.155 = 0.155 (95% CI = 0.014 to 0.296). Admittedly, it seems unlikely that an unmeasured confounding variable could have an effect on the outcome sufficiently large (after control for 74 measured covariates) to invalidate the qualitative conclusion of surgery providing higher proportion of relief at 6 months than medical therapy. A 30 percentage point difference in the prevalence of the unmeasured confounder would be inadequate to explain away the estimated effect, even if the unmeasured confounder had an effect of the same magnitude, 0.517.

In our second example, we will consider a study by Reinisch et al35 that examined the effect of in utero exposure to phenobarbital on intelligence in men. Subjects were selected from the largest hospital in Copenhagen. The exposure group consisted of those who had been exposed in utero to phenobarbital (A = 1) and the control group of those who had not (A = 0). Propensity-score matching and regression techniques36 were used to adjust for background characteristics in making intelligence comparisons using the Danish Military Board Intelligence Test (Y) taken by the exposed and unexposed men when they had reached their early 20s. The background characteristics (X) for which adjustment was made included family socioeconomic status, breadwinner's education, sibling position, whether the pregnancy was wanted, whether the mother attempted an abortion, maternal marital status, predisposing risk score indicating conditions were less than optimal for conception, mother's age, father's age, gestational length, birth weight, birth length, number of cigarettes per day in the third trimester, maternal weight gain divided by height cubed, and the maternal complaint score.

Subjects exposed to phenobarbital were found to have significantly lower scores on the Danish Military Board Intelligence Test than they would have had they not been exposed. Specifically, under the assumption that the effect of exposure is unconfounded given X, Reinisch et al obtained an estimate of E(Y1|A = 1) of 39.58 and an estimate of E(Y0|A = 1) of 44.35 to obtain an estimate of the effect of the exposure on the exposed of E(Y1|A = 1) − E(Y0|A = 1) = −4.77 (95% CI = −7.96 to −1.58). Reinisch et al suggest that parental intelligence, which was not measured in the study, may partially confound the analysis. Reinisch et al reason informally, without a quantitative analysis, to argue that it is unlikely that parental intelligence, rather than drug effects, are responsible for the observed intelligence deficits. Using Theorem 1 above, if we hypothesize an unmeasured confounding variable U of, say, the average of maternal and paternal intelligence measured by the Danish Military Board Intelligence Test, and we assume that if unexposed, for all x, a one-point increase in U would on average result in a 0.3 point increase in Y so that E(Y|a0,x,u) − E(Y|a0,x,u′) = (0.3)(uu′), then it follows from Theorem 1 that da1 = (0.3){E(U|A = 1) − E(U|A = 0)}. It would thus require a difference in parental intelligence of E(U|A = 1) − E(U|A = 0) = −4.77/(0.3) = −15.9 between the parents of the exposed and unexposed on the Danish Military Board Intelligence Test to completely eliminate the estimated deficit. Reinisch et al note that the standard deviation for a national sample of subjects taking the Danish Military Board Intelligence Test was 11.38. A 1.3 standard deviation difference in parental intelligence between the exposed and unexposed, although not entirely implausible, seems unlikely.

Note that the sensitivity-analysis technique of Rosenbaum and Rubin6 is not applicable to the study of Reinisch et al because the outcome is continuous, not binary; similarly, the prior external-adjustment literature1,8,21 is not applicable because the outcome is continuous. The sensitivity-analysis techniques of Imbens14 and Lin et al,11 described in the Appendix, apply to continuous outcomes but presuppose a regression model, whereas Reinisch et al35 obtained results from a matched propensity-score analysis rather than simply through an outcome-regression model. Furthermore, the sensitivity-analysis technique of Lin et al11 presupposes no interaction between the treatment and the covariates, whereas the analysis of Reinisch et al does not assume the absence of such interactions, and, in fact Reinisch et al report evidence of such interactions. Importantly, a sensitivity analysis using the results in Theorem 1 above is not dependent on any particular model or method for obtaining estimates of the prima facie estimate of the causal effect adjusted only for x; sensitivity analysis using Theorem 1 is applicable irrespective of how the initial adjusted estimates are obtained. Furthermore, a sensitivity analysis using the results in Theorem 1 does not presuppose any particular functional form or the absence of interactions.

Our third example illustrates the flexibility of our approach by applying it to a setting in which inverse-probability-of-treatment weighting37 is used to estimate causal effects. This application shows how the bias formulas in Theorem 1 can be employed in greater generality. We use data from the National Center for Health Statistics (NCHS) Birth Certificate Files for year 2000 to consider the effect of adequate prenatal care on birthweight. Prenatal care was classified as adequate versus inadequate, as defined by a modification of the Adequacy of Prenatal Care index.38,39 Baseline covariates in the NCHS data include maternal age, race, place of birth, place of residence, education, marital status, plurality, gravidity, prior preterm birth, prior birth greater than 4000 g, alcohol consumption, and tobacco use. Socioeconomic status might serve as an unmeasured confounding variable for the relationship between prenatal care and birth weight. Although measured education may serve as a proxy for socioeconomic status, there are likely aspects of socioeconomic status that education does not capture. The relationships between socioeconomic status and birthweight, and between socioeconomic status and adequate prenatal care, may vary by age. For example, the relationship between socioeconomic status and adequate prenatal care may be weaker for those age 19 years or under at the time of childbirth than for those at older ages, due to State Children's Health Insurance Programs (SCHIP). On the other hand, the birthweight of infants of younger mothers may be more sensitive to adverse socioeconomic circumstances. Inverse probability of treatment weighting37 was used to obtain estimates of the effect of prenatal care on birthweight, stratified by age (≤19 vs. >19 years), controlling by weighting for the aforementioned confounding variables. The estimated effect for younger mothers was 82.4 g (95% CI = 78.2 to 86.7) and for older mothers was 78.3 g (95% CI = 76.3 to 80.3); 12.2% of the mothers were age 19 years or under, and thus the overall estimate of the effect is 78.8 g (95% CI = 76.4 to 81.2). Let U denote adverse versus adequate socioeconomic status. Suppose we hypothesize that the average effect of adverse socioeconomic status for younger mothers is 120 g and the average effect for older mothers is 80 g whereas the difference in the likelihood of adverse socioeconomic status, comparing those with adequate versus inadequate prenatal care, was only 20% for young mothers (due to SCHIP); however, for older mothers, comparing those receiving adequate versus inadequate prenatal care, the difference was 50%. Employing the bias formula in Theorem 1, we would have a corrected estimate of effect of 40.8 g (95% CI = 38.3 to 43.2). Other values for the sensitivity-analysis parameters could similarly be considered. For example, if the effect sizes for the unmeasured confounder were doubled to 240 and 160 g for younger and older mothers, respectively, this would reverse the sign of the point estimate of the effect. Here, in contrast to the first 2 examples, the values of the sensitivity-analysis parameters needed to completely explain away the effect are perhaps not as implausible.

In this third example, we have allowed the effect of the sensitivity-analysis parameters to vary over one covariate (age), and we thus obtain an approach between the complexity of the simple technique described above and the fully general formula given in Theorem 1. Of course it would also be possible to allow the sensitivity parameter to vary across other measured covariates; however, as noted in the discussion following Theorem 1, the more general the approach, the more sensitivity-analysis parameters need to be specified.

In the eAppendix (https://links.lww.com/EDE/A429), to yet further demonstrate the flexibility of our approach, we apply it to a recent study40 employing a doubly robust estimator for treatment effects.41–44

DISCUSSION

In this paper, we have derived bias formulas for sensitivity analysis for causal effects. These formulas have allowed for binary, ordinal, or continuous outcomes; categorical or continuous treatment; and categorical or continuous measured and unmeasured confounding variables. We have obtained results for additive, risk-ratio and odds-ratio scales. We have shown that these bias formulas generalize many of the existing sensitivity-analysis results in the bias-modeling literature, and can be used in a broad range of settings; the results do not presuppose a particular functional form relating the outcome and the observed covariates and treatment. Furthermore, the results can be used to perform a very simple form of sensitivity analysis. The sensitivity-analysis approach that we have taken fixes the sensitivity parameters and considers how the conclusions would be affected; an alternative Bayesian approach gives a distribution of parameter values and incorporates the uncertainty of this distribution into the confidence intervals for the corrected estimate.15,16,19

As noted above, when comparing mean outcome differences, a researcher might hypothesize a binary unmeasured confounding variable U with constant effect γ on the outcome across treatment and covariate levels and with a constant prevalence difference, δ, comparing treatment levels a1 and a0 across strata of the covariates. Under these simplifying assumptions, the bias comparing the estimated outcome difference (adjusted only for measured covariates X) and the true causal effect (adjusted for both X and U) is then given simply by δγ. We have shown that this simple technique is in fact applicable irrespective of the particular method by which the initial adjusted estimates of the causal effects were obtained, and furthermore does not assume the absence of interactions between treatment and covariates. Although, in many contexts, more sophisticated sensitivity-analysis techniques may be desirable, this simple approach leaves researchers without excuse for not performing, at the very least, a simple sensitivity analysis.

APPENDIX

Proof of Theorem 1.

We give the proof for da+. The proofs for da1 and da0 are given in the eAppendix (https://links.lww.com/EDE/A429).

by iterated expectations

by unconfoundedness conditional on (U,X)

by consistency

since E(Y|a1, x, u′) and E(Y|a1, x, u′) are constants.

Proof of Theorems 2 and 3.

See eAppendix (https://links.lww.com/EDE/A429) for details.

Relation to Other Sensitivity Analysis Techniques

Relation to the Sensitivity Analysis of Rosenbaum and Rubin (1983)

In the sensitivity analysis proposed by Rosenbaum and Rubin,6 they consider a binary outcome Y, binary treatment A, covariate(s) X (in their application, X indicates propensity score strata), and hypothesize a binary unmeasured confounder U such that

The researcher specifies sensitivity parameters

For specified values of πx, αx, and δxt, maximum likelihood estimates of the causal effect can then be obtained; sensitivity analysis proceeds by specifying different values of the parameters πx, αx, and δxt. In their application (described above), πx, αx, and δxt are assumed to be constant over x. As noted above, assuming no unmeasured confounding the estimate of the causal effect is 0.31. Rosenbaum and Rubin6 consider values for π (the overall prevalence of U) of 0.1, 0.5, and 0.9. They first consider α = 2 and values δt=0 and δt=1 either ½ or 2; under this set of scenarios, the smallest causal effect estimate, an estimate of 0.28, is obtained when the prevalence of U is 0.5 and when U doubles the odds of surgery (α = 2) and also doubles the odds of improvement (δt=0 = 2 and δt=1 = 2). They then consider α = 3 and values δt=0 and δt=1 either 1/3 or 3; under this set of scenarios, the smallest causal effect estimate, an estimate of 0.25, is obtained when the prevalence of U is 0.5 and when U triples the odds of surgery (α = 3) and also triples the odds of improvement (δt=0 = 3 and δt=1 = 3). They conclude that for an unobserved confounder to explain the outcome difference comparing medical and surgical patients, it would have to more than triple the odds of surgery and of improvement. Admittedly, this seems unlikely. We saw above, however, that when the sensitivity analysis is conducted on a risk-difference scale rather than an odds-ratio scale, although the degree of uncontrolled confounding that would be needed to explain away the estimate of the causal effect is still unlikely, the numbers are perhaps slightly less inconceivable.

The bias formula for da+ in Theorem 1 can also be used in a reasonably straightforward way to replicate the odds-ratio sensitivity analysis of Rosenbaum and Rubin.6 Let a0 = 0 and a1 = 1 and let u = 1 and u′ = 0. For specified values of the sensitivity-analysis parameters πxx, and δxt in the approach proposed by Rosenbaum and Rubin we will show how to obtain the quantities needed for the application of the formula for da+ in Theorem 1, namely, (i) {E(Y|a1, x, u) − E(Y|a1, x, u′)} and {E(Y|a0, x, u) − E(Y|a0, x, u′)} and (ii) {P(u|a1, x)−P(u|x)} and {P(u|a0, x)−P(u|x)}. Given αx and πx, we can use the equations

to solve for P(a0|x, u) and P(a0|x, u′; note that P(u|x) = πx and P(u′|x) = 1 − πx. It then follows from Bayes' rule that

A similar procedure can be used to obtain P(u|a1, x) and thus also P(u|x). Furthermore, to obtain {E(Y|a1, x, u) − E(Y|a1, x, u′)} and {E(Y|a0, x, u)−E(Y|a0, x, u′)}, we can use the equations

to solve for E(Y|a0, x, u) and E(Y|a0, x, u′); note that P(u|a0, x) and P(u′|a0, x) have already been obtained. A similar procedure can be used to obtain E(Y|a1, x, u) and E(Y|a1, x, u′). We can then proceed by using the bias formula for da+

Although the bias formula for da+ in Theorem 1 can be used to replicate the odds-ratio sensitivity analysis of Rosenbaum and Rubin,6 the formula in Theorem 1 is considerably more general since, as we have seen in the applications above, it can be applied to binary or continuous outcomes and to binary, categorical, or continuous confounding and treatment variables. The result of Rosenbaum and Rubin was restricted to binary outcomes.

Relation to Sensitivity Analysis of Lin et al (1998)

Lin et al11 considered settings including binary and continuous outcomes Y, binary treatment A, and binary and continuous unmeasured confounding variable U. They compared the 2 regression models

and

for linear, log-linear, and logistic links g and derived algebraic formulas to relate β and β* under 2 possible alternative assumptions. Their first assumption was that U and X were conditionally independent given A. Their second possible assumption was that the mean of U conditional on A and X was additive in A and X, ie, μa,x: = E(U|A = a,X = x)= μa + q(x). Hernán and Robins45 showed that the first assumption concerning the independence of U and X conditional on A could not be satisfied if both U and X were causes of A, and thus that the results of Lin et al11 concerning the conditional independence assumption could not be employed in those contexts in which the formulas would be most useful, ie, when both U and X contained confounding variables. VanderWeele20 showed that the second assumption of Lin et al concerning additivity held for an entire family of distributions even if both U and X were causes of A. Under this second assumption of additivity, Lin et al11 showed that when the conditional distribution of U given X and A is normal with mean μα,x = μα + q(x) then the regression coefficients β and β* were related by

for linear and log-linear links and that this relationship held approximately for a logistic link. Lin et al11 also noted that this relationship would hold for linear link (but not log-linear or logistic) if U were binary (rather than normally distributed) with a conditional mean μα,x = μα + q(x).

The results for a linear link follow immediately from the bias formula for da+(x) by replacing expressions of the form E(Y|a, u, x) with the linear combination of regression coefficients and by using E(U|a, x) = μa + q(x). See also Cox and McCullagh.32 Furthermore, from the bias formula for da+ it also follows that to obtain β = β* − γ(μ1 − μ0), the unmeasured confounding variable U does not need to be binary nor to be conditionally normally distribution; all that is needed to obtain β = β* − γ(μ1 − μ0), or more generally to obtain β = β* − γ(μa1 − μa0), is that the conditional mean of U given X and A is given by μa,x = μa + q(x). Not only do all of these results for linear regression follow immediately from Theorem 1, but the bias formula for da+(x) can in fact be used to relax the additivity assumption. Without making an assumption about additivity, it follows immediately from the bias formula for da+ that β and β* are related by

The bias formulas in Theorem 1 are yet more general than this in that, as we saw in the applications above, unlike the results of Lin et al11 and Imbens,14 Theorem 1 does not presuppose a regression context; furthermore it does not assume that there are no interactions between A, U, and X, and does not presuppose any particular functional form.

Relation to Prior External Adjustment Results

As noted above, the bias formulas in Theorem 1 are a generalization of prior bias formulas in the external adjustment literature.1,8,21 This prior external adjustment literature generally considered the setting of a dichotomous treatment, a dichotomous outcome, and a categorical unmeasured covariate. If in Theorem 1, X = ℏ, Y is dichotomous and U is categorical, the formulas given above reduce to the results for the risk difference in Kitagawa1 and Arah et al.21 The bias formula results given here thus generalize these prior external adjustment results in 2 ways. First, our results apply not only to dichotomous outcomes but also to continuous outcomes. Second, our results allow for control of some set of measured covariates X. Whereas prior risk difference results compared the average outcome difference unadjusted for X, E(Y|a1) − E(Y|a0), to the causal effect E(Ya1) − E(Ya0), our results compare the average outcome difference adjusted for X, Σx{E(Y|a1, x) − E(Y|a0, x)}P(x), to the causal effect E(Ya1) − E(Ya0). Most of the prior external adjustment literature effectively presupposed that the analysis was within strata of, or conditional on, X (or that there were no measured covariates to control for). To combine results over strata of X, Lee and Wang46 and Flanders and Khoury9 for instance propose an assumption of homogeneity of effects across of X; Greenland15,16 and Arah et al21 discuss Bayesian and Monte Carlo method approaches; using risk ratios, Flanders and Khoury9 also derive an external adjustment formula for the effect in the exposed group with both a measured and an unmeasured covariate.

REFERENCES

1.Kitagawa EM. Components of a difference between two rates. J Am Stat Assoc. 1955;50:1168–1194.
2.Cornfield J, Haenszel W, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder LL. Smoking and lung cancer: recent evidence and a discussion of some questions. J Natl Cancer Inst. 1959;22:173–203.
3.Bross ID. Spurious effects from an extraneous variable. J Chronic Dis. 1966;19:637–647.
4.Schlesselman JJ. Assessing effects of confounding variables. Am J Epidemiol. 1978;108:3–8.
5.Breslow NE, Day NE. Statistical Methods in Cancer Research. Vol 1: The Analysis of Case-Control Studies. Lyon: International Agency for Research on Cancer; 1980: chap 3.
6.Rosenbaum PR, Rubin DB. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J R Stat Soc Series B. 1983;45:212–218.
7.Yanagawa T. Case-control studies: assessing the effect of a confounding factor. Biometrika. 1984;71:191–194.
8.Gail MH, Wacholder S, Lubin JH. Indirect corrections for confounding under multiplicative and additive risk models. Am J Ind Med. 1988;13: 119–130.
9.Flanders WD, Khoury MJ. Indirect assessment of confounding: graphic description and limits on effect of adjusting for covariates. Epidemiology. 1990;1:239–246.
10.Copas JB, Li HG. Inference for non-random samples. J R Stat Soc Series B. 1997;59:55–95.
11.Lin DY, Psaty BM, Kronmal RA. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics. 1998;54:948–963.
12.Robins JM, Scharfstein D, Rotnitzky A. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In: Halloran E, Berry D, eds. Statistical Models for Epidemiology, the Environment, and Clinical Trials. New York: Springer-Verlag; 2000:1–95.
13.Rosenbaum PR. Observational Studies. 2nd ed. New York: Springer-Verlag; 2002.
14.Imbens GW. Sensitivity to exogeneity assumptions in program evaluation. Am Econ Rev. 2003;93:126–132.
15.Greenland S. The impact of prior distributions for uncontrolled confounding and response bias: a case study of the relation of wire codes and magnetic fields to childhood leukemia. J Am Stat Assoc. 2003;98: 47–54.
16.Greenland S. Multiple-bias modeling for analysis of observational data (with discussion). J R Stat Soc Series A. 2005;168:267–308.
17.Brumback BA, Hernán MA, Haneuse SJ, et al. Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat Med. 2004;23:749–767.
18.Stürmer T, Schneeweiss S, Avorn J, et al. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am J Epdiemiol. 2005;162:279–289.
19.McCandless LC, Gustafson P, Levy A. Bayesian sensitivity analysis for unmeasured confounding in observational studies. Stat Med. 2007;26: 2331–2347.
20.VanderWeele TJ. Sensitivity analysis: distributional assumptions and confounding assumptions. Biometrics. 2008;64:645–649.
21.Arah OA, Chiba Y, Greenland S. Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. Ann Epidemiol. 2008;18:637–646.
22.Lash TL, Fox MP, Fink AK. Applying Quantitative Bias Analysis to Epidemiologic Data. New York: Spring; 2009.
23.Manski CF. Nonparametric bounds on treatment effects. Am Econ Rev. 1990;80:319–323.
24.Manski CF. Monotone treatment response. Econometrica. 1997;65: 1311–1334.
25.Manski CF. Partial Identification of Probability Distributions. New York: Springer; 2003.
26.MacLehose RF, Kaufman S, Kaufman JS, et al. Bounding causal effects under uncontrolled confounding using counterfactuals. Epidemiology. 2005;16:548–555.
27.VanderWeele TJ. The sign of the bias of unmeasured confounding. Biometrics. 2008;64:702–706.
28.Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;66:688–701.
29.Rubin DB. Bayesian inference for causal effects: the role of randomization. Ann Stat. 1978;6:34–58.
30.Rubin DB. Formal modes of statistical inference for causal effects. J Stat Plann Infer. 1990;25:279–292.
31.Cox DR. The Planning of Experiments. New York: Wiley; 1958.
32.Cox DR, McCullagh P. Some aspects of the analysis of covariance. Biometrics. 1982;38:541–561.
33.Pearl J. Causality. Cambridge: Cambridge University Press; 2009.
34.Rosenbaum PR, Rubin DB. The central role of propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55.
35.Reinisch J, Sanders S, Mortensen E, et al. In-utero exposure to phenobarbital and intelligence deficits in adult men. J Am Med Assoc. 1995;274:1518–1525.
36.Rosenbaum PR, Rubin DB. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat. 1985;39:33–38.
37.Hernán MA, Robins JM. Estimating causal effects from epidemiologic data. J Epidemiol Community Health. 2006;60:578–586.
38.Kotelchuck M. An evaluation of the Kessner adequacy of prenatal care index and a proposed adequacy of prenatal care utilization index. Am J Public Health. 1994;84:1414–1420.
39.VanderWeele TJ, Lantos JD, Siddique J, et al. A comparison of four prenatal care indices in birth outcome models: comparable results for predicting small-for-gestational-age outcome but different results for preterm birth or infant mortality. J Clin Epidemiol. 2009;62:438–445.
40.Lambert D, Pregibon D. More bang for their bucks: assessing new features for online advertisers. In: Proceedings of the 1st International Workshop on Data Mining and Audience Intelligence for Advertising; 2007; San Jose, CA: 7-15.
41.Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994;89:846–866.
42.Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61:692–972.
43.Tan Z. A distributional approach for causal inference using propensity scores. J Am Stat Assoc. 2006;101:1619–1637.
44.Cao W, Tsiatis AA, Davidian M. Improving efficiency and robustness of the doubly robust estimator for a population mean with incomplete data. Biometrika. 2009;96:723–734.
45.Hernán M, Robins JM. Assessing the sensitivity of regression results to unmeasured confounders in observational studies [letter]. Biometrics. 1999;55:1316–1317.
46.Lee WC, Wang LY. Simple formulas for gauging the potential impacts of population stratification bias. Am J Epidemiol. 2008;167:86–89.

Supplemental Digital Content

© 2011 Lippincott Williams & Wilkins, Inc.