Mediators, also known as mediating variables, play a significant role in most models of behavioral intervention. In many programs, changes in mediators are targeted to promote change on the outcome of interest. Mediation occurs as part of a hypothesized causal chain of events: The independent variable (e.g., intervention) has an effect on the mediator (e.g., risky relationship thoughts), which then affects the outcome (e.g., sex without condom). Identification of mediators can reveal important pathways that lead to behavioral change (MacKinnon, 2008).
An important step in making interventions more efficacious and cost-effective is to identify the mechanisms by which the intervention has an effect on the outcome. That is, researchers not only need to know that the intervention works but also need to know how it works. Therefore, establishing the causal link between the mediator and the outcome is critical. The commonly used Baron and Kenny (1986) approach to mediation and its extensions (MacKinnon, 2008; see also, e.g., Krause et al., 2010; Levy, Landerman, & Davis, 2011, for introductions to these extensions for nursing researchers) rely on an assumption of no unmeasured confounding, which holds if individuals are randomly assigned to both the intervention and the mediator. However, as noted in the statistics and methodology literature (e.g., Robins & Greenland, 1992; Rosenbaum, 1984), establishing this causal link can be difficult because randomization of individuals to levels of the mediator is usually impossible. Therefore, confounders that influence both the mediator and the outcome may exist and bias the estimate of the effect of the mediator on the outcome. The potential outcomes framework (Rubin, 1974) for causal inference has been proposed for assessing mediation (e.g., Coffman, 2011; Imai, Keele, & Tingley, 2010; Jo, 2008; VanderWeele, 2009). This framework clearly defines the causal effects to be estimated and states the assumptions needed to do so.
The goal of the present article is to introduce an approach to mediation that falls under the potential outcomes framework for causal inference. We will discuss the scientific question(s) that may be addressed and the assumptions required for identification and estimation of causal mediation effects. Much has been written in the statistics and epidemiology literature about the various approaches to assessing mediation, but there are very few published applications of these approaches and, to our knowledge, no published applications in the nursing research literature.
After three decades, the persistence of the HIV/AIDS epidemic in the United States requires sustained attention. Distinct subgroups are infected disproportionately with HIV, such as individuals who enter the criminal justice system, especially women, with three to four times greater infection rates than the general population (Maruschak, 2009).
The risk factors underlying the disproportionate HIV rates among incarcerated women include exchange of sex for drugs or money, having a high-risk sexual partner, inconsistent condom use, and use of drugs or other substances (Cotton-Oldenburg, Jordan, Martin, & Kupper, 1999); these often occur at greater frequency than in the general population. The underlying causes for greater frequency of these risk factors is not clear, but it is important to intervene with this population because many incarcerated women return to the same behaviors, relationships, and financial context upon release (Adams et al., 2011). To date, very few interventions have focused on this population (Lichtenstein & Malow, 2010).
To fill in the gap in the literature, the Reducing Risky Relationships-HIV (RRR-HIV) intervention, as part of the Criminal Justice Drug Abuse Treatment Studies cooperative agreement (Fletcher & Wexler, 2005), was developed, piloted, and tested among incarcerated women (Havens et al., 2009; Staton-Tindall et al., 2007). The intervention was designed to address the context of a relationship and to change risky relationship thoughts with the aim of decreasing risky sex after release (Staton-Tindall et al., 2007). Risky relationship thoughts mediate the effect of the intervention on risky sexual behavior. That is, the intervention was hypothesized to change risky relationship thoughts, which in turn were hypothesized to change risky sexual behaviors. These effects are denoted with β1 and β2, respectively, in Figure 1. Also illustrated in Figure 1 is the direct effect, β3; that is, the effect of the intervention on risky sexual behavior that is not due to risky relationship thoughts.
Potential Outcomes Framework for Causal Inference
In the potential outcomes framework (Rubin, 1974), each individual has a potential outcome for each possible treatment condition. For simplicity, consider a binary treatment indicator, Ti, wherein Ti = 1 denotes the intervention condition and Ti = 0 denotes the control condition for participant i, i = 1,…,n. In this case, there are two potential outcomes for each individual: the potential outcome if the individual receives the intervention, denoted Yi(1), and the potential outcome if the individual is in the control condition, denoted Yi(0). The individual causal effect is the difference between these two potential outcomes. Because each participant is observed in only one of these conditions, only one potential outcome is observed; the other potential outcome is missing, and therefore, the individual causal effect cannot be computed. However, strategies have been implemented to estimate the causal effect averaged over participants in the study, called the average causal effect. The average causal effect is defined as E[Yi(1) − Yi(0)]; that is, the expected (or average) difference between the two potential outcomes. Introductions to the potential outcomes framework outside the mediation context are provided by Little and Rubin (2000) and Schafer and Kang (2008).
A mediator is an outcome assessed within the context of the intervention, and therefore, there are also potential values for the mediator under each intervention condition for each individual: the potential value of the mediator under the intervention condition, denoted Mi(1), and the potential value of the mediator under the control condition, denoted Mi(0). The potential values for the outcome are then expanded to include potential values of the mediator. Thus, Yi(1,Mi(1)) is the potential outcome if individual i receives the intervention, and Yi(0,Mi(0)) is the potential outcome if individual i is in the control condition. As before, only one of these two potential outcomes is observed for each individual.
Throughout this article, Yi denotes the observed value of unprotected sex (operationalized in this study as having had at least one sexual encounter without a condom in the previous 30 days), Mi denotes the observed value for risky relationship thoughts, and Yi(ti,Mi(ti)) denotes potential outcomes, wherein ti is one of the levels of the intervention. Xi denotes measured confounders. It is assumed throughout that if an individual receives the intervention, then Yi = Yi(1) = Yi(1,Mi(1)) and Mi = Mi(1). Likewise, if an individual is in the control condition, then Yi = Yi(0) = Yi(0,Mi(0)) and Mi = Mi(0). This usually is referred to as the consistency assumption. Also implicit in this notation is that there is no interference among individuals, because the potential outcomes are a function of only Ti and not Tj, wherein i and j denote two different individuals. In other words, an individual’s outcome does not depend on another individual’s treatment assignment. Throughout this article, this noninterference is assumed.
Using the Potential Outcomes Framework to Define Mediation Effects
Three different definitions of mediation using the potential outcomes framework have been introduced in the statistics literature: principal strata effects, natural effects, and controlled effects. The focus here is on controlled effects, first introduced by Robins and Greenland (1992).
In the motivating example, the controlled direct effect is the causal effect of the RRR-HIV intervention on unprotected sex when risky relationship thoughts is set (i.e., held constant) to a specific value, m, for the entire population. That is, E[Yi(1,m) − Yi(0,m)], wherein Yi(t,m) is the potential outcome when Ti = ti and Mi(ti) = m. For the controlled direct effect, the mediator is set to the same value for every individual. Also for a binary intervention, such as RRR-HIV, there are as many controlled direct effects as there are possible values of the mediator. If the controlled direct effects are different across levels of the mediator, it implies that there is an interaction between the intervention and the mediator. If the controlled direct effects are equal across levels of the mediator, it implies that there is no interaction between the intervention and mediator.
Next, consider defining the effect E[Yi(1,m) − Yi(1,m′)] for two different values of m and m′. This is the effect of, for example, a one-unit change in risky relationship thoughts on unprotected sex when Ti = 1. Similarly, the difference E[Yi(0,m) − Yi(0,m′)] defines the effect of a one-unit change in risky relationship thoughts on unprotected sex when Ti = 0. These differences define the effect of risky relationship thoughts on unprotected sex at each level of the intervention. The effect of the intervention on risky relationship thoughts is defined as E[Mi(1) − Mi(0)]. Finally, if there is no interaction between the intervention and mediator, such that there is only one controlled direct effect, the controlled direct effect may be subtracted from the total effect to obtain the indirect effect. The total effect is defined as the effect of the intervention on unprotected sex, E[Yi(1) − Yi(0)].
Identification and Estimation
Much less attention has been given to estimating mediation via controlled effects than via natural effects or principal strata effects. However, as discussed by VanderWeele (2009), if there is no interaction between the intervention and the mediator, then the controlled direct effect is the same for every level of the mediator. In this case, the controlled direct effect can be subtracted from the total effect to obtain the indirect effect. This approach requires four assumptions: (a) that there are no unmeasured confounders of the intervention and unprotected sex; (b) that there are no unmeasured confounders of the intervention and risky relationship thoughts; (c) that there are no unmeasured confounders of risky relationship thoughts and unprotected sex; and (d) that there are no interactions between the intervention and risky relationship thoughts. If individuals are randomized to levels of the intervention, as they are in the RRR-HIV study, then Assumptions (a) and (b) hold. However, this randomization does not imply that Assumption (c) holds. Randomization guarantees that the intervention groups are equivalent on prerandomization variables, but it does not preclude the possibility of differences between the intervention groups on postrandomization variables. Furthermore, randomization to levels of the intervention does not mean that individuals are randomized to levels of the mediator. Without randomization to levels of the mediator, there is no guarantee that there are no confounders of the mediator and outcome.
Coffman and Zhong (2011) proposed to define and estimate all of the effects given above using marginal structural models (MSMs) with an inverse propensity weighted (IPW) estimator using identifying Assumptions (a) through (c). MSMs are models for the potential outcomes and have been described previously, along with IPW estimation, in the prevention literature (e.g., Bray, Almirall, Zimmerman, Lynam, & Murphy, 2006; Coffman, Caldwell, & Smith, in press) and epidemiology literature (e.g., Cole et al., 2003; Robins, Hernan, & Brumback, 2000). For example, for continuous outcomes, the MSMs may be given as
is the controlled direct effect defined above,
is the effect of the intervention on risky relationship thoughts, and
is the effect of risky relationship thoughts on unprotected sex, holding constant intervention condition. The MSMs are fit by choosing an appropriate model for the observed outcome (e.g., linear regression, logistic regression, and survival model) and using the IPW estimator rather than the usual ordinary least squares or maximum likelihood estimator. If individuals are randomized to levels of the intervention, then Assumption (a) holds, and weights are unnecessary for estimating Equation (1). The models for the observed data, as opposed to the potential outcomes, are given as
Coffman and Zhong (2011) also proposed a null hypothesis test of no mediation. Specifically, the null hypothesis is that either the effect of the intervention on risky relationship thoughts or the effect of risky relationship thoughts on unprotected sex, holding constant intervention condition, is zero. If Assumption (d) holds, then an estimate of the indirect effect may be obtained as described above by subtracting the controlled direct effect from the total effect. However, it is not necessary to make Assumption (d). Although an estimate of the indirect effect itself requires Assumption (d), the null hypothesis test of no mediation is still valid, and unbiased estimates can be obtained of the causal effect of the intervention on risky relationship thoughts and of risky relationship thoughts on unprotected sex, holding constant intervention condition regardless of whether Assumption (d) holds.
Study Design and Participants
The data for these analyses came from the RRR-HIV intervention. Briefly, the intervention includes five group sessions in prison (prior to release) and one individual telephone or face-to-face session after release. The purpose of the intervention is to change risky relationship thoughts, such as “Having sex without a condom will strengthen my relationship,” with the intent of reducing HIV-risk behaviors. Participants were recruited from correctional facilities in Connecticut, Delaware, Kentucky, and Rhode Island. Women were eligible to participate if they were at least 18 years of age, scheduled to go before the parole board in the next 6 weeks, had at least weekly substance use before incarceration, and were willing to be randomized to study condition. Intervention details, including consent procedures, are described in Havens et al. (2009) and Staton-Tindall et al. (2007).
Data were collected at three time points: baseline, 30 days postintervention, and 90 days postintervention. The participants include 243 women who were present at each of the three time points. The characteristics of this sample can be found in Table 1.
All study measures were based on participant self-report data collected by trained research staff. Each of the measures reports on behavior in the previous 30 days. The outcome for these analyses, measured at 90 days postintervention, is unprotected sex, a binary variable defined as 1 = having had sex at least once without a condom in the previous 30 days and 0 = no sex without a condom or no sex in the previous 30 days. The mediator, risky relationship thoughts, is a scale score computed by taking the mean of six items. The scale of each item ranged from 1 (never) to 10 (everyday); higher scores correspond to more frequent risky relationship thoughts. Coefficient alpha for the items was .68 (95% confidence interval [CI]: 0.61, 0.74; Maydeu-Olivares, Coffman, & Hartmann, 2007). The items were based on focus groups by the study investigators prior to implementation of the intervention (Staton-Tindall et al., 2007).
Eighteen characteristics were included in the propensity score model to control for the potential confounding between the mediator and the outcome. Baseline sociodemographic characteristics were included, such as race (Black vs. not), housing (own home vs. other), live with spouse or partner (yes or no), marital status (married, previously married, or single), high school graduate (yes or no), employment status (full-time, part-time, or unemployed), financial support from job (number of months), and have medical insurance (yes or no). They also included numerous baseline items, such as arrested (yes or no); spent time in jail (yes or no); been abused physically, sexually, or emotionally in the previous 6 months (yes or no); drank alcohol (yes or no); alcohol use problems (mean of eight-item scale, 0 = never to 4 = always); received drug treatment since locked up (yes or no); drug use problems (mean of eight-item scale, 0 = never to 4 = always); substance use problems (mean of three-item scale, 1 = not at all to 4 = extremely); substance use treatment important after release (yes or no); and sex without a condom at baseline intake (yes or no).
Assumptions (a) through (c) were made, and MSMs and an IPW estimator were used. The IPW relies on propensity scores, defined as the probability that an individual receives a particular level of the intervention or exposure variable given measured confounders (Rosenbaum & Rubin, 1983).
Estimating propensity scores and creating weights for continuous variables, such as risky relationship thoughts, is only slightly more difficult than it is for a binary variable. The propensity score may be obtained from the probability density function (p.d.f.) of risky relationship thoughts given the measured confounders and treatment history, φ(Mi|Xi,Ti) (Robins et al., 2000). The propensity scores are obtained by a linear regression of Mi on Xi and Ti, and a probability is obtained by inserting the fitted values from the regression, denoted
in the normal p.d.f. (denoted as φ())
is the residual standard error from the regression of Mi on Xi and Ti.
Essentially, this approach treats the intervention–mediator sequence as a time-varying treatment. Therefore, the probability for the numerator of the weights for risky relationship thoughts is given by the p.d.f. of M given treatment history, φ(Mi|Ti). The probability for the denominator of the weights for the mediator is given by the p.d.f. of M given treatment history and the measured confounders, φ(Mi|Ti,Xi); the weights for risky relationship thoughts are φ(Mi|Ti)/ φ(Mi|Ti,Xi). For further details about creating weights and the numerator and denominator models for the weights, see Cole and Hernan (2008) and Robins et al. (2000). After the weights are created, they are incorporated into Equation (4) in the same manner as survey weights using SAS PROC GENMOD. SAS PROC GLM was used for fitting Equation (3), because no weights were needed because the intervention was randomized (see Document, Supplemental Digital Content 1, for an example of SAS code, http://links.lww.com/NRES/A69).
Figure 2 illustrates what is commonly referred to as balance on the measured confounders and presents the correlations between risky relationship thoughts and the confounders included in the propensity model before and after weighting. It is recommended that these differences be less than .1 (in absolute value), which is considered a small effect size (Cohen, 1988). Before weighting, several of the correlations were greater than |.1|, but after weighting, they were all less than |.1|. If they were not all less than |.1|, the propensity model should be revisited and interaction or quadratic terms added to the propensity model until balance is achieved.
To determine whether the effect of the intervention was mediated by risky relationship thoughts, each of the effects was estimated as described above. The effect of the intervention on risky relationship thoughts was significant (β1 = −0.529, p = .03, 95% CI: −1.00, −0.06), such that the intervention resulted in a decrease in risky relationship thoughts. The effect of risky relationship thoughts on unprotected sex was also statistically significant (β2 = 0.447, p < .001, 95% CI: 0.22, 0.68). A one-unit increase in risky relationship thoughts resulted in a 1.56 times increase in the odds of unprotected sex. The effect of the intervention on unprotected sex, holding risky relationship thoughts constant, was not significant (β3 = 0.388, p = .479, 95% CI: −0.69, 1.46). Also included was a term for the interaction between the intervention and risky relationship thoughts; however, it was not statistically significant (−0.112, p = .495, 95% CI: −0.43, 0.21). The total effect of the RRR-HIV intervention on unprotected sex (i.e., risky relationship thoughts were not in the model) was not significant (−0.092, p = .731, 95% CI: −0.61, 0.43). The requirement of a significant total effect is controversial because there can be a mediated effect even if the total effect is not significant. For example, this may happen if there is another mediator and the effect through that mediator cancels out the effect through the original mediator. It may happen also if there are moderators, and there is a positive mediated effect in one subgroup and a negative mediated effect in another subgroup. It is now generally accepted in the statistical mediation literature that a significant total effect is not required (MacKinnon, 2008).
Following the procedures in Coffman and Zhong (2011), we rejected the null hypothesis that either the effect of the intervention on risky relationship thoughts or the effect of risky relationship thoughts on unprotected sex, holding constant intervention condition, was zero (p = .028). Based on this information, we concluded that the intervention’s effect on unprotected sex was indeed mediated by risky relationship thoughts.
Mediation is, by definition, a question about causal pathways. Even if individuals are randomly assigned to levels of the intervention and randomization does not fail (e.g., no noncompliance), this does not imply that individuals are randomly assigned to levels of the mediator. In fact, confounders of the mediator and outcome almost always exist. Without proper control of these confounders, the estimate of the effect of the mediator on the outcome and the estimate of the direct effect of the intervention on the outcome will be biased. Using IPW is one approach in controlling for confounders. Another approach would be to control for all confounders using regression adjustment (i.e., ANCOVA); however, propensity scores are advantageous because they reduce a potentially large number of confounders into a single-number summary. Furthermore, regression adjustment may still result in biased estimates of the direct effect if posttreatment confounders are included in the regression model (Robins et al., 2000).
The RRR-HIV intervention was used as a motivating example because of the public health significance of HIV/AIDS, especially among incarcerated women. However, the public use data set posed challenges because access to all the potential mediators and confounders of this study were unavailable. In particular, there was no measure of the mediator at baseline. The baseline measure of the mediator is an obvious potential confounder of the mediator at 30-day follow-up and of the outcome at 90-day follow-up. The method described above relies on the assumption that all confounders are measured and controlled properly. This is a very strong assumption that cannot be tested in practice. However, the more potential confounders included in the propensity model, the more plausible the assumption becomes. Thus, it is imperative that researchers measure as many potential confounders as possible. In addition, the impact of the unmeasured confounder is mitigated if a measured potential confounder is highly correlated with the unmeasured confounder.
Nevertheless, a next step with approaches that rely on the assumption of no unmeasured confounding is to conduct a sensitivity analysis, which attempts to determine how strongly influential an unmeasured confounder would need to be to change the estimate in a meaningful way (e.g., change the estimate from statistically significant to not, reverse the sign of the estimate). Sensitivity analysis is being developed for continuous exposures/mediators. Despite challenges of the data set, it was possible to demonstrate how this approach can be used to assess mediation. It should be noted that the traditional regression approach to mediation, which is likely familiar to most readers, also assumes no unmeasured confounding of the mediator and outcome, although this assumption is rarely stated explicitly. Furthermore, in the traditional approach, researchers usually control for only a few demographic variables, if they control for any potential confounders at all. As shown by Steiner, Cook, Shadish, and Clark (2010), controlling for only a few demographic variables is generally insufficient to obtain unbiased estimates.
For comparison, the model was fit without the weights; that is, a logistic regression of unprotected sex on the intervention and risky relationship thoughts. As with the IPW estimates, the effect of the intervention on unprotected sex, holding constant risky relationship thoughts, was not statistically significant. The effect of risky relationship thoughts on unprotected sex, holding constant the intervention condition, was statistically significant (0.352, p < .001, 95% CI: 0.20, 0.50). Thus, a one-unit increase in risky relationship thoughts resulted in a 1.42 times increase in the odds of unprotected sex. This estimate is not dramatically different from the IPW estimate (the CIs overlap); however, it is a 14% difference in the odds ratio point estimates.
The controlled effects approach is valuable in nursing research. Controlled effects differ from the traditional regression approach to mediation, primarily in that weights are used in estimation, accounting for potential confounding. The controlled effects approach addresses questions such as “What is the effect of the RRR-HIV intervention on unprotected sex, holding constant the level of risky relationship thoughts?”, “What is the effect of the RRR-HIV intervention on risky relationship thoughts?”, and “What is the effect of risky relationship thoughts on unprotected sex, holding constant intervention status?” If Assumption (d) holds, then this approach also addresses the question: “What is the effect of the RRR-HIV intervention on unprotected sex that is due to risky relationship thoughts?” The indirect effect itself is not identified unless there is no interaction between the intervention and mediator. Nevertheless, the two effects that make up the indirect effect (i.e., the effect of the intervention on the mediator and the effect of the mediator on the outcome, holding constant the intervention condition) are identified regardless of whether or not there is an interaction between the intervention and mediator. In summary, an advantage of the potential outcomes framework is that it allows for the careful definition of causal effects and of the assumptions needed for identification and estimation of the causal effects.
Adams J., Nowels C., Corsi K., Long J., Steiner J. F., Binswanger I. A. (2011). HIV risk after release from prison: A qualitative study of former inmates. Journal of Acquired Immune Deficiency Syndromes, 57, 429–434.
Baron R. M., Kenny D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.
Bray B. C., Almirall D., Zimmerman R. S., Lynam D., Murphy S. A. (2006). Assessing the total effect of time-varying predictors in prevention research. Prevention Science, 7, 1–17.
Coffman D. L. (2011). Estimating causal effects in mediation analysis using propensity scores. Structural Equation Modeling, 18, 357–369.
Coffman, D. L., Caldwell, L., & Smith, E. (in press). Introducing the at-risk average causal effect with application to HealthWise South Africa. Prevention Science
. doi: 10.1007/s11121-011-0271-0
Coffman, D. L., & Zhong, W. (2012). Assessing mediation using marginal structural models in the presence of confounding and moderation. Psychological Methods. Manuscript accepted for publication.
Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: LEA.
Cole S. R., Hernan M. A. (2008). Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology, 168, 656–664.
Cole S. R., Hernan M. A., Robins J. M., Anastos K., Chmiel J., Detels R., Munoz A. (2003). Effect of highly active antiretroviral therapy on time to aquired immunodeficiency syndrome or death using marginal structural models. American Journal of Epidemiology, 158, 687–694.
Cotton-Oldenburg N. U., Jordan B. K., Martin S. L., Kupper L. (1999). Women inmates’ risky sex and drug behaviors: Are they related? American Journal of Drug and Alcohol Abuse, 25, 129–149.
Fletcher B. W., Wexler H. K. (2005). National Criminal Justice Drug Abuse Treatment Studies (CJ-DATS): Update and progress. Justice Research and Statistics Association Forum, 23, 1–7.
Havens J. R., Leukefeld C. G., Oser C. B., Staton-Tindall M., Knudsen H. K., Mooney J., Inciardi J. A. (2009). Examination of an interventionist-led HIV intervention among criminal justice-involved female prisoners. Journal of Experimental Criminology, 5, 245–272.
Imai K., Keele L., Tingley D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334.
Jo B. (2008). Causal inference in randomized experiments with mediational processes. Psychological Methods, 13, 314–336.
Krause M. R., Serlin R. C., Ward S. E., Rony R. Y., Ezenwa M. O., Naab F. (2010). Testing mediation in nursing research: Beyond Baron and Kenny. Nursing Research, 59, 288–293.
Levy J. A., Landerman L. R., Davis L. L. (2011). Advances in mediation analysis can facilitate nursing research. Nursing Research, 60, 333–339.
Lichtenstein B., Malow R. (2010). A critical review of HIV-related interventions for women prisoners in the United States. Journal of the Association of Nurses in AIDS Care, 21, 380–394.
Little R. J. A., Rubin D. B. (2000). Causal effects in clinical and epidemiological studies via potential outcomes: Concepts and analytical approaches. Annual Review of Public Health, 21, 121–145.
MacKinnon D. P. (2008). Introduction to statistical mediation analysis. New York, NY: LEA.
Maruschak L. M. (2009). HIV in Prisons, 2007–08. Washington, DC: U.S. Department of Justice, Bureau of Justice Statistics.
Maydeu-Olivares A., Coffman D. L., Hartmann W. M. (2007). Asymptotically distribution-free interval estimation for coefficient alpha. Psychological Methods, 12, 157–176.
Robins J. M., Greenland S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3, 143–155.
Robins J. M., Hernan M. A., Brumback B. A. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11, 550–560.
Rosenbaum P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society, Series A (General), 147, 656–666.
Rosenbaum P. R., Rubin D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rubin D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688–701.
Schafer J. L., Kang J. (2008). Average causal effects from non-randomized studies: A practical guide and simulated example. Psychological Methods, 13, 279–313.
Staton-Tindall M., Leukefeld C., Palmer J., Oser C., Kaplan A., Krietemeyer J., Surratt H. L. (2007). Relationships and HIV risk among incarcerated women. The Prison Journal, 87, 143–165.
Steiner P. M., Cook T. D., Shadish W. R., Clark M. H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15 (3), 250–267.
VanderWeele T. J. (2009). Marginal structural models for the estimation of direct and indirect effects. Epidemiology, 20, 18–26.