Institutional members access full text with Ovid®

Share this article on:

The Right Tool for the Job: Choosing Between Covariate-balancing and Generalized Boosted Model Propensity Scores

Setodji, Claude M.a; McCaffrey, Daniel F.b; Burgette, Lane F.a; Almirall, Danielc; Griffin, Beth Anna

doi: 10.1097/EDE.0000000000000734

Estimating the causal effect of an exposure (vs. some control) on an outcome using observational data often requires addressing the fact that exposed and control groups differ on pre-exposure characteristics that may be related to the outcome (confounders). Propensity score methods have long been used as a tool for adjusting for observed confounders in order to produce more valid causal effect estimates under the strong ignorability assumption. In this article, we compare two promising propensity score estimation methods (for time-invariant binary exposures) when assessing the average treatment effect on the treated: the generalized boosted models and covariate-balancing propensity scores, with the main objective to provide analysts with some rules-of-thumb when choosing between these two methods. We compare the methods across different dimensions including the presence of extraneous variables, the complexity of the relationship between exposure or outcome and covariates, and the residual variance in outcome and exposure. We found that when noncomplex relationships exist between outcome or exposure and covariates, the covariate-balancing method outperformed the boosted method, but under complex relationships, the boosted method performed better. We lay out criteria for when one method should be expected to outperform the other with no blanket statement on whether one method is always better than the other.

aRAND Corporation, Pittsburgh, PA; bEducational Testing Service, Princeton, NJ; and cInstitute for Social Research, University of Michigan, Ann Arbor, MI.

Editor’s Note: A commentary on this article appears on p. 812.

Submitted 23 February 2016; accepted 31 July 2017.

The authors report no conflicts of interest.

Supported by an National Institutes of Health grant R01DA034065 from the National Institute on Drug Abuse (NIDA) and a funding from the RAND center for causal inference.

All the listed authors were part of the team that conceptualized and designed the study. They all drafted different parts of the initial article, reviewed or revised it and approved the final article as submitted.

Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article (

Correspondence: Claude M. Setodji, RAND Corporation, 4570, 5th Avenue, Suite 600, Pittsburgh, PA 15213-2665. E-mail:

Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.