Institutional members access full text with Ovid®

Share this article on:

Comparison of Treatment Effect Estimates From Prospective Nonrandomized Studies With Propensity Score Analysis and Randomized Controlled Trials of Surgical Procedures

Lonjon, Guillaume MD*,†; Boutron, Isabelle MD, PhD*,‡,§,¶; Trinquart, Ludovic MSc*,‡,§,¶; Ahmad, Nizar MD, PhD*,‡; Aim, Florence MD*,†; Nizard, Rémy MD, PhD; Ravaud, Philippe MD, PhD*,‡,§,¶,‖

doi: 10.1097/SLA.0000000000000256

Objective: We aimed to compare treatment effect estimates from NRSs with PS analysis and RCTs of surgery.

Background: Evaluating a surgical procedure in randomized controlled trials (RCTs) is challenging. Nonrandomized studies (NRSs) involving use of propensity score (PS) analysis to limit bias are of increasing interest.

Design: Meta-epidemiological study.

Methods: We systematically searched MEDLINE via PubMed for all prospective NRSs with PS analysis evaluating a surgical procedure. Related RCTs, addressing the same clinical questions, were systematically retrieved. Our primary outcome of interest was all-cause mortality. We also selected 1 subjective outcome. We calculated the summary odds ratios (OR) for each study design, the ratio of OR (ROR) between the designs and the summary ROR across clinical questions. An ROR < 1 indicated that the experimental intervention is more favorable in NRSs with PS analysis than RCTs.

Results: We retrieved 70 reports of NRSs with PS analysis and 94 related RCTs evaluating 31 clinical questions, of which 22 assessed all-cause mortality and 26 a subjective outcome. The combined ROR for all-cause mortality was 0.83 (95% confidence interval: 0.65–1.04). For subjective outcomes, the combined ROR was 1.07 (0.87–1.33).

Conclusions: There was no statistically significant difference in treatment effect between NRSs with PS analysis and RCTs. Prospective NRSs with suitable and careful PS analysis can be relied upon as evidence when RCTs are not possible.

Supplemental Digital Content is Available in the Text.Nonrandomized studies with propensity score (PS) analysis are an alternative at randomized controlled trials (RCTs). We aimed to compare treatment effect estimates from PS analysis and RCTs of surgery. There was no statistically significant difference in treatment effect between nonrandomized studies with PS analysis and RCTs.

*Unité INSERM U738

Assistance Publique-Hôpitaux de Paris, Hôpital Lariboisière, Service d'orthopédie et traumatologie

Assistance Publique-Hôpitaux de Paris, Hôpital Hôtel-Dieu, Centre d'Epidémiologie Clinique

§Université Paris Descartes—Sorbonne Paris Cité

French Cochrane Centre, Paris, France; and

Columbia University, Mailman School of Public Health, NY.

Reprints: Guillaume Lonjon, MD, Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu, 1 place du parvis Notre Dame, 75004 Paris, France. E-mail:

Disclosure: This study received source from Equipe “Espoirs de la Recherche” par la Fondation pour la Recherche Médicale (FRM). All the authors declare no support from any organization for the submitted work; no financial relationships with any organization that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (

© 2014 by Lippincott Williams & Wilkins.