Nonrandomized safety and effectiveness studies are often initiated immediately after the approval of a new medication, but patients prescribed the new medication during this period may be substantially different from those receiving an existing comparator treatment. Restricting the study to comparable patients after data have been collected is inefficient in prospective studies with primary collection of outcomes.
We discuss design and methods for evaluating covariate data to assess the comparability of treatment groups, identify patient subgroups that are not comparable, and decide when to transition to a large-scale comparative study. We demonstrate methods in an example study comparing Cox-2 inhibitors during their postmarketing period (1999–2005) with nonselective nonsteroidal anti-inflammatory drugs (NSAIDs).
Graphical checks of propensity score distributions in each treatment group showed substantial problems with overlap in the initial cohorts. In the first half of 1999, >40% of patients were in the region of nonoverlap on the propensity score, and across the study period this fraction never dropped below 10% (the a priori decision threshold for transitioning to the large-scale study). After restricting to patients with no prior NSAID use, <1% of patients were in the region of nonoverlap, indicating that a large-scale study could be initiated in this subgroup and few patients would need to be trimmed from analysis.
A sequential study design that uses pilot data to evaluate treatment selection can guide the efficient design of large-scale outcome studies with primary data collection by focusing on comparable patients.
Supplemental Digital Content is available in the text.
From the aDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA; bDepartment Global Epidemiology, Boehringer Ingelheim GmbH, Ingelheim, Germany; and cInstitute for Epidemiology, Social Medicine and Health Systems Research, Hannover Medical School, Hannover, Germany.
This research was funded by an investigator-initiated contract from Boehringer Ingelheim GmbH to the Brigham and Women’s Hospital
Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article (www.epidem.com). This content is not peer-reviewed or copy-edited; it is the sole responsibility of the author.
Correspondence: Jessica M. Franklin, 1620 Tremont St., Suite 3030, Boston, MA 02120. E-mail: JMFranklin@partners.org.