Institutional members access full text with Ovid®

Share this article on:

Using Electronic Health Care Records for Drug Safety Signal Detection: A Comparative Evaluation of Statistical Methods

Schuemie, Martijn J. PhD*; Coloma, Preciosa M. MD, MS*; Straatman, Huub MS; Herings, Ron M. C. PhD*,†; Trifirò, Gianluca MD, PhD*,‡; Matthews, Justin Neil MS§; Prieto-Merino, David PhD§; Molokhia, Mariam PhD; Pedersen, Lars PhD; Gini, Rosa PhD#; Innocenti, Francesco MS#,**; Mazzaglia, Giampiero MD, PhD**; Picelli, Gino††; Scotti, Lorenza MS‡‡; van der Lei, Johan MD, PhD*; Sturkenboom, Miriam C. J. M. PhD*

doi: 10.1097/MLR.0b013e31825f63bf
Original Articles

Background: Drug safety monitoring relies primarily on spontaneous reporting, but electronic health care record databases offer a possible alternative for the detection of adverse drug reactions (ADRs).

Objectives: To evaluate the relative performance of different statistical methods for detecting drug-adverse event associations in electronic health care record data representing potential ADRs.

Research Design: Data from 7 databases across 3 countries in Europe comprising over 20 million subjects were used to compute the relative risk estimates for drug-event pairs using 10 different methods, including those developed for spontaneous reporting systems, cohort methods such as the longitudinal gamma poisson shrinker, and case-based methods such as case-control. The newly developed method “longitudinal evaluation of observational profiles of adverse events related to drugs” (LEOPARD) was used to remove associations likely caused by protopathic bias. Data from the different databases were combined by pooling of data, and by meta-analysis for random effects. A reference standard of known ADRs and negative controls was created to evaluate the performance of the method.

Measures: The area under the curve of the receiver operator characteristic curve was calculated for each method, both with and without LEOPARD filtering.

Results: The highest area under the curve (0.83) was achieved by the combination of either longitudinal gamma poisson shrinker or case-control with LEOPARD filtering, but the performance between methods differed little. LEOPARD increased the overall performance, but flagged several known ADRs as caused by protopathic bias.

Conclusions: Combinations of methods demonstrate good performance in distinguishing known ADRs from negative controls, and we assume that these could also be used to detect new drug safety signals.

Supplemental Digital Content is available in the text.

*Department of Medical Informatics, Erasmus University Medical Center, Rotterdam

PHARMO Institute, Utrecht, The Netherlands

Department of Clinical and Experimental Medicine and Pharmacology, University of Messina, Messina, Italy

§Faculty of Epidemiology and Population Health, London School of Hygiene & Tropical Medicine

Department of Primary Care and Public Health Sciences, Kings College London, UK

Department of Clinical Epidemiology, Aarhus University Hospital, Århus Sygehus, Denmark

#Agenzi Regionali di Sanità della Toscana

**Health Search, Italian College of General Practitioners

††Pedianet, Societa’ Servizi Telematici SRL

‡‡Department of Statistics, Università di Milano-Bicocca, Italy

Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Website,

On behalf of the EU-ADR consortium.

Supported by the European Commission FP7 Program (FP7/2007-2013) under grant no. 215847 (the EU-ADR Project).

The authors declare no conflict of interest.

Reprints: Martijn J. Schuemie, PhD, Department of Medical Informatics, Erasmus University Medical Center Rotterdam, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands. E-mail:

© 2012 Lippincott Williams & Wilkins, Inc.