Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Development of an Emergency Department Trigger Tool Using a Systematic Search and Modified Delphi Process

Griffey, Richard Thomas MD, MPH, FACEP*; Schneider, Ryan M. ACNP-BC*; Adler, Lee M. MD; Capp, Roberta MD; Carpenter, Christopher R. MD, MSc*; Farmer, Brenna M. MD§; Groner, Kathyrn Y. MD; Hodkins, Sheridan RN, MSN; McCammon, Craig A. PharmD, BCPS; Powell, Jonathan T. MD; Sather, Jonathan E. MD**; Schuur, Jeremiah D. MD, MHS††; Shapiro, Marc J. MD**; Sharp, Brian R. MD‡‡; Venkatesh, Arjun K. MD, MBA**; Vrablik, Marie C. MD§§; Wiler, Jennifer L. MD, MBA

doi: 10.1097/PTS.0000000000000243
Original Article: PDF Only

Objective This study aimed to develop an emergency department (ED) trigger tool to improve the identification of adverse events in the ED and that can be used to direct patient safety and quality improvement. This work describes the first step toward the development of an ED all-cause harm measurement tool by experts in the field.

Methods We identified a multidisciplinary group of emergency medicine safety experts from whom we solicited candidate triggers. We then conducted a modified Delphi process consisting of 4 stages as follows: (1) a systematic literature search and review, including an independent oversampling of review for inclusion, (2) solicitation of empiric triggers from participants, (3) a Web-based survey ranking triggers on specific performance constructs, and (4) a final in-person meeting to arrive at consensus triggers for testing. Results of each step were shared with participants between each stage.

Results Among an initial 804 unique articles found using our search criteria, we identified 94 that were suitable for further review. Interrater reliability was high (κ = 0.80). Review of these articles yielded 56 candidate triggers. These were supplemented by 58 participant-submitted triggers yielding a total of 114 candidate triggers that were shared with team members electronically along with their definitions. Team members then voted on each measure via a Web-based survey, ranking triggers on their face validity, utility for quality improvement, and fidelity (sensitivity/specificity). Participants were also provided the ability to flag any trigger about which they had questions or they felt merited further discussion at the in-person meeting. Triggers were ranked by combining the first 2 categories (face validity and utility), and information on fidelity was reviewed for decision making at the in-person meeting. Seven redundant triggers were eliminated. At an in-person meeting including representatives from all facilities, we presented the 50 top-ranked triggers as well as those that were flagged on the survey by 2 or more participants. We reviewed each trigger individually, identifying 41 triggers about which there was a clear agreement for inclusion. Of the seven additional triggers that required subsequent voting via e-mail, 5 were adopted, arriving at a total of 46 consensus-derived triggers.

Conclusions Our modified Delphi process resulted in the identification of 46 final triggers for the detection of adverse events among ED patients. These triggers should be pilot field tested to quantify their individual and collective performance in detecting all-cause harm to ED patients.

From the *Division of Emergency Medicine, Washington University School of Medicine, St. Louis, Missouri; †University of Central Florida College of Medicine, Orlando, Florida; Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida; ‡Department of Emergency Medicine, University of Colorado School of Medicine, Denver, Colorado; §Department of Emergency Medicine, Weill Cornell Medical College, New York, New York; ∥Department of Emergency Medicine, Christiana Care, Wilmington, Delaware; ¶Barnes-Jewish Hospital, St. Louis, Missouri; **Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut; ††Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts; ‡‡Department of Emergency Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin; and §§Division of Emergency Medicine, University of Washington, Seattle, Washington.

Correspondence: Richard T. Griffey, MD, MPH, FACEP, Division of Emergency Medicine, Associate Professor, Washington University School of Medicine, Washington University Institute for Public Health, Campus Box 8072, 660 S Euclid Ave, Barnes-Jewish Hospital, St. Louis, MO 63117 (e-mail:

The authors disclose no conflict of interest.

Dr Capp is supported in part by an award from the NIH (KL2), TR001080.

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved