Objectives: To develop and evaluate a more robust approach for the detection of adverse events in hospital patients using an enhanced Institute for Healthcare Improvement Global Trigger Tool methodology.
Methods: We refined the Institute for Healthcare Improvement Global Trigger methodology using a 2-stage review approach with initial review by 4 clinicians followed by a second review by 2 physicians and then a consensus process. We used a 2-step process for evaluation, and all 6 reviewers participated in both training and, then, testing phase. We measured agreement between all reviewers for the presence and severity of adverse events during the training and testing phase using percent agreement and kappa.
Results: This study used 15 training records containing 22 adverse events and 50 testing records containing 49 adverse events. Initially, the 6 reviewer's agreement with the training records ranged from 38.5% to 76.9%, with kappa ranges from −0.077 to 0.512. After training, percent agreement with the testing records ranged from 66.7% to 93.9%, with kappa values ranging from 0.164 to 0.703. All levels of improvement in reviewer agreement were statistically significant, at the 0.05 level or better, except one of the nonphysician reviewers, whose improvement (71.7% to 83.5%) was significant at only the 0.06 level.
Conclusions: We have developed a robust trigger methodology for detection of adverse events in hospital patients using a retrospective record review approach with a 2-stage review process. This approach has shown the capability, when used with trained reviewers, to achieve high levels of agreement on the presence and severity of adverse events.
From the *Division of Infectious Diseases, University of Utah School of Medicine, Salt Lake City, Utah; and †the Institute for Healthcare Improvement, Cambridge, Massachusetts.
This study was funded by the Institute for Healthcare Improvement (IHI). Dr. Classen is a faculty member at the IHI and a full-time employee of Computer Sciences Incorporation (CSC), a health care technology consulting company, and has an interest in Theradoc, a medical software company.
Correspondence: David C. Classen, MD, MS, 561 East Northmont Way, Salt Lake City, UT 84103 (e-mail: firstname.lastname@example.org).