If there are systematic differences in the types of patients captured in registries, then differences in outcomes in centers might be related not to differences in the practice of care, but differences in registry inclusion criteria. We set out to evaluate the effect of variable case ascertainment of dead on arrivals on external benchmarking of risk-adjusted mortality using a form of sensitivity analysis.
We used data from the National Trauma Data Bank to look for indirect evidence of systematic differences in case ascertainment. We evaluated whether there was any relationship between fewer than expected early (≤24 hours) deaths and overall risk-adjusted mortality. Fewer than expected early deaths were estimated through the W statistic and through an adjusted ratio of early to late (E/L) deaths. E/L ratios were assessed due to the potential correlation between performance and absolute number of early deaths as assessed by the W statistic.
We estimate that as many as 47% of all deaths might be missing due to problems with case ascertainment. Centers with unexpectedly few early deaths (W statistic) were consistently high performing centers with a lower than expected overall mortality. More importantly, there was no relationship between the E/L death ratio and overall risk-adjusted mortality.
Variable case ascertainment of dead on arrivals does not affect the ability to assess performance. Given that our approach has several assumptions, it is critically important that external validation of trauma registries be performed. If centers are to be judged through the quality of their data, then it is incumbent to first assure that data quality meets expectations.
From the Division of Trauma and the Department of Surgery (D.G., W.X., B.H., N.A., A.B.N.), St. Michael’s Hospital, University of Toronto, Toronto, Canada; and National Trauma Data Bank (S.G.), American College of Surgeon, Chicago, Illinois.
Submitted for publication October 9, 2008.
Accepted for publication December 23, 2008.
Presented at the 67th Annual Meeting of the American Association for the Surgery of Trauma, September, 24–27, 2008, Maui, Hawaii.
Supported, in part, by funds from a Canada Research Chair Program (to A.B.N.).
Address for reprints: David Gomez, MD, 30 Bond Street, Queen Wing, 3-076, Toronto, Canada M5B 1W8; email: GomezJaramilloD@smh.toronto.on.ca.