Objectives: To examine to what extent random variation and variation in case-mix influence hospital rankings on the basis of mortality rates and to determine the suitability of mortality for ranking hospitals in colorectal surgery.
Background: Comparing and ranking postoperative mortality rates between hospitals becomes increasingly popular. Differences in hospital case-mix, and chance variation related to caseload, may influence rankings. The suitability of mortality for rankings remains unclear.
Methods: Data were derived from the Dutch Surgical Colorectal Audit. Hospital rankings based on fixed- and random-effects logistic regression models, unadjusted and adjusted for case-mix were compared with the percentile based on expected ranks (the chance that a hospital performs better than a random hospital). Rankability, measuring which part of variation between hospitals is not due to chance, was calculated.
Results: Some 25,591 patients undergoing colorectal resections in 92 hospitals were evaluated. Postoperative mortality rates ranged between 0% and 8.8%. Adjustment for case-mix with a fixed-effects model caused large changes in rankings. A smaller additional effect on changes in rankings occurred after adjusting with a random-effects model, with lower volume hospitals moving toward the mean. Percentile based on expected ranks ranged between 10% and 85%. Rankability was 38%, meaning that 62% of hospital variation in mortality was due to chance.
Conclusions: Hospital ranks changed after case-mix adjustment and random-effects models, compared with unadjusted analysis. A large proportion of hospital variation in mortality was due to chance. Caution should be warranted when interpreting hospital rankings on the basis of postoperative mortality. Percentiles of expected ranks may help identify hospitals with exceptional performance.
This study examined to what extent random variation and variation in case-mix between hospitals influence rankings on postoperative mortality after colorectal cancer resections in 92 Dutch hospitals and determined the suitability and reliability of mortality rates for hospital rankings.
*Department of Surgery, Leiden University Medical Center, Leiden, The Netherlands
†Faculty of Economics and Business, Department of Econometrics, University of Amsterdam, Amsterdam, The Netherlands
‡Department of Surgery, Netherlands Cancer Institute (NKI)—Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands; and
§Department of Medical Statistics and Bioinformatics, Leiden University Medical Center, Leiden, The Netherlands.
Reprints: Daniel Henneman, MD, Department of Surgery, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden, The Netherlands. E-mail: firstname.lastname@example.org.
This research received no specific funding.
Disclosure: The authors declare no conflicts of interest.
Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journal's Web site (www.annalsofsurgery.com).