Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Do Hospital Rankings Mislead Patients? Variability Among National Rating Systems for Orthopaedic Surgery

Shah, Romil F. MD; Manning, David W. MD; Butler, Bennet A. MD; Bilimoria, Karl Y. MD

Journal of the American Academy of Orthopaedic Surgeons: October 08, 2019 - Volume Publish Ahead of Print - Issue - p
doi: 10.5435/JAAOS-D-19-00165
Research Article: PDF Only

Introduction: A growing number of online hospital rating systems for orthopaedic surgery are found. Although the accuracy and consistency of these systems have been questioned in other fields of medicine, no formal analysis of these systems in orthopaedics has been found.

Methods: Five hospital rating systems (US News, HealthGrades, CareChex, Women's Choice, and Hospital Compare) were examined which designate “high-performing” and “low-performing” hospitals for orthopaedic surgery. Descriptive analysis was conducted for all hospitals defined as high- or low-performing in any of the five rating systems, and assessment for agreement/disagreement between ratings was done. A subsample of hospitals ranked by all systems was then created, and agreement between rating systems was investigated using a Cohen's kappa. Each hospital was included in a multinomial logistic regression model investigating which hospital characteristics increased the odds of being favorably/unfavorably rated by each system.

Results: One thousand six hundred forty hospitals were evaluated by every rating system. Six hundred thirty-eight unique hospitals were identified as high-performing by at least 1 rating system; however, no hospital was ranked as high-performing by all five rating systems. Four hundred fifty-two unique hospitals were identified as low-performing; however, no hospital was ranked as low-performing by all the three rating systems which define low-performing hospitals. Within the study subsample of hospitals evaluated by each system, little agreement between any combination of rating systems (κ < 0.10) regarding top-tier or bottom-tier performance was found. It was more likely for a hospital to be considered high-performing by one system and low-performing by another (10.66%) than for the majority of the five rating systems to consider a hospital high-performing (3.76%).

Conclusion: Little agreement between hospital quality rating systems for orthopaedic surgery is found. Publicly available hospital ratings for performance in orthopaedic surgery offer conflicting results and provide little guidance to patients, providers, or payers when selecting a hospital for orthopaedic surgery.

Level of Evidence: Level 1 economic study

From the Department of Surgery (Dr. Shah, Dr. Bilimoria), Surgical Outcomes and Quality Improvement Center (SOQIC), Feinberg School of Medicine, Northwestern University, and Department of Orthopaedics (Dr. Shah, Dr. Manning, Dr. Butler), Northwestern University, Chicago, IL.

Correspondence to Dr. Bilimoria:

None of the following authors or any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Dr. Shah, Dr. Manning, Dr. Butler, and Dr. Bilimoria.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (

© 2019 by American Academy of Orthopaedic Surgeons
You currently do not have access to this article

To access this article:

Note: If your society membership provides full-access, you may need to login on your society website