Institutional members access full text with Ovid®

Share this article on:

Impact of Cross-level Measurement Noninvariance on Hospital Rankings Based on Patient Experiences With Care in 7 European Countries

Orindi, Benedict O. MSc*,†; Lesaffre, Emmanuel PhD*; Sermeus, Walter PhD; Bruyneel, Luk PhD

doi: 10.1097/MLR.0000000000000580
Online Articles: Applied Methods

Background: Hospital-level findings on patient experiences with care are increasingly reported publicly. A critical aspect left unexamined is the commonality of composite measures of patient experiences across different groups of patients, nursing units, hospitals, and countries. Absence of commonality is termed measurement noninvariance and is hypothesized to have a strong impact on performance assessment.

Aim: The aim of this study is to examine measurement invariance across groups and levels under study (patients, nursing units, hospitals, and countries) and illustrate the degree to which this method of analysis impacts hospital rankings.

Research Design: Data were collected from 11,289 patients in 7 European countries, 186 hospitals, and 824 nursing units. Multilevel factor analytic models were applied to evaluate measurement invariance across the hierarchical levels of the study and across groups at specific levels (self-perceived health at patient level; unit speciality at nursing unit level). Hospital rankings for the final multilevel model were compared with those from a single-level factor model that is unsuspecting of measurement invariance.

Results: Cross-group invariance was shown for levels of self-perceived health and to a large degree also for nursing unit speciality. Patient experience composite measures were, however, not invariant across patient, unit, and hospital levels. Hospital rankings were largely impacted when accounted for this cross-level invariance. The percentage of hospitals with discordant ranks by >10 percentile points varied from 26.7% in Spain to 70% in Poland.

Conclusions: Leaving unexamined possible noninvariance across groups and hierarchical levels may have far reaching consequences for how the public perceives hospitals’ position relative to other hospitals.

Supplemental Digital Content is available in the text.

*Department of Public Health and Primary Care, Leuven Biostatistics and Statistical Bioinformatics Centre, Katholieke Universiteit Leuven, Leuven, Belgium

Biostatistics Unit, ICIPE, Nairobi, Kenya

Department of Public Health and Primary Care, Leuven Institute for Healthcare Policy, Katholieke Universiteit Leuven, Leuven, Belgium

The authors declare no conflicts of interest.

Reprints: Benedict O. Orindi, MSc, Department of Public Health and Primary Care, Leuven Biostatistics and Statistical Bioinformatics Centre, Katholieke Universiteit Leuven, Kapucijnenvoer 35, Leuven 3000, Belgium. E-mail:

Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.