Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

A Methodology For Studying Organizational Performance

A Multistate Survey of Front-line Providers

Lasater, Karen B. PhD, RN*; Jarrín, Olga F. PhD, RN; Aiken, Linda H. PhD, RN, FAAN, FRCN*; McHugh, Matthew D. PhD, JD, MPH, RN, CRNP, FAAN*; Sloane, Douglas M. PhD*; Smith, Herbert L. PhD

doi: 10.1097/MLR.0000000000001167
Statistical Workshop

Background: Rigorous measurement of organizational performance requires large, unbiased samples to allow inferences to the population. Studies of organizations, including hospitals, often rely on voluntary surveys subject to nonresponse bias. For example, hospital administrators with concerns about performance are more likely to opt-out of surveys about organizational quality and safety, which is problematic for generating inferences.

Objective: The objective of this study was to describe a novel approach to obtaining a representative sample of organizations using individuals nested within organizations, and demonstrate how resurveying nonrespondents can allay concerns about bias from low response rates at the individual-level.

Methods: We review and analyze common ways of surveying hospitals. We describe the approach and results of a double-sampling technique of surveying nurses as informants about hospital quality and performance. Finally, we provide recommendations for sampling and survey methods to increase response rates and evaluate whether and to what extent bias exists.

Results: The survey of nurses yielded data on over 95% of hospitals in the sampling frame. Although the nurse response rate was 26%, comparisons of nurses’ responses in the main survey and those of resurveyed nonrespondents, which yielded nearly a 90% response rate, revealed no statistically significant differences at the nurse-level, suggesting no evidence of nonresponse bias.

Conclusions: Surveying organizations via random sampling of front-line providers can avoid the self-selection issues caused by directly sampling organizations. Response rates are commonly misinterpreted as a measure of representativeness; however, findings from the double-sampling approach show how low response rates merely increase the potential for nonresponse bias but do not confirm it.

*Center for Health Outcomes and Policy Research, School of Nursing, University of Pennsylvania, Philadelphia, PA

Institute for Health, Health Care Policy, and Aging Research, Rutgers, The State University of New Jersey, New Brunswick, NJ

Population Studies Center, University of Pennsylvania, Philadelphia, PA

Funding for this study came from the National Institutes of Health (R01NR014855, L.H.A.; P2CHD044964, H.L.S.; R00-HS022406, O.F.J.) and the Robert Wood Johnson Foundation (71654, L.H.A.). The funding organizations did not participate in the research. The content of this article is solely the responsibility of the authors.

These data were presented on October 26, 2018 in Lyon, France at the 10th French-speaking Colloquium on polls.

The authors declare no conflict of interest.

Reprints: Karen B. Lasater, PhD, RN, Center for Health Outcomes and Policy Research, School of Nursing, University of Pennsylvania, 418 Curie Boulevard, Room 374R, Philadelphia, PA 19104. E-mail:

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.