Institutional members access full text with Ovid®

Share this article on:

When Policy Gets It Right: Variability in U.S. Hospitals’ Diagnosis of Ventilator-Associated Pneumonia*

Stevens, Jennifer P. MD, MS1,2,3; Kachniarz, Bartlomiej BA3; Wright, Sharon B. MD, MPH2,3,4; Gillis, Jean RN, MPH2; Talmor, Daniel MD3,5; Clardy, Peter MD1,3; Howell, Michael D. MD, MPH6

doi: 10.1097/CCM.0b013e3182a66903
Feature Articles

Objective: The Centers for Disease Control has recently proposed a major change in how ventilator-associated pneumonia is defined. This has profound implications for public reporting, reimbursement, and accountability measures for ICUs. We sought to provide evidence for or against this change by quantifying limitations of the national definition of ventilator-associated pneumonia that was in place until January 2013, particularly with regard to comparisons between, and ranking of, hospitals and ICUs.

Design: A prospective survey of a nationally representative group of 43 hospitals, randomly selected from the American Hospital Association Guide (2009). Subjects classified six standardized vignettes of possible cases of ventilator-associated pneumonia as pneumonia or no pneumonia.

Subjects: Individuals responsible for ventilator-associated pneumonia surveillance at 43 U.S. hospitals.

Interventions: None.

Measurements and Main Results: We measured the proportion of standardized cases classified as ventilator-associated pneumonia. Of 138 hospitals consented, 61 partially completed the survey and 43 fully completed the survey (response rate 44% and 31%, respectively). Agreement among hospitals about classification of cases as ventilator-associated pneumonia/not ventilator-associated pneumonia was nearly random (Fleiss κ 0.13). Some hospitals rated 0% of cases as having pneumonia; others classified 100% as having pneumonia (median, 50%; interquartile range, 33–66%). Although region of the country did not predict case assignment, respondents who described their region as “rural” were more likely to judge a case to be pneumonia than respondents elsewhere (relative risk, 1.25, Kruskal-Wallis chi-square, p = 0.03).

Conclusions: In this nationally representative study of hospitals, assignment of ventilator-associated pneumonia is extremely variable, enough to render comparisons between hospitals worthless, even when standardized cases eliminate variability in clinical data abstraction. The magnitude of this variability highlights the limitations of using poorly performing surveillance definitions as methods of hospital evaluation and comparison, and our study provides very strong support for moving to a more objective definition of ventilator-associated complications.

Supplemental Digital Content is available in the text.

1Department of Medicine, Division of Pulmonary, Critical Care, and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA.

2Center for Healthcare Delivery Science, Beth Israel Deaconess Medical Center, Boston, MA.

3Harvard Medical School, Boston, MA.

4Department of Medicine, Division of Infectious Diseases, Beth Israel Deaconess Medical Center, Boston, MA.

5Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Boston, MA.

6Section of Pulmonary and Critical Care, Department of Medicine, University of Chicago, Chicago, IL.

* See also p. 722.

Dr. Stevens had full access to all of the data, conducted all analyses, and takes responsibility for the integrity of the data and the accuracy of the data analysis, as well as contributed to and revised the manuscript. Mr. Kachniarz recruited subjects for the study and critically revised the manuscript. Dr. Wright, Ms. Gillis, Dr. Talmor, and Dr. Clardy participated in the study design, in the analyses, and critically revised the manuscript. Dr. Howell served as senior author, participating in all phases of the study and contributed to and critically revised the manuscript.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (

Dr. Wright received support for development of educational presentations from Society for Healthcare Epidemiology of America (helped to develop and present two lectures as part of a training course for state epidemiologists). She consulted for Agency for Healthcare Research and Quality (AHRQ) (participated in an expert panel on hospital-acquired infections). Her institution received grant support from AHRQ (received funding for a 3-year R18 grant to study methicillin-resistant Staphylococcus aureus in postpartum women and newborns). The remaining authors have disclosed that they do not have any potential conflicts of interest.

For information regarding this article, E-mail:

© 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins