The Centers for Disease Control (CDC) uses surveillance definitions to monitor preventable conditions that affect patient outcomes. To improve clinical outcomes for patients receiving mechanical ventilation (MV), the CDC adopted the ventilator-associated pneumonia (VAP) definition in 1994 (1). Almost 2 decades later, in 2013, the CDC convened an expert panel to address shortcomings of the VAP definition (2). The working group developed the concept of ventilator-associated events (VAEs) to extend the definition beyond infection and identify both infectious and noninfectious nosocomial respiratory deterioration. The VAE surveillance definition provides more specific criteria for detecting events of interest using a three-tier hierarchy. An event in the first tier is a ventilator-associated condition (VAC). The second tier is an infection-related ventilator-associated complication. The third tier is a possible/probable VAP (3–5). The VAC tier includes all events in subsequent tiers, thereby identifying patients at risk of preventable negative clinical outcomes.
One of the CDC panel’s intentions in proposing revisions was to use specific clinical criteria that could be readily extracted from the electronic health record (EHR). Retrospective analyses demonstrated that components of the VAE definition can be effectively implemented in existing EHR databases (6,7). Automated surveillance models have been used to implement VAE surveillance in real time (4,8,9). Surveillance definitions may need to be revised and calibrated to improve capture of patients at risk or to improve implementation of health policy. Low event frequency, small population sizes, and different care settings complicate surveillance definition calibration. The availability of large publicly available databases, representing a heterogeneous patient population in diverse critical care settings and with documented patient outcomes, allows further exploration of definitions and their associations with patient outcomes.
Surveillance definitions aim to detect patients at risk of adverse clinical outcomes and are especially relevant for conditions with relatively low incidence but detrimental repercussions on patient status. This is the case for VAE, which have an incidence of 2.0-7.9/1000 ventilation-days and are associated with increased mortality (10). Most hospitals use considerable human resources to support VAE surveillance, although automated computer surveillance programs can be developed to provide real-time VAE monitoring (9–11).
This study proposes a methodology and offers an interactive data visualization tool to test variations in the VAC component of the VAE surveillance definition, using EHR data from publicly available U.S. databases. In this proof-of-concept study, the impact of changes to the VAC definition is assessed based on how they stratify patient outcomes.
MATERIALS AND METHODS
We extracted clinical data from two different EHR datasets, selected patients receiving MV, and developed a visual tool to analyze different subpopulations and the incidence of VAC across combinations of clinical criteria. The corresponding code for the proposed framework to frequently reevaluate VAC definitions can be found on GitHub at https://github.com/SCCM2020-team7/VAC-paper01. This dashboard will be hosted live for several years after publication at https://vacdefinitionexplorer.duhs.duke.edu/.
eICU Clinical Research Database (CRD) and Medical Information Mart for Intensive Care III (MIMIC-III) provided the data source for these analyses (12–15). We extracted data for all patients receiving MV for at least 4 calendar days (eTable 2,https://links.lww.com/CCX/B84), in accordance with the CDC criteria for VAC:
- 1) Baseline: MV for 2 calendar days, with at least one measurement of positive end-expiratory pressure (PEEP) and fractional inspired oxygen (Fio2) recorded for greater than or equal to 1 hr/d; and
- 2) Instability: MV for 2 calendar days immediately following the baseline period, with at least one measurement of increased Fio2 and/or of PEEP recorded for greater than or equal to 1 hr/d.
Exclusion criteria were:
- 1) age less than 18 years,
- 2) missing Fio2 data, or
- 3) missing PEEP data.
Missing Fio2 and PEEP data were defined as no data within a day period. Only the first MV episode in each ICU stay was included to avoid confounding. Hospital and ICU length of stay (LOS) were calculated for survivors and nonsurvivors separately. In addition to statistical analyses stratified by dataset, patient populations were combined across eICU and MIMIC-III to maximize power. To support pooled analysis, ICU type was harmonized across eICU and MIMIC-III, resulting in the four following categories: medical, surgical, mixed medical-surgical, and neurological ICUs.
Data in the Medical Information Mart for Intensive Care III (MIMIC-III) and the eICU-CRD have been previously deidentified, and the institutional review boards of the Massachusetts Institute of Technology (No. 0403000206) and Beth Israel Deaconess Medical Center (2001-P-001699/14) both approved the use of the databases for research.
The CDC VAC definition requires an initial baseline period of 2 calendar days of MV stability, followed by 2 calendar days of MV instability, defined by worsening oxygenation (eFig. 1,https://links.lww.com/CCX/B84)(3).
CDC-Calendar Daytime Interval.
If a patient receives MV during any part of a calendar day, they are considered to have received MV for the whole “CDC-Calendar Day” time interval. Per the CDC definition, once an MV session crosses the midnight boundary, a new day is counted. This creates a situation where 2 calendar days can range from 25 to 48 hours (eFigs. 2–4,https://links.lww.com/CCX/B84). With similar variability in both stability and instability periods, the total minimum MV duration for CDC-calendar day criteria is between 50 and 96 hours. More details are provided on the CDC website (3).
CDC Oxygenation Definition.
Worsening oxygenation after the baseline stability period is defined as an increase above minimum Fio2 (ΔminFiO2) of greater than or equal to 20% or an increase above minimum PEEP (ΔminPEEP) of greater than or equal to 3 cm H2O—relative to the patient’s baseline oxygenation parameters—maintained for longer than 1 hour (3).
In this study, comparative definitions for worsening oxygenation were explored. Furthermore, an interactive visualization tool was built and made publicly available to explore the influence of varying selection criteria on sample size, VAC incidence rates, and patient outcomes—within and across the EHR databases used. Through the interactive tool, distributions of patient outcomes can be used to evaluate different VAC definition parameters.
Comparative 24-Hour Period Time Interval.
The baseline stability period was established using 24-hour time periods, from intubation onwards. Each subsequent “day” began 24 hours after intubation. All patients spent an equal amount of time in the stability phase (≥48 hr). Time spent in the subsequent 2-day instability period was also identical for all patients (≥48 hr), resulting in a minimum of 96 hours. This alternative definition of time periods is described in eFigures 2–4 (https://links.lww.com/CCX/B84).
Comparative Oxygenation Definition.
Worsening oxygenation after the baseline stability period was defined by different combinations of change in ΔminFiO2 and ΔminPEEP. From baseline values, ΔminFiO2 values were modified using 10% increments (10–70%); similarly, ΔminPEEP values were modified using 1 cm H2O increments (1–7 cm H2O).
The datasets were analyzed with R Core Team, Auckland, New Zealand 3.6.3 and Python 3.6 Centrum Wiskunde & Informatica (CWI), Netherlands. In both the eICU-CRD and MIMIC-III databases, comparisons between the characteristics of patients with and without VAC were conducted using two-sided t tests (for numeric variables) or chi-square tests (for categorical variables). Kaplan-Meier separation used log-rank tests.
An interactive visualization was created to further explore the association of selected criteria (dataset, time interval definition, and ICU type) with patient outcomes. Specifically, this tool features heatmaps as well as Kaplan-Meier curves characterizing inhospital mortality among patients with and without VAC. Furthermore, the effect of varying thresholds for ventilation settings (e.g., ΔminPEEP, ΔminFiO2) can be tested empirically.
Heatmaps showcasing differences in outcomes between VAC and non-VAC patients were generated. Odds ratios were used as evaluation metrics for inhospital mortality, whereas mean time-to-event measures were used for hospital LOS and ICU LOS). To facilitate comparing outcome differences resulting from varying time intervals (calendar day and 24-hr periods), distinct heatmaps were provided for each time interval definition. Within a given heatmap, the relative influence of ΔminPEEP and ΔminFiO2 criteria is rendered through color-coded matrix cells, where color intensity increases with the magnitude of adverse outcomes, that is, VAC mortality odds ratio and the prolongation of the hospital/ICU LOS.
Kaplan-Meier Step Functions
Kaplan-Meier step functions were generated to illustrate differences in 28-day inhospital mortality, measured from the start of MV, between patient subgroups. If the patient survived their hospitalization, the mortality outcome was considered right-censored on hospital discharge. In contrast, if the patient died during their hospitalization, the length of time between the start of MV and death was considered.
VAC incidence was computed as the number of VAC per 1,000 person-days of MV (3).
Across the two ICU databases, 14,517 patients met eligibility criteria for this study (Fig. 1). The eICU-CRD database contains 200,859 patients, and 8,210 (4.0%) met inclusion criteria. The MIMIC-III database contains 61,051 patients, and 6,307 (10.3%) met inclusion criteria. In total, when applying the CDC criteria, 1,366 out of 14,517 patients (9.41%) met the CDC criteria for VAC (eICU-CRD: 600/8,210 [7.3%], MIMIC-III: 766/6,307 [10.5%]).
Clinical Outcomes for Patients With and Without VAC
In both databases, patients meeting the CDC criteria for VAC (ΔminFiO2 ≥ 20%, ΔminPEEP ≥ 3, 4 ventilated calendar days) had higher mortality than their counterparts without VAC (34.4% vs 25.9%; p < 0.001), a longer duration of MV (11.5 ± 7.9 vs 7.7 ± 5.4 d; p < 0.001), and a longer ICU-LOS (15.6 ± 11.3 vs 11.74 ± 9.3 d; p < 0.001). Patient characteristics are further described in Tables 1 and 2.
TABLE 1. -
Patient Characteristics, Stratified by Ventilator-Associated Condition Versus No Ventilator-Associated Condition Status
|Race and ethnicity
|Age (yr, mean ± sd)
||58.96 ± 15.86
||62.57 ± 16.46
||62.23 ± 16.44
|Body mass index (kg/m2, mean ± sd)
||31.94 ± 11.3
||29.82 ± 9.54
||30.01 ± 9.74
|Worst Sequential Organ Failure Assessment (first 24 hr, mean ± sd)
||10.42 ± 4.0
||9.28 ± 3.75
||9.39 ± 3.79
|Mechanical ventilation duration (d, mean ± sd)
||11.51 ± 7.89
||7.73 ± 5.35
||8.09 ± 5.74
|Outcomes by survivorship status
| ICU LOS (d)
||17.47 ± 11.54
||11.8 ± 9.09
||12.26 ± 9.44
||11.92 ± 9.89
||11.56 ± 9.95
||11.61 ± 9.94
| Hospital LOS (d)
||26.85 ± 18.12
||21.05 ± 17.35
||21.52 ± 17.48
||15.99 ± 14.59
||17.16 ± 17.39
||17.0 ± 17.03
LOS = length of stay, VAC = ventilator-associated condition.
The table characterizes patients in the pooled sample, emanating from the combination of the eICU Clinical Research Database and Medical Information Mart for Intensive Care III electronic health record critical care databases. Missingness rates are provided, when applicable. VAC labels are computed from Centers for Disease Control criteria (minimum Fio2 ≥ 20%, minimum positive end-expiratory pressure ≥ 3, calendar day time intervals).
TABLE 2. -
Patient Stratification and Proposed Analysis Paradigm to Evaluate Alternative (Ventilator-Associated Condition) Surveillance Definitions
|Patient Population (Underlying Characteristics)
||Clinical Criteria (Ventilation Parameter Settings)
||Patient Outcomes (Evaluation Metrics)
|Data source (i.e., Medical Information Mart for Intensive Care III or eICU Clinical Research Database)
||Tier 1 of VAC criteria
| Minimum FiO2
| Minimum positive end-expiratory pressure
|Type of ICU (i.e., medical, surgical, and neuro
|| Calendar day vs 24-hr day
| Early vs late VAC (<7 vs ≥7 d)
eICU = electronic ICU, LOS = length of stay, VAC = ventilator-associated condition.
The levels of analysis triggering alternative VAC surveillance definitions (see clinical criteria) across patient subpopulations (based on data source and ICU type) are described. Further, three distinct outcomes were used to compare/contrast the performance of varying definitions.
The interactive tool supports open exploration of variations in VAC criteria (Fig. 2). Heatmaps showcasing variability in mortality odds ratios and time-to-event outcomes (for ICU LOS and hospital LOS) as part of the interactive visualization tool (Figs. 2 and 3) offer perspective on the effects of jointly modifying the VAC criteria. Additionally, a closer look at stratified patient demographics (i.e., both the prevalence of VAC vs no VAC but also differential inhospital mortality rates) clarifies the combined role of the time definition, ΔminFiO2, ΔminPEEP, VAC timing, ICU types, and respiratory Sequential Organ Failure Assessment (SOFA) score in identifying a wide range of patients at risk for adverse clinical outcomes.
These impacts are illustrated using two examples.
- 1) In this database, using the CDC definition of ΔminFiO2 greater than or equal to 20% and ΔminPEEP greater than or equal to 3 presented in Figure 2 yields 1,366 patients (9.7%) with VAC, that is, an incidence of 11.63 VAC/ 1,000 ventilator-days. In contrast with the significant difference in overall mortality (34.4% vs 25.9%; p < 0.001), the Kaplan-Meier curve suggests no statistically significant difference in survival curves between patients with and without VAC by log-rank test (p = 0.06). Patients with VAC were on average younger (59.0 vs 62.6), had higher BMIs (31.94 vs 29.82), had higher SOFA scores (10.42 vs 9.28), and remained mechanically ventilated longer (11.51 vs 7.73 d). Notably, ICU LOS was similar between survivors and nonsurvivors.
- 2) In contrast, when applying an increased threshold of ΔminPEEP greater than or equal to 6 while keeping a ΔminFiO2 greater than or equal to 20%, the Kaplan-Meier curve presents a statistically significant difference in survival between patients with and without VAC (p < 0.005), despite using the same database as in the first example.
When converting from CDC-Calendar Day to 24-hour day, there were 28.1% (10,436 vs 14,517) fewer patients examined and demonstrated a slightly higher mortality rate (25.2% vs 23.2%). Further examples are provided in eFigure 5 (https://links.lww.com/CCX/B84).
In this study, we examined the implications of varying definitions of VAC using two publicly available critical care databases from the United States. To facilitate such an analysis, we developed a dynamic interactive tool to graphically visualize the impacts of changes in patient populations, clinical criteria, and clinical outcomes. Each of these can be adjusted in the interactive tool.
Selecting a patient population allows examination both by dataset (i.e., selecting either both available databases through a pooled analysis or a single database) and by VAC status (VAC vs non-VAC). The heatmaps and initial cohort characterizations (e.g., descriptive statistics) provide context as to how individual subpopulations compare in terms of inhospital mortality when selecting varying combinations of VAC criteria.
The selection of various VAC criteria allows nuanced examination of definition standards for ΔminFiO2, ΔminPEEP, or calendar versus 24-hour day. Different combinations of these parameters affect the number of patients labeled with VAC. The interactive tool provides descriptive statistics of patient cohort composition, showcases differences in the distributions of LOS outcomes in patients with and without VAC, and characterizes the overlap between Kaplan-Meier curves across distinct subpopulations. This analysis framework presents several advantages. First, Kaplan-Meier curves are visual tools that are well-understood by clinical care teams and allow detection of striking differences between patient outcomes as time progresses. Second, their use does not involve the formulation of any parametric assumptions.
The interactive tool can easily be modified to fit other clinical definitions. In this capacity, it would have the potential to examine additional surveillance definitions from the CDC and other regulatory entities (e.g., Food and Drug Administration [FDA]/European Medicines Agency [EMA]). Furthermore, this tool also allows for examination of approaches to VAE detection in specific patient subpopulations, including among trauma and neurocritical care patients who have been identified as high risk for VAC (16–18).
In other data-driven studies, the previous definition of VAP established by the CDC was used to identify a sample of patients with VAC (7,19,20). In these studies, patients with VAP had worse outcomes (LOS and mortality) than their counterparts without VAP (8,11). In recent years, both the original VAC surveillance definitions and modified versions have been implemented in retrospective observational studies and yielded similar results (4,6,16) (see eTable 1, https://links.lww.com/CCX/B84). The question of whether the CDC-National Healthcare Safety Network (NHSN) definitions could be improved was explored by Klompas et al (11), demonstrating different performances when clinical variables were modified.
VAC, as currently defined (events per 1,000 MV days), is a relatively rare event. Although studies published to date come from a variety of clinical settings and countries, in many cases, the low VAC incidence requires a national surveillance effort to achieve a sufficient sample size for analysis. Databases integrate data spanning multiple years and often predate the creation of the VAC definition. This presents distributional shifts in patient characteristics and temporal changes in clinical practice to facilitate statistical reevaluation (4,7,19). In this context, although most large databases have these limitations, the advent of sizable databases, involving more than one center and large patient populations, opens a new avenue to interrogate VAE definitions and identify patients with worse clinical outcomes under multiple clinical and time interval criteria.
Our study has some limitations. Despite evaluating the impact of varying clinical criteria, we only explored the first tier of VAE. As such, the current study lacks granularity on the etiology of the VAE; examining the implications of alternative surveillance definitions in subsequent tiers will be the object of future research. Additionally, because we examined datasets comprising adult patients only, our results are not generalizable to the pediatric population. Further, our statistical analyses of patient outcomes, although already accounting for age, sex, ICU type, and early/late VAC, do not yet address other influential clinical criteria (e.g., patient comorbidities). Future work would thus test the importance of these potential confounders and incorporate them as needed. Finally, future research will build upon the associations derived in this study, effectively linking patient characteristics, clinical criteria, and time-interval criteria, as well as VAC incidence and long-term outcomes, and further leverage causal analysis to estimate the relative contribution of each individual factor toward patient morbidity and mortality.
The main objective of this research study was not to demonstrate the causal effects of VAE on patient outcomes, but to instead provide a principled methodology implemented in two distinct and sizable databases to retrospectively explore the implementation of various surveillance definitions in different critical care patient cohorts. The appeal of our proposed framework, beyond the opportunity to visually and interactively investigate VAE surveillance definitions through a customizable interface, is the potential for replication in other areas of public health and medicine.
A strength of this study derives from the heterogeneity and complementarity of the two EHR datasets analyzed. Although the eICU-CRD database presents temporal homogeneity (2014–2015) but geographic heterogeneity, the MIMIC-III database is notable for geographic homogeneity but temporal heterogeneity. As generalizability across time and geographies are key considerations for the elaboration of desired nationwide surveillance definitions, this tool provides the opportunity to examine both “overall” performance and “dataset-specific” performance. Going forward, we recommend the CDC-NHSN, in their role as a national surveillance organization, and foster the use of open-access and expert-validated databases to unlock a new era of clinical research to develop future definitions (21).
More importantly, surveillance priorities may change in the years to come, requiring the dynamic reassessment of existing definitions to facilitate health policy decision-making, with an eye toward improving clinical outcomes in patient populations with evolving conditions and needs. For example, the performance of existing VAC criteria will almost undoubtedly change from before the COVID pandemic through the COVID pandemic. Additional applications include the near real-time processing of EHR data, enabling the effective comparison of VAC incidence at a more granular resolution (e.g., surgical ICUs at academic institutions located in different geographic areas). Such an endeavor would allow testing for equal performance of surveillance definitions among vulnerable patient populations and signal the need to refine criteria to increase clinical impact.
This study presents a methodology to evaluate variations on surveillance definitions for VAC in large publicly available critical care databases. We present a visual interactive tool that illustrates differences in patient outcomes induced by definition changes. The proposed framework utilizes heatmaps to illustrate the effect of comparative VAC definitions on patient outcomes (mortality, ICU LOS, and hospital LOS). This framework can be used to reevaluate VAC definitions in additional datasets, as well as for emerging conditions (e.g., COVID-19) and evolving patient populations. Further, national and international surveillance agencies can invest in the deployment of user-friendly VAC surveillance tools that allow critical care teams to learn from not only their patients, but other sites as well.
1. Tablan OC, Anderson LJ, Arden NH, et al.: Guideline for prevention of nosocomial pneumonia. Infect Control Hosp Epidemiol 1994; 15:587588–587627
2. Klompas M: Ventilator-associated events: What they are and what they are not. Respir Care 2019; 64:953–961
3. ACH Surveillance for VAE. NHSN | CDC. 2020. Available at: https://www.cdc.gov/nhsn/acute-care-hospital/vae/index.html
. Accessed June 8, 2020
4. Wolffers O, Faltys M, Thomann J, et al.: An automated retrospective VAE-surveillance tool for future quality improvement studies. Sci Rep 2021; 11:22264
5. Ventilator-associated Events (VAE) [Internet]. 2021. Available at: https://www.cdc.gov/nhsn/psc/vae/index.html
. Accessed February 15, 2022
6. He Q, Wang W, Zhu S, et al.: The epidemiology and clinical outcomes of ventilator-associated events among 20,769 mechanically ventilated patients at intensive care units: An observational study. Crit Care 2021; 25:44
7. Kobayashi H, Uchino S, Takinami M, et al.: The impact of ventilator-associated events in critically ill subjects with prolonged mechanical
ventilation. Respir Care 2017; 62:1379–1386
8. Klein Klouwenberg PMC, van Mourik MSM, Ong DSY, et al.; MARS Consortium: Electronic implementation of a novel surveillance paradigm for ventilator-associated events. Feasibility and validation. Am J Respir Crit Care Med 2014; 189:947–955
9. Shenoy ES, Rosenthal ES, Shao Y-P, et al.: Real-time, automated detection of ventilator-associated events: Avoiding missed detections, misclassifications, and false detections due to human error [Internet]. Infect Control Hosp Epidemiol 2018; 39:826–833
10. Magill SS, Li Q, Gross C, et al.: Incidence and characteristics of ventilator-associated events reported to the National Healthcare Safety Network in 2014. Crit Care Med 2016; 44:2154–2162
11. Klompas M, Magill S, Robicsek A, et al.; CDC Prevention Epicenters Program: Objective surveillance definitions for ventilator-associated pneumonia. Crit Care Med 2012; 40:3154–3161
12. Pollard T, Johnson A, Raffa J, et al. (2019). eICU Collaborative Research Database (version 2.0). PhysioNet. Available at: https://physionet.org/content/eicu-crd/2.0/
. Accessed February 15, 2020
13. Pollard TJ, Johnson AEW, Raffa JD, et al.: The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Sci Data 2018; 5:180178
14. Goldberger AL, Amaral LAN, Glass L, et al.: PhysioBank, PhysioToolkit, and PhysioNet [Internet]. Circulation 2000; 101:e215–e220.
15. Johnson AEW, Pollard TJ, Shen L, et al.: MIMIC-III, a freely accessible critical care database. Sci Data 2016; 3:160035
16. Wu VKS, Fong C, Walters AM, et al.: Prevalence, clinical characteristics, and outcomes related to ventilator-associated events in neurocritically ill patients. Neurocrit Care 2020; 33:499–507
17. Younan D, Griffin R, Swain T, et al.: Trauma patients meeting both Centers for Disease Control and Prevention’s definitions for ventilator-associated pneumonia had worse outcomes than those meeting only one. J Surg Res 2017; 216:123–128
18. Meagher AD, Lind M, Senekjian L, et al.: Ventilator-associated events, not ventilator-associated pneumonia, is associated with higher mortality in trauma patients. J Trauma Acute Care Surg 2019; 87:307–314
19. Bouadma L, Sonneville R, Garrouste-Orgeas M, et al.; OUTCOMEREA Study Group: Ventilator-associated events: Prevalence, outcome, and relationship with ventilator-associated pneumonia. Crit Care Med 2015; 43:1798–1806
20. Pouly O, Lecailtel S, Six S, et al.: Accuracy of ventilator-associated events for the diagnosis of ventilator-associated lower respiratory tract infections. Ann Intensive Care 2020; 10:6
21. Kozlov M: NIH issues a seismic mandate: Share data publicly [Internet]. Nature 2022; 602:558–559