Secondary Logo

Journal Logo

Digital Quality Improvement Approach Reduces the Need for Rescue Antiemetics in High-Risk Patients

A Comparative Effectiveness Study Using Interrupted Time Series and Propensity Score Matching Analysis

Gabel, Eilon, MD; Shin, John, MD; Hofer, Ira, MD; Grogan, Tristan, MS; Ziv, Keren, MD; Hong, Joe, MD; Dhillon, Anahat, MD; Moore, James, MD; Mahajan, Aman, MD, PhD; Cannesson, Maxime, MD, PhD

doi: 10.1213/ANE.0000000000003828
Ambulatory Anesthesiology: Original Clinical Research Report

BACKGROUND: Affecting nearly 30% of all surgical patients, postoperative nausea and vomiting (PONV) can lead to patient dissatisfaction, prolonged recovery times, and unanticipated hospital admissions. There are well-established, evidence-based guidelines for the prevention of PONV; yet physicians inconsistently adhere to them. We hypothesized that an electronic medical record–based clinical decision support (CDS) approach that incorporates a new PONV pathway, education initiative, and personalized feedback reporting system can decrease the incidence of PONV.

METHODS: Two years of data, from February 17, 2015 to February 16, 2016, was acquired from our customized University of California Los Angeles Anesthesiology perioperative data warehouse. We queried the entire subpopulation of surgical cases that received general anesthesia with volatile anesthetics, were ≥12 years of age, and spent time recovering in any of the postanesthesia care units (PACUs). We then defined PONV as the administration of an antiemetic medication during the aforementioned PACU recovery. Our CDS system incorporated additional PONV-specific questions to the preoperative evaluation form, creation of a real-time intraoperative pathway compliance indicator, initiation of preoperative PONV risk alerts, and individualized emailed reports sent weekly to clinical providers. The association between the intervention and PONV was assessed by comparing the slopes from the incidence of PONV pre/postintervention as well as comparing observed incidences in the postintervention period to what we expected if the preintervention slope would have continued using interrupted time series analysis regression models after matching the groups on PONV-specific risk factors.

RESULTS: After executing the PONV risk-balancing algorithm, the final cohort contained 36,796 cases, down from the 40,831 that met inclusion criteria. The incidence of PONV before the intervention was estimated to be 19.1% (95% confidence interval [CI], 17.9%–20.2%) the week before the intervention. Directly after implementation of the CDS, the total incidence decreased to 16.9% (95% CI, 15.2%–18.5%; P = .007). Within the high-risk population, the decrease in the incidence of PONV went from 29.3% (95% CI, 27.6%–31.1%) to 23.5% (95% CI, 20.5%–26.5%; P < .001). There was no significant difference in the PONV incidence slopes over the entire pre/postintervention periods in the high- or low-risk groups, despite an abrupt decline in the PONV incidence for high-risk patients within the first month of the CDS implementation.

CONCLUSIONS: We demonstrate an approach to reduce PONV using individualized emails and anesthesia-specific CDS tools integrated directly into a commercial electronic medical record. We found an associated decrease in the PACU administration of rescue antiemetics for our high-risk patient population.

From the David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California.

Published ahead of print 23 August 2018.

Accepted for publication August 23, 2018.

Funding: None.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

E. Gabel and J. Shin contributed equally and share first authorship.

Reprints will not be available from the authors.

Address correspondence to Eilon Gabel, MD, David Geffen School of Medicine, University of California Los Angeles, 757 Westwood Plaza, Suite 3325, Los Angeles, CA 90095. Address e-mail to

See Editorial, p 847


  • Question: Using a major vendor’s electronic medical record system, can postoperative nausea and vomiting (PONV) be improved using an automated clinical decision support tool?
  • Findings: PONV was significantly decreased in our surgical population who received an inhalational anesthetic and had ≥3 or 4 PONV risk factors as described by Apfel et al.
  • Meaning: We have shown a successful way to implement a clinical decision support tool within an Epic electronic medical record that has reduced PONV in high-risk patients for an entire year after implementation.

Postoperative nausea and vomiting (PONV) is one of the most frequent adverse effects observed in patients undergoing anesthesia for surgical procedures, affecting approximately 30% of all surgical patients, and >70% of patients in high-risk patient groups.1–4 A number of published comprehensive reviews, risk scoring systems, and evidence-based guidelines have been developed for the prevention and management of PONV.5–9 However, evidence-based guidelines in general have poor physician compliance.10,11 Specifically with PONV, previous attempts to change anesthesiologists’ practice habits and improve guideline compliance have included didactic sessions reviewing the most current PONV literature and retrospective quarterly reports of individual and department-wide compliance rates.12–14

Electronic medical records (EMRs) offer alternative possibilities with clinical decision support (CDS) tools that can deliver relevant, patient-specific information to the clinician at an appropriate time during patient care. Classically, CDS systems are categorized as being active (requiring some level of automatic processing at the time of an alert) or passive (simply relying on hard-stops that are preset within the EMR for all patients), then secondarily distinguished by real-time versus non–real-time processing.15 Using unique combinations of these principles, CDS tools have successfully demonstrated benefit for many intraoperative issues such as decreasing the incidence of hypotension, controlling intraoperative glucose levels, and timing of antibiotic prophylaxis.16–28 Many of the prior studies, especially those relating to PONV, involved highly customized and proprietary EMRs that are not widely used.12–14 Some are even specific to a single institution.12–14,16–18,20,25,26,28–31 Because our health system utilizes one of the most popular EMRs in the United States, this provides the opportunity to more easily share our CDS with other hospitals that use the same EMR.32 Furthermore, our PONV CDS implementation was approved as an Epic (EPIC Systems, Verona, WI) Clinical Program, meaning that it meets the necessary requirements for being easily portable to other Epic installations, has an accompanying build instructions for local installation, and can be distributed to customers for free.33–35

We hypothesized that using a CDS system both with real-time intraoperative feedback using EMR integration and, concurrently, with non–real-time checkpoint-triggered elements would successfully reduce the incidence of PONV in patients undergoing general anesthesia. We designed and implemented a comprehensive real-time and non–real-time mixed CDS tool out of EPIC in conjunction with a personalized feedback reporting system. Success was determined by comparing outcomes against a historical control group of risk-matched patients.

Back to Top | Article Outline


This study (University of California Los Angeles [UCLA] institutional review board No. 16-001880) qualified for UCLA institutional review board exception status (“waiver of consent”) by virtue of having no direct patient contact and using a deidentified dataset. All study data were acquired via our previously published Department of Anesthesiology and Perioperative Medicine at UCLA perioperative data warehouse (PDW).36 The PDW is a structured reporting schema that contains all the relevant clinical data entered into the EMR via the use of Clarity, the relational database created by EPIC for data analytics and reporting. While Clarity contains raw clinical data, the PDW was designed to organize, filter, and improve data so that it can be used reliably for creating these types of metrics. Last, the PDW servers interfaced with other health system resources to allow for automated emailed reports and the generation of web-based graphical dashboards. The design, implementation, and evaluation of the study methodology had the following phases.

Back to Top | Article Outline

Preintervention Preparation: Creation of the UCLA PONV Pathway

To create a preoperative risk assessment and intraoperative prophylaxis recommendations, patient-specific risk factors were chosen based on a modification of the original Apfel PONV Simplified Risk Scoring System.37 Apfel et al37 described the use of 4 PONV risk factors: female gender, nonsmoker, history of PONV and/or motion sickness, and anticipated postoperative opioid usage. The risk factor of anticipated postoperative opioids was omitted due to the difficulty in creating a specific definition within the EMR and in predicting opioid usage during the postoperative period. We created a modified Apfel risk score as defined below.

Our modified Apfel risk score included female gender, current nonsmoker, prior history of PONV or motion sickness, high-risk surgery, and a novel factor named “Very High Risk.” The first 3 factors are taken directly from the original Apfel risk factors. The surgery-specific risk factor was created under the title “High-Risk Surgery” that encompassed surgical procedures with very likely increased incidence of PONV: laparoscopic surgery, gynecological surgery including breast, strabismus surgery, and neurosurgical procedures.5 Last, the element VHR patient was created to identify any patient who was verified to have received appropriate antiemetic prophylaxis during prior procedures (as defined by the pathway described in the next paragraph) but still experienced PONV. Verification was completed through patient interview (in rare instances in which a patient was able to recall the specific medications that were given) or review of preexisting anesthetic records, if available. For example, a patient with several known risk factors who received the appropriate type and number of antiemetic agents as per our pathway recommendation but still experienced PONV would be someone who “failed” the therapy as recommended in our pathway and therefore would warrant prophylaxis that is even more extensive. In our definition, VHR is differentiated from a simple “history of PONV” by the actual verification (through patient interview or prior anesthetic records) of nausea despite the previous administration of appropriate prophylactic agents (as defined by the pathway described in the next paragraph). If this was not able to be verified (via a prior anesthetic record or patient recollection of receiving specific antiemetics) but the patient simply recalls a prior incidence of PONV, then this would be considered “history of PONV” rather than “VHR.”

To summarize, our modified Apfel risk score includes female gender, current nonsmoker, prior history of PONV or motion sickness, high-risk surgery, and VHR patient. This modified Apfel risk score is assessed for all patients ≥12 years of age undergoing general anesthesia with an inhalational volatile agent. Twelve years is considered the cutoff for a nonpediatric patient as defined by our institution. Excluding VHR, the other risk factors carried the same weight and were aggregated into a single patient risk score; patients who were considered VHR belonged in their own category irrespective of how many other risk factors were identified. The pathway recommended that patients with 1 risk factor should receive a single antiemetic agent (dexamethasone). Patients with 2 risk factors should receive 2 antiemetic agents (dexamethasone and ondansetron). Patients with 3 risk factors should receive 3 agents (dexamethasone, ondansetron, and scopolamine) in addition to consideration of adequate intravenous (IV) fluid administration (20–30 mL/kg) and avoidance of nitrous oxide.38,39 Patients with 4 risk factors should receive all 3 of these agents plus an additional intraoperative agent of their choosing, which was described in department’s PONV guideline document. The most commonly recommended additional intraoperative agents (based on institutional availability) were haloperidol, diphenhydramine, and metoclopramide. These additional agents were selected based on relative availability from the pharmacy as well as faculty consensus. Last, based on published recommendations for patients at high risk for PONV, the pathways recommended that VHR patients receive all of the above PONV prophylaxis and undergo surgery with propofol-based total IV anesthesia.5,40 A summary table of the department’s PONV pathway can be found in Supplemental Digital Content 1, Figure 1, For the purpose of this study, PONV was defined as the administration of any antiemetic agent in the postanesthesia care unit (PACU).

In July 2015, the final PONV pathway was approved by the oversight committees and distributed by the PONV pathway committee, a small group of volunteer faculty members. The faculty, residents, and Certified Registered Nurse Anesthetists (CRNA) were informed how to access the documents via the departmental intranet and the main points were highlighted in an initial email. For the next 8 months, there were no interventions done to promote the use of the PONV pathway. Starting in February 2016, a 3 staged intervention was implemented to promote the compliance to the PONV pathway; a timeline is shown in Supplemental Digital Content 1, Figure 2,

Back to Top | Article Outline

Intervention Stage Number 1: Education and Reintroduction of the PONV Pathway to the Department

A formal lecture was given to all anesthesia providers during a grand rounds lecture on February 17, 2016. The presentation reviewed the highlights of the PONV pathway, reminded everyone where to access the pathway via the intranet, and explained the major changes planned to boost PONV pathway compliance. That same week, a similar lecture was given specifically to the anesthesia residents and CRNA to further emphasize the pathway and PONV treatment options.

Back to Top | Article Outline

Intervention Stage Number 2: Use of the EMR for CDS

Preoperative Risk Stratification: Best Practice Advisory.

To incorporate any real-time CDS into the EMR, a series of PONV items were integrated into the existing preoperative anesthesia evaluation form. The preoperative evaluation form is typically completed by the anesthesia provider the day before surgery after assignment of the case. In the event that a patient is triaged to a telephone review or clinic visit, the evaluation form can be prepopulated with relevant data. Regardless of when an evaluation was initiated, the practice is to reopen the form on the day of surgery to enter the physical examination findings and enter an anesthetic plan; this allows providers to update any PONV targeted questions.

Before any implementation, the only PONV-specific item on the preoperative evaluation form was “History of PONV/motion sickness.” On February 17, 2016, the following binary checkboxes were added: “female sex,” “nonsmoker,” “high-risk surgery,” and “high-risk patient” (VHR). The gender characteristic was preset by the EMR and was not editable by the providers because it was very unlikely that the EMR had the wrong patient gender. Each of these items, with the exception of the VHR risk factor, was treated as a separate risk factor in the system. To comply with the PONV pathway, providers were advised to give a single antiemetic prophylaxis agent to counteract each checked PONV risk factor. VHR patients were thought to be more resistant and required the use of 4 antiemetic agents in addition to avoiding volatile anesthetics altogether. Providers were informed that the pathway should apply to all patients ≥12 years of age undergoing general anesthesia.

As part of the same deployment, a pop-up alert was set to appear in the center of the EMR user’s screen when ≥3 PONV risk factors were selected or the VHR data element was checked, and the anesthesia provider selected the option of “general anesthesia” in the plan portion of the preoperative evaluation. This functionality in Epic is known as best practice advisories (BPAs). The BPA messages contained a clear warning that the patient was at elevated risk for PONV, recommendations for intraoperative prophylactic medications, suggestions to consider avoiding nitrous oxide or volatile anesthetics, and a link to the department’s PONV guidelines. The BPA was designed to fire when the preevaluation was completed on the day of surgery before the start of a case. In the event that a preevaluation form was not completed before case initiation, then the BPA would still be deployed at the moment that the form was completed; this is a requirement for closing an anesthetic case for billing.

On July 1, 2016, four and a half months after the start of the intervention, an update was applied to have the system automate the “nonsmoking” risk factor to improve the accuracy of the CDS. The preoperative evaluation form checkbox for “nonsmoker” status remained editable by the clinician despite being automated, unlike the gender checkbox, which did not allow for user changes.

Back to Top | Article Outline

Intraoperative Real-Time Checklist.

Concurrent with the preoperative evaluation form changes on February 17, 2016, a real-time intraoperative checklist was reconfigured to give providers feedback as to whether the case complies with the PONV pathway. Two months earlier, an independent group of physician informaticists released the checklist functionality using a set of logic that did not align with the department’s PONV pathway. This was done as a silent upgrade without much integration with the department’s informatics division.

The checklist items can be either red or green depending on whether the preset criteria was satisfied. If a volatile anesthetic was detected by the system, patient age was ≥12 years of age, and the number of antiemetics was less than or equal to the number of risk factors (or <4 for a VHR patient), then the checklist item displayed a red indicator; otherwise the indicator was green. The checklist criteria were automatically reevaluated every minute looking for the addition of the recognized intraoperative PONV prophylactic medications: ondansetron, dexamethasone, haloperidol, diphenhydramine, metoclopramide, and scopolamine.

Back to Top | Article Outline

Intervention Stage Number 3: Automated Personalized Email Reports

On June 16, 2016, weekly emails were automatically sent to providers with personalized PONV statistics via the PDW servers. Before sending any reports, a sample was presented at a monthly faculty meeting to get provider buy-in and format approval. The email contained a summary of the number of cases done, the number of patients that met inclusion criteria for the PONV pathway, the number of patients with documented risk factors, and the number of patients that received treatment for PONV in the recovery room. Each case with PONV from the prior week was listed so that providers could use the EMR to look up the case and evaluate for any possible shortcomings. Cases were attributed to the provider who began the case, regardless of whether an intraoperative handoff of care occurred. The list of PONV patients included every patient regardless of age, anesthesia type, or pathway inclusion. Last, the email contained a 6-month graph of the provider’s PONV incidence superimposed on the incidence for all faculty members combined. Alongside was a second chart showing pathway compliance over the same 6-month period. The emails were automated and sent out every Monday morning. Faculty members understood that their reports were not viewed by the informatics team and that the department leadership was not monitoring pathway compliance for each individual provider. After the successful implementation of the PONV email system to the department’s faculty, identical emails were also sent to all the anesthesia residents and CRNA beginning on July 18, 2016. Accuracy of the data through feedback from the faculty was addressed throughout. Before the first email transmission, an explanation and user guide were emailed to the residents and CRNAs from a member of the informatics team. That same week, a member of the informatics team attended a meeting with resident representatives to answer any questions about formatting and to further explain the intention of each data point.

Back to Top | Article Outline

Statistical Methods

In this study, data from the year leading up to the intervention (February 17, 2015 to February 16, 2016) were denoted as the control cohort, while the data from the year after that intervention (February 17, 2016 to February 16, 2017) were used as the intervention cohort. For each group, we defined PONV as the administration of antiemetic medications in the recovery area postoperatively, independent of whether it was given by an anesthesia provider or a PACU nurse. This definition was adapted from the Anesthesia Quality Institute/National Anesthesia Clinical Outcomes Registry Outcome Data Element Conceptual Definitions which defines PONV as the documentation of an antiemetic medication during the anesthesia recovery period.41 The subjective element was excluded due to inconsistent data entry from recovery room nursing and insufficient manpower to interview all the postoperative patients. Additionally, we divided the patient population into low-risk patients, having 1 or 2 modified Apfel risk factors, and high-risk patients who had 3 or 4 modified Apfel risk factors.

The antiemetics that flagged PONV in the PACU were ondansetron, promethazine, haloperidol, diphenhydramine, and metoclopramide. Inclusion criteria for the PONV pathway were documentation of isoflurane, desflurane, or sevoflurane with an inspiratory or expiratory concentration ≥0.5%, ≥12 years of age, and documented time in any of the postanesthesia recovery units (PACU). Patients with no existing social history documentation of a smoking status or any documented time in any of the PACUs were excluded from the analysis; this includes patients taken to intensive care unit straight from the operating room.

Because PONV risk factors were added to the EMR as part of the pathway, all risk adjustment for analysis purposes was done using 4 PONV criteria suggested by Apfel et al7: female gender, nonsmoking status, actual opioid usage in the postoperative recovery, and history of PONV or motion sickness. These variables were calculated for both the pre- and postintervention periods. Gender was defined using the sex data in the EMR. Smoking status was obtained from nursing documentation in the preoperative suite under social history. Opioid usage was defined by the recorded administration of an opioid drug during the postoperative recovery period, which aligned with methodology of Apfel et al.7 The opioid classification used for generating an appropriate drug list was defined by the Medical Subjects Headings by the US National Library of Medicine.42 Last, a history of PONV and/or motion sickness was already an existing data point on the anesthesia preoperative evaluation before the intervention and thus could be used in the pre- and postintervention periods. Each of the 4 modified Apfel criteria was analyzed as a Boolean flag for all of the selected pre- and postintervention patients.

A propensity score matching algorithm was performed to balance the PONV risk factors between the pre- and postintervention cohorts. The outcome of the propensity model was postperiod, and the predictors were the 4 Apfel risk factors (history of PONV, female sex, nonsmoker, and PACU opioid usage), duration of volatile anesthesia >30 minutes, use of regional anesthesia, high-risk surgery stratified by surgical service (Gynecology, Neurosurgery, Plastic Surgery, Otolaryngology), laparoscopic cases, PACU IV morphine equivalents administered within the first hour, and whether an anesthesia resident or CRNA provider was involved in the case. A full description of each risk factor can be seen in Table 1.

Table 1

Table 1

This mitigated the possibility of known risk factor differences between the pre/postperiods from biasing our results in one direction or another (eg, intervention may look overly successful simply due to more patients with PONV history in the preperiod). The matching was conducted using match function in R V3.1.2 (Vienna, Austria) from the data generated by the risk factor algorithm in the PDW. A successful match was determined by the suggested starting caliper width of 0.25 SDs of the logit of the propensity score as originally proposed by Rosenbaum and Rubin43 and then confirmed by Cochran and Rubin.44 After the matched database was constructed, we assessed the standardized mean differences and observed them all to be <0.10.

With our matched dataset, we estimated the change in the incidence of PONV between the pre- to postintervention periods using an interrupted time series analysis as described by Wagner et al.45 We also calculated “no intervention scenarios” from these models where we assume the intervention did not occur and assess the differences from what actually happened. The models were stratified by PONV risk groups (high or low risk) because the intervention was thought to affect the risk groups differently. The piecewise regression model had PONV as the outcome with intervention period (yes/no) and the time relative to the intervention (in weeks) to quantify the immediate intervention effect as well as compare the slopes of PONV pre/postintervention. All analyses and plots were constructed using SAS V9.4 (Cary, NC) and IBM SPSS V23 (Armonk, NY). P values <.05 were considered statistically significant. Results are presented as predicted incidence (95% confidence interval [CI]) unless otherwise noted.

Additionally, to evaluate the effect of the postintervention automated emails, the same models were run as described above with the extra interventional stages included. We defined day 0 as the time of initiation for the preoperative evaluation, intraoperative red/green module, and the BPA. On day 120, the first emails were sent out to the department faculty members. Last, on day 152, the emailed reports were sent to all the providers in the department including CRNA and residents. Because the last 2 stages were roughly 30 days apart, we decided to combine them into a single event.

Last, we examined adherence of the providers to the pathway in the postintervention phase for the high-risk patients (low-risk patients were excluded because providers were mostly compliant by virtue of these patients having few or no PONV risk factors) using the same type of model as described for PONV incidence. Because the necessary data for the pathway risk factors were unreliably generated in algorithm simulations for the preintervention phase, the analysis was limited to the period after initiation of the PONV CDS. Within the PDW, an algorithm generated a flag for whether or not a patient was compliant with the PONV pathway. This used the newly created PONV pathway questions from the preoperative evaluation and exercised the same logic that was used for the intraoperative red/green checklist. We evaluated the pathway compliance (the administration of an equal number or greater antiemetics to PONV risk factors) continuously from the start of the intervention until the very end and then assessed for any increases in compliance after each of the implementation stages.

Back to Top | Article Outline


Incidence of PONV

Data were analyzed from February 17, 2015 to February 16, 2017; this allowed for 1 year for the preintervention cohort and 1 year for the postintervention cohort. In total, there were 53,748 general anesthetics, of which, 40,831 met inclusion criteria. After executing the risk-balancing algorithm, the final cohorts each contained 18,398 anesthetics and had an equal incidence of each PONV risk factors. The CONSORT diagram, shown in Figure 1, details the excluded anesthetics. Patient demographics can be found in Table 2.

Table 2

Table 2

Figure 1

Figure 1

The overall incidence of PONV estimated from our model decreased from 19.1% (95% CI, 17.9%–20.2%) the week before implementation of the CDS to 16.9% (95% CI, 15.2%–18.5%; P = .007) the week after implementation. When isolating the high-risk population (those with 3 or 4 modified Apfel risk factors), there was a greater estimated decrease in the incidence of PONV from 29.3% (95% CI, 27.6%–31.1%) before CDS implementation to 23.5% (95% CI, 20.5%–26.5%; P < .001) after implementation. After running a model with the interaction term between the intervention period and PONV risk strata (P < .001), we found that the reduction in the cumulative PONV rate was likely driven by a significant decrease in the incidence of PONV for the high-risk patients (5.9%) with no statistically significant decrease in the low-risk patients (0.7%). The estimated difference between the high- and low-risk groups in change from pre- to immediately postimplementation was 5.3% (95% CI, 3.5%–7.2%; P < .001). These results are summarized in Table 3. Additionally, Figure 2 demonstrates the incidence of PONV over time during the study period, separating the high-risk and low-risk patients. The figure also demonstrates an abrupt decline in the PONV incidence within the first month of the CDS implementation for the high-risk patients followed by a flattened slope that resembles the preintervention phase. Cumulatively over the entire year of the intervention, there was no significant slope change compared to the year leading up to the implementation for the low-risk group, with an estimated difference in slopes of 0.0002 (95% CI, 0.0004–0.0008; P = .467), or high-risk patients, with an estimated difference in slopes of −0.0004 (−0.0001 to 0.0006; P = .445). The immediate intervention effect for the high-risk patients was estimated to be a reduction of (5.8%– 95% CI, 2.8%-8.8%; P < .001; Figure 2) with no significant difference in slopes (see Supplemental Digital Content 2, Table 1,

Table 3

Table 3

Figure 2

Figure 2

The effect of the intervention was also assessed in high-risk patients by comparing the incidence of PONV observed after the intervention to the estimated incidence of PONV had the intervention not occurred. We presumed that the preintervention slope would have remained constant in light of the intervention never being initiated. Looking at the cut-points of 13, 26, 39, and 52 weeks postintervention, there were significant deviations in the incidence of PONV between the intervention to the predicted scenarios with no intervention. A table with the results can be found in Supplemental Digital Content 2, Table 2,

Back to Top | Article Outline

Adherence to the PONV Pathway

Pathway compliance was measured at each stage in the postintervention period; preintervention data were difficult to obtain and unreliably generated through surrogates. Low-risk patients were excluded because they typically had meager compliance requirements and had compliance rates typically >90% (10,783/11,355 overall in the postperiod). There was no significant trend in provider compliance with the CDS over the entire course of the intervention, shown in Figure 3; the change was ultimately minimal and not statistically significant (level change P = .292 and comparing slopes pre/post P = .143). Furthermore, there were no significant changes in compliance after the initialization of the email reports from the regression analysis. This aligns with the results shown in Figure 2 where the incidence of PONV declined in a relatively short period after the initiation of the intervention and then remained level until the end of the data collection. Presumably, compliance with the pathway improved dramatically during the intervention rollout, but then shortly after plateaued.

Figure 3

Figure 3

The method for inclusion into our analysis required the use of inhalational anesthetics intraoperatively; however, the departmental PONV pathway recommends that VHR patients receive total IV anesthesia with propofol rather than inhalational anesthetics. Therefore, there were VHR patients who were excluded from the analysis when anesthesia providers correctly followed the PONV pathway and refrained from administering a volatile agent. We retrospectively identified all of these patients. There were 250 patients in total with a 12% incidence of PONV. There is no appropriate preintervention comparison group for these patients because we were not identifying VHR patients before implementation of our pathway. Nonetheless, it is highly unlikely that including these additional patients would have significantly affected our overall results.

Back to Top | Article Outline

Effects of Emailed Reports

We found no statistically significant change in the incidence of PONV after the initiation of the emailed reports. The incidence of PONV in the high-risk patients remained unchanged with a reduction of 0.9% (−5.0% to 3.3%; P = .682) afterward. Similarly, there was minimal improvement in CDS compliance with the addition of the provider-specific emails. Compliance remained stable at 85.5% (95% CI, 83.8%–87.1%) preemail and 84.7% (95% CI, 81.0%–86.9%; P = .623) postemail in the high-risk patients. This lack of effect included the reports for both the attending and resident/CRNA groups combined as a single entity.

Back to Top | Article Outline


In this study, we demonstrate the potential value of a comprehensive, self-sustained system to reduce PONV that was implemented on an institutional level across several hundred providers. The CDS tools were created within a commercial EMR with the adjunct of external reporting. Our PONV reduction pathway was associated with an overall reduction in PONV by 2.2% (95% CI, 0.6%–3.9%; P = .007). High-risk patients with 3 or 4 modified Apfel risk factors were associated with a higher reduction of 5.8% (95% CI, 2.8%–8.8%; P < .001). Ultimately, any significant reduction in PONV from this pathway was observed entirely in high-risk patients, rather than in low-risk patients (who were exempt from extensive PONV prophylaxis recommendations).

Contrasting to previously published PONV CDS systems, our pathway took advantage of several forms of active CDS with the addition of the weekly emailed reports. More specifically, unlike the works by Kooij et al12,14 that relied solely on medication orders from the preoperative clinic or Kappen et al13 that used a complex 7 variable predictive model, our system leveraged the patients’ own anesthesia providers and had transparent CDS triggers that permitted for easy modification of precalculated risk factors.

The most significant contrast in our CDS implementation was the use of an EMR from a vendor with the majority share of the US acute care hospital market.32 The aforementioned CDS systems were developed out of custom made or small-scale anesthesia information management systems, which makes mass dissemination challenging without adoption of an anesthesia information management systems with the same functionalities. Thus, the current challenge with CDS systems is to develop them from a major vendor’s EMR, most of which do not necessarily offer the same degree of customization.46

Epic designed a set of Epic Clinical Programs (ECP) that serves as a resource to highlight achievements from customers that have fashioned Epic EMR features to improve patient care. To become an ECP, programs undergo an approval process where they are evaluated for usefulness/effect and ease of portability to other Epic customers. Our PONV implementation was accepted as an ECP and can now be installed to any Epic site free of charge. Furthermore, all of the custom tools that we built in the Epic 2014 version are now available prebuilt into the Epic 2017 Foundation system.33–35

Within the cohort of patients most at risk for PONV, there was a reduction in the incidence of PONV within the first month of the CDS followed by a period of stability. The later, near-zero, slope was unaffected by the initiation of the automated emailed reports. These results suggest the following possibilities: the PONV incidence had immediately dropped to the lowest feasible level, while the emails only served to prolong the effect, or the existence of a continuous CDS system was sufficient motivation for behavior modification (Hawthorne effect). Entertaining the latter, if providers were reminded to prioritize PONV at their first interaction with the CDS, then this suggests that multiple levels of a CDS system are redundant and possibly counterproductive. Nonetheless, the reports may be associated with the sustained reduction in the incidence of PONV given that we did not see a reversal of the effect up to 1 year.

This study has some limitations. First, the entire system relied on correctly inputting PONV risk factors into the preoperative evaluation form. In the event that a provider omitted data from the medical record, the BPA and checklist features would not function properly and the risk estimations would be incorrect. To overcome this issue, we chose to automate as many of the risk factors as possible. Presumably, the automated data capture PONV risk factors correctly or at least outperformed the providers’ entries. Second, we were limited by functional constraints within the EMR. For example, intraoperative BPAs are not available in our version of EPIC, and the checklist tool is limited to the use of red and green colors (which are difficult to decipher in those that are colorblind).

It may be difficult to compare our results directly with those from other studies because our definition for PONV did not include patient surveys or questionnaires but rather applied an adapted definition from National Anesthesia Clinical Outcomes Registry/Anesthesia Quality Institute.41 In addition, other studies routinely defined PONV to encompass the first 24 hours postoperatively, whereas ours was constrained to the PACU recovery time. Thus, our definition likely had higher specificity at the cost of some sensitivity.

In the era of digital quality improvement, the goal is to perfect the complicated interface between providers and machines.47 In this instance, we observed a decrease in the incidence of PONV for high-risk patients that was sustained for 1 year after the initiation of a CDS system using intraoperative real-time guidance. While it is unknown whether the addition of the weekly emailed PONV reports was significant given the lack of further PONV reduction, we believe every element of the CDS system played a role in sustaining the results far beyond what is typical from education alone.

Back to Top | Article Outline


Name: Eilon Gabel, MD.

Contribution: This author helped with correspondence, edit the manuscript, create the clinical decision support, and implement the study.

Name: John Shin, MD.

Contribution: This author helped edit the manuscript, create the postoperative nausea and vomiting pathway and clinical decision support, and implement the study.

Name: Ira Hofer, MD.

Contribution: This author helped edit the manuscript, create the clinical decision support, and implement the study.

Name: Tristan Grogan, MS.

Contribution: This author helped edit the manuscript and analyze the statistical analysis.

Name: Keren Ziv, MD.

Contribution: This author helped edit the manuscript, create the postoperative nausea and vomiting pathway, and implement the study.

Name: Joe Hong, MD.

Contribution: This author helped edit the manuscript, create the postoperative nausea and vomiting pathway creation, and implement the study.

Name: Anahat Dhillon, MD.

Contribution: This author helped edit the manuscript, create the postoperative nausea and vomiting pathway, and implement the study.

Name: James Moore, MD.

Contribution: This author helped edit the manuscript, create the clinical decision support, and implement the study.

Name: Aman Mahajan, MD, PhD.

Contribution: This author helped edit the manuscript. He acts as a project advisor.

Name: Maxime Cannesson, MD, PhD.

Contribution: This author helped edit the manuscript. He acts as a project advisor.

This manuscript was handled by: Tong J. Gan, MD.

Back to Top | Article Outline


1. Gan TJPostoperative nausea and vomiting–can it be eliminated? JAMA. 2002;287:1233–1236.
2. Frenzel JC, Kee SS, Ensor JE, Riedel BJ, Ruiz JROngoing provision of individual clinician performance data improves practice behavior. Anesth Analg. 2010;111:515–519.
3. Koivuranta M, Läärä EA survey of postoperative nausea and vomiting. Anaesthesia. 1998;53:413–414.
4. Habib AS, Chen YT, Taguchi A, Hu XH, Gan TJPostoperative nausea and vomiting following inpatient surgeries in a teaching hospital: a retrospective database analysis. Curr Med Res Opin. 2006;22:1093–1099.
5. Gan TJ, Diemunsch P, Habib AS, et alSociety for Ambulatory Anesthesia. Consensus guidelines for the management of postoperative nausea and vomiting. Anesth Analg. 2014;118:85–113.
6. Apfel CC, Kranke P, Eberhart LH, Roos A, Roewer NComparison of predictive models for postoperative nausea and vomiting. Br J Anaesth. 2002;88:234–240.
7. Pierre S, Benais H, Pouymayou JApfel’s simplified score may favourably predict the risk of postoperative nausea and vomiting. Can J Anaesth. 2002;49:237–242.
8. van den Bosch JE, Kalkman CJ, Vergouwe Y, et alAssessing the applicability of scoring systems for predicting postoperative nausea and vomiting. Anaesthesia. 2005;60:323–331.
9. Apfel CC, Korttila K, Abdalla M, et alIMPACT Investigators. A factorial trial of six interventions for the prevention of postoperative nausea and vomiting. N Engl J Med. 2004;350:2441–2451.
10. Cabana MD, Rand CS, Powe NRWhy don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–1465.
11. James BCMaking it easy to do it right. N Engl J Med. 2001;345:991–993.
12. Kooij FO, Klok T, Hollmann MW, Kal JEDecision support increases guideline adherence for prescribing postoperative nausea and vomiting prophylaxis. Anesth Analg. 2008;106:893–898.
13. Kappen TH, Vergouwe Y, van Wolfswinkel L, Kalkman CJ, Moons KG, van Klei WAImpact of adding therapeutic recommendations to risk assessments from a prediction model for postoperative nausea and vomiting. Br J Anaesth. 2015;114:252–260.
14. Kooij FO, Vos N, Siebenga P, Klok T, Hollmann MW, Kal JEAutomated reminders decrease postoperative nausea and vomiting incidence in a general surgical population. Br J Anaesth. 2012;108:961–965.
15. Nair BG, Gabel E, Hofer I, Schwid HA, Cannesson MIntraoperative clinical decision support for anesthesia. Anesth Analg. 2016;124:1.
16. Bijker JB, van Klei WA, Vergouwe YIntraoperative hypotension and 1-year mortality after noncardiac surgery. Anesthesiology. 2009;111:1217–1226.
17. Ehrenfeld JM, Epstein RH, Bader S, Kheterpal S, Sandberg WSAutomatic notifications mediated by anesthesia information management systems reduce the frequency of prolonged gaps in blood pressure documentation. Anesth Analg. 2011;113:356–363.
18. Nair BG, Horibe M, Newman SF, Wu WY, Schwid HANear real-time notification of gaps in cuff blood pressure recordings for improved patient monitoring. J Clin Monit Comput 2013;27:265–271.
19. Nair BG, Horibe M, Newman SF, Wu WY, Peterson GN, Schwid HAAnesthesia information management system-based near real-time decision support to manage intraoperative hypotension and hypertension. Anesth Analg. 2014;118:206–214.
20. Panjasawatwong K, Sessler DI, Stapelfeldt WHA randomized trial of a supplemental alarm for critically low systolic blood pressure. Anesth Analg. 2015;121:1500–1507.
21. Nair BG, Grunzweig K, Peterson GNIntraoperative blood glucose management: impact of a real-time decision support system on adherence to institutional protocol. J Clin Monit Comput. 2016;30:301–312.
22. Sathishkumar S, Lai M, Picton PBehavioral modification of intraoperative hyperglycemia management with a novel real-time audiovisual monitor. Anesthesiology. 2015;123:29–37.
23. Zanetti G, Flanagan HL Jr, Cohn LH, Giardina R, Platt RImprovement of intraoperative antibiotic prophylaxis in prolonged cardiac surgery by automated alerts in the operating room. Infect Control Hosp Epidemiol. 2003;24:13–16.
24. St Jacques P, Sanders N, Patel N, Talbot TR, Deshpande JK, Higgins MImproving timely surgical antibiotic prophylaxis redosing administration using computerized record prompts. Surg Infect (Larchmt). 2005;6:215–221.
25. O’Reilly M, Talsma A, VanRiper S, Kheterpal S, Burney RAn anesthesia information system designed to provide physician-specific feedback improves timely administration of prophylactic antibiotics. Anesth Analg. 2006;103:908–912.
26. Wax DB, Beilin Y, Levin M, Chadha N, Krol M, Reich DLThe effect of an interactive visual reminder in an anesthesia information management system on timeliness of prophylactic antibiotic administration. Anesth Analg. 2007;104:1462–1466.
27. Nair BG, Newman SF, Peterson GN, Wu WY, Schwid HAFeedback mechanisms including real-time electronic alerts to achieve near 100% timely prophylactic antibiotic administration in surgical cases. Anesth Analg. 2010;111:1293–1300.
28. Nair BG, Newman SF, Peterson GN, Schwid HAAutomated electronic reminders to improve redosing of antibiotics during surgical cases: comparison of two approaches. Surg Infect (Larchmt). 2011;12:57–63.
29. Nair BG, Horibe M, Newman SF, Wu WY, Peterson GN, Schwid HAAnesthesia information management system-based near real-time decision support to manage intraoperative hypotension and hypertension. Anesth Analg. 2014;118:206–214.
30. Nair BG, Gabel E, Hofer I, Schwid HA, Cannesson MIntraoperative clinical decision support for anesthesia: a narrative review of available systems. Anesth Analg. 2017;124:603–617.
31. Nair BG, Newman SF, Peterson GN, Wu WY, Schwid HAFeedback mechanisms including real-time electronic alerts to achieve near 100% timely prophylactic antibiotic administration in surgical cases. Anesth Analg. 2010;111:1293–1300.
32. Castellano TAcute care EMR Market Share 2015 - KLAS press releases. 2015. Available at: Accessed May 22, 2017.
33. Epic Systems Corporation. Reducing PONV rates with risk scoring. Available at:!68!421!3686101. Accessed January 3, 2018.
34. Epic Systems Corporation. Clinical and Financial Programs. Available at:!68!601!1733006. Accessed January 3, 2018.
35. Epic Systems Corporation. Introduction to Clinical Programs and Financial Programs. Available at:!68!625!3363792. Accessed January 3, 2018.
36. Hofer IS, Gabel E, Pfeffer M, Mahbouba M, Mahajan AA systematic approach to creation of a perioperative data warehouse. Anesth Analg. 2016;122:1880–1884.
37. Apfel CC, Läärä E, Koivuranta M, Greim CA, Roewer NA simplified risk score for predicting postoperative nausea and vomiting: conclusions from cross-validations between two centers. Anesthesiology. 1999;91:693–700.
38. Maharaj CH, Kallam SR, Malik A, Hassett P, Grady D, Laffey JGPreoperative intravenous fluid therapy decreases postoperative nausea and pain in high risk patients. Anesth Analg. 2005;100:675–682.
39. Magner JJ, McCaul C, Carton E, Gardiner J, Buggy DEffect of intraoperative intravenous crystalloid infusion on postoperative nausea and vomiting after gynaecological laparoscopy: comparison of 30 and 10 ml kg(-1). Br J Anaesth. 2004;93:381–385.
40. Hubbs RMMC post-operative nausea/vomiting (PONV) Algorithm for adults w/general anesthesia. 2009. Available at: Accessed May 22, 2017.
41. AQI - Anesthesia Quality Institute. Nacor Definitions and Collection Forms. Available at: Accessed May 22, 2017.
42. US National Library of Medicine. Medical subject headings. 2005. Available at: Accessed May 22, 2017.
43. Rosenbaum PR, Rubin DBThe central role of the propensity score in observational atudies for causal effects. Biometrika. 1983;70:41.
44. Cochran WG, Rubin DBControlling bias in observational studies: a review. Sankhyā Indian J Stat Ser A. 1973;35:417–446.
45. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan DSegmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27:299–309.
46. Simpao AF, Gálvez JA, Cannesson MShould we fear computers or the lack of them? Technology, digital quality improvement, and the care redesign process. Anesthesiology. 2017;126:369–370.
47. Gabel E, Hofer I, Cannesson MAdvancing perioperative medicine and anesthesia practices into the era of digital quality improvement. Anesth Analg 2016;122:1740–1741.

Supplemental Digital Content

Back to Top | Article Outline
© 2019 International Anesthesia Research Society