Secondary Logo

Journal Logo

Trends in the Prevalence of Intraoperative Adverse Events at Two Academic Hospitals After Implementation of a Mandatory Reporting System

Wanderer, Jonathan P. MD, MPhil*; Gratch, David M. DO; Jacques, Paul St. MD*; Rodriquez, Luis I. MD; Epstein, Richard H. MD, CPHIMS

doi: 10.1213/ANE.0000000000002447
Patient Safety: Original Clinical Research Report
Free
SDC

BACKGROUND: Anesthesia information management systems (AIMSs) have been effectively used to improve quality in anesthesia care, and have enabled the development of mandatory quality assurance (QA) reporting systems for adverse events (AEs). While this approach has been shown to increase event reporting over time, the long-term effect of such a system on quality is unknown. We investigated the trends in AE reporting over time after implementing AIMS-based mandatory reporting systems at 2 academic medical centers.

METHODS: At Thomas Jefferson University Hospital, AEs were retrieved after implementation of a mandatory QA process in 2013. These AEs were categorized as preventable and unpreventable. The rates of overall preventable and unpreventable AEs were analyzed over time. At Vanderbilt University Medical Center, the rates of AEs were analyzed after establishing a mandatory QA process in 2002. Data were binned by quarter, and trends over time were analyzed using the Mann-Kendall test.

RESULTS: At Thomas Jefferson University Hospital, over a period of 2 years after implementation of a mandatory QA process, the documented AE rate decreased from 1.23% to 0.64% (P < .0001). This decrease occurred primarily in the group of preventable AEs, which declined from 1.01% to 0.52% (P = .014). The rate of unpreventable AEs was unchanged, from 0.22% to 0.12% (P = .12). At Vanderbilt University Medical Center, over a 7-year period after implementation, the AE rate decreased from 4.20% to 1.36% (P < .0001).

CONCLUSIONS: After implementation of a mandatory QA process at 2 academic medical centers, documented AE rates decreased significantly. The decrease observed in preventable AEs, with unchanged rates of unpreventable AEs, demonstrates that mandatory reporting of intraoperative AEs was followed by a reduction in preventable intraoperative AEs. If our findings are replicated at other institutions and are shown to have a stronger trend compared with institutions where the implementation was not conducted, or there was a change from preimplementation trends, adding mandatory perioperative outcomes reporting in the AIMS may represent a valuable method to improve the overall safety of anesthesia.

Published ahead of print August 30, 2017.

From the *Vanderbilt University Medical Center, Nashville, Tennessee

Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania

University of Miami, Miller School of Medicine, Miami, Florida.

Published ahead of print August 30, 2017.

Accepted for publication July 27, 2017.

Funding: Departmental.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

Listen to this Article of the Month podcast and more from OpenAnesthesia.org® by visiting http://journals.lww.com/anesthesia-analgesia/pages/default.aspx.

This study was presented in part as an abstract at the 2016 Annual Meeting of the Society for Technology in Anesthesia, January 6-9, Palm Beach, Florida.

Institutional review board contact information: Approved September 3, 2015, Jacquie Wright, Office of Human Research, Thomas Jefferson University. Institutional review board exemption granted, Human Research Protection Program, Vanderbilt University.

Reprints will not be available from the authors.

Address correspondence to Jonathan P. Wanderer, MD, MPhil, Vanderbilt University Medical Center, 1301 Medical Center Dr, the Vanderbilt Clinic, Suite 4648, Nashville, TN 37232-5614. Address e-mail to jonathan.p.wanderer@vanderbilt.edu.

The safety of anesthesia has improved considerably over the history of the specialty. After the implementation of many technological and systems innovations, the adverse event (AE) rate for major anesthetic complications is currently only 0.92 per 1000 surgical patient discharges.1 A significant innovation in anesthesia that has reached nearly full implementation in academic medical centers is the anesthesia information management system (AIMS).2 These systems have become platforms that enable advanced clinical decision support. Automated prompts for antibiotic administration, for instance, have been associated with reduced incidence of surgical site infection.3 AIMSs have also facilitated the development of quality assurance (QA) reporting systems.4 An area of ongoing interest is reducing the incidence of perioperative complications, yet data related to intraoperative complications are limited.

Paper-based QA systems, by their nature, are difficult to track, and they present challenges with ensuring compliance with data entry. While education on QA documentation, workflow integration, and performance feedback has been demonstrated to increase utilization of QA systems progressively from 48%, to 68% and 78%, respectively,5 mandatory QA data capture can increase reporting to essentially all cases if functionality for a “hard stop”a in documentation is supported. This, in turn, has the potential to increase the number of QA events captured by reducing the rate of unreported events.

This study was originally designed as a QA initiative at Thomas Jefferson University Hospital (TJUH) to identify possible AEs amenable to potential reduction in their prevalence through education and possibly other interventions. It was anticipated that with mandatory reporting, the rate of reported complications would increase, but the opposite result was found. Colleagues at Vanderbilt University Medical (VUMC) were then contacted to look at the time series of reported complication rates within their AIMS to see whether they had a similar result and to understand the long-term trends in AE reporting after implementation of a mandatory intraoperative QA reporting system.

Back to Top | Article Outline

METHODS

Institutional Review Board Approval

This study was approved by the institutional review boards of both Vanderbilt University and Thomas Jefferson University without a requirement for written patient consent.

Back to Top | Article Outline

AE Recording at TJUH

A departmental policy was established in June 2013, requiring that the presence or absence of all intraoperative AEs occurring must be entered in the “outcomes” section when documenting anesthesia care in the AIMS (Innovian, Dräger, Telford, PA). However, a hard stop was not implemented if the provider failed to complete this documentation. The AIMS has been in continuous use since October 2005, and is used to document more than 98% of the anesthetics. Such recording is part of the departmental QA process, and thus, entries made in this section are not printed as part of the anesthesia record. Before mandatory reporting, the outcomes section was available, but the use was not required. Consequently, entries were infrequent and capture of AEs was sporadic. As part of the policy change, the list of possible AEs was extensively expanded (Supplemental Digital Content 1, Document, http://links.lww.com/AA/B983), and providers were instructed to check all applicable items or the outcome “no intraoperative–anesthesia-related complications were noted” if the anesthetic was without incident. This latter option was also available before the policy change. AEs are organized by those most commonly occurring (in the default “favorites” list) and by category (Table 1); this facilitates rapid entry, typically taking a few seconds. The department discouraged the addition of free-text explanations, unless necessary to describe the event. The department reinforced on multiple occasions that if comments were added, only descriptive information should be included, not opinions, accusations, or self-serving commentary. Providers were instructed that any documentation necessary for patient care was to be entered as a comment in the AIMS. For example, a patient who suffered a chipped tooth during laryngoscopy would have this recorded in the printed anesthesia record as part of the intubation note, and the outcome “dental trauma” noted in the outcomes section. However, a patient who had delayed emergence might only have this indirectly noted in the anesthesia record as a 20-minute interval from end of surgery to extubation, with “delayed emergence due to drug sensitivity” noted in the outcomes section.

Table 1.

Table 1.

Although providers could enter AEs at any point during the anesthesia care, the provider who was present at the end of the case was held responsible for ensuring that at least 1 outcome entry was made, typically, the absence of any AEs. To enhance compliance, if no outcome was entered within 10 minutes after the end of anesthesia care was documented (or 30 minutes after the patient left the operating room, if the end of anesthesia care event was missing), a text message was sent to the alphanumeric pager of the final provider. If an outcome was still missing 5 minutes after the notification, the anesthesiologist responsible for the case when it ended was sent a text message advising of the deficiency. Every morning at 6:00 am, the cases from the previous day that were missing an outcome were obtained, and an e-mail was sent to the last provider with instructions to complete this documentation. A summary report was also sent each day by secure e-mail to the departmental quality officer (D.M.G.) listing all AEs for follow-up and the percentage completion of outcome documentation. Providers consistently forgetting to enter outcomes were contacted individually by the QA officer (D.M.G.) and reminded of their responsibility to complete the QA section.

At both TJUH and VUMC, morbidity and mortality conferences are held. These conferences typically focus on analysis of specific patient-safety events and are not focused on AEs globally. While both institutions have robust patient-safety programs and strong patient-safety cultures, during the study interval, aside from the change in reporting described in this report, there were no other significant changes implemented related to intraoperative QA management.

Back to Top | Article Outline

AE Recording at VUMC

The AIMS used at VUMC, Vanderbilt Perioperative Information Management System (VPIMS), was developed in-house and fully implemented in August 2002. VPIMS requires documenting the presence or absence of an intraoperative AE as a hard stop. As is the case at TJUH, these entries are considered part of the departmental QA process and are not printed as part of the anesthetic record. Also, an extensive list of potential AEs was included (Supplemental Digital Content 2, Document, http://links.lww.com/AA/B984). Providers were instructed to select the appropriate intraoperative AEs, or indicate that “no intraoperative events occurred.” Selecting an intraoperative AE only requires an additional few clicks compared with selecting no event. For each case, an optional comment box was available to describe the event in detail. Entry was mandatory, in that it is not possible to otherwise finalize the anesthetic record and send to billing. Furthermore, it was not possible to document that no intraoperative events occurred until the out-of-room time has been documented. This design prevented the preemptive documentation of no AEs until that determination could be made validly.

All documented intraoperative AEs were made available to our departmental quality officer (P.S.J.) and departmental leadership through an ad hoc reporting tool. Selected events enter the departmental peer review or quality improvement processes for additional review. During the study interval, aside from the change in reporting described in this report, there were no other significant changes related to QA management or safety culture.

Back to Top | Article Outline

Data Collection

At the TJUH main hospital and the TJUH ambulatory surgical center, AEs for all cases between July 1, 2013, and June 30, 2015, were retrieved from the AIMS database. Each AE, associated free-text comment (if added), and other chart documentation (if needed) were examined independently by R.H.E. and D.M.G., reassigned to a different AE (if incorrectly specified by the provider), and grouped into categories (Table 1). Nonanesthesia-related AEs were excluded. AEs were characterized as being likely not preventable or possibly preventable. If the 2 reviewers disagreed, the AEs were reexamined and agreement reached as to the correct classification. After this process, no outcomes remained over which there was a difference in opinion. Data were extracted to determine whether there had been a decrease in patient age, case duration, or American Society of Anesthesiologists’ physical status (ASAPS) to ensure that these changes would not contribute to confounding.

At the VUMC main adult hospital, outcomes for all cases between August 2, 2002, and December 31, 2009, were retrieved from the AIMS. The intraoperative AEs list was substantially revised in a VPIMS release on January 22, 2010; thus, entries after these dates were not analyzed for the purposes of this project. AEs selected by the provider were accepted as entered. As at TJUH main adult hospital, patient age, case duration, and ASAPS were extracted.

Back to Top | Article Outline

Statistical Analysis

The number of AEs, cases performed, and cases with QA reporting during each quarter were determined, with quarterly bins selected to allow for trend analysis at TJUH. The prevalence of AEs was computed as the number of events divided by the number of cases in which a QA entry had been made (overall complication rate) and as the number of cases with at least 1 AE divided by the number of cases with QA reporting (case complication rate). Cases in which no QA report entry was made were excluded from the denominator (ie, a case with no QA entry was not considered as a case with no complication). Decreases in the prevalence of complications were determined using the Mann-Kendall test, which assesses the presence of a significant monotonic trend over time. R v3.2.2 (R Foundation for Statistical Computing, Vienna, Austria) was used, with P < .05 required for statistical significance.

A power analysis was not conducted as we analyzed all available data, and we did not have a priori knowledge of an expected drop in the AE prevalence rate over time.

Back to Top | Article Outline

RESULTS

Thomas Jefferson University Hospital

Table 2.

Table 2.

Figure 1.

Figure 1.

Figure 2.

Figure 2.

Figure 3.

Figure 3.

After the change in departmental policy to require entry of intraoperative AEs in the AIMS, there was a rapid increase in compliance with the intraoperative AE reporting process, sustained at approximately 97% over the first 2 years of the program (Figure 1). There was an overall reduction in the number of AEs in all categories (Figure 2), despite no change in the case volume over time. Airway issues and physical injuries were most commonly reported (Table 2), but the overall prevalence of any individual AE was low. An assessment of the cases that were canceled after entering the operating room revealed that most were due to the AEs that likely were not preventable. A significant linear decrease was noted in the overall AE rate by approximately 50%, from 1.23% of the cases having at least 1 reported AE at the start of the program to 0.64% currently (n = 8 quarters, P= .014, Figure 3). The decrease in the AE rate cannot be explained by progressive underreporting, as the rate of AEs classified as likely not preventable did not change over the reporting interval (n = 8 quarters, P= .12, Figure 3), while the prevalence of likely preventable AEs decreased from 1.01% to 0.52% (n = 8 quarters, P = .014, Figure 3). This trend was not confounded by changes in patient factors. Specifically, there was no significant change in patient age or case duration over the study period, and an increase in ASAPS (P = .001).

Back to Top | Article Outline

Vanderbilt University Medical Center

Figure 4.

Figure 4.

As expected from the hard stop in VPIMS, compliance was 100% throughout the 7 years of data analyzed. Respiratory and cardiovascular events were reported most frequently (Table 2), but like TJUH, the overall prevalence of individual AEs was low. During this period, the AE rate (Figure 4) decreased progressively from 4.20% to 1.36% (n = 30 quarters, P < .0001) and then appeared to have leveled off. Over this period of time, there was no significant change in case duration, and increases in ASAPS (P = .0004) and patient age (P < .0001).

Back to Top | Article Outline

DISCUSSION

The primary unexpected finding from this QA study was that implementing mandatory intraoperative AE reporting in an AIMS reduced the reported prevalence of complications documented in the AIMS at both study hospitals. The reduction in complications continued over many years at VUMC, but ultimately stabilized, suggesting a finite limit to the occurrence in AEs. While cardiovascular and airway AEs were most commonly reported at both institutions, there were no individual categories of AEs that occurred with sufficient frequency to warrant targeted intervention.

Reanalysis of the data from the 2006 study at Jackson Memorial Hospital by Vigoda et al4 suggests strongly that mandatory reporting is necessary to accurately capture the intraoperative complication rate. During the 2-year interval when the institution incrementally implemented a mandatory process of intraoperative AEs reporting (ie, first, education; then workflow integration; and finally individual feedback), the reported complication rate fell progressively from 11.6% to 2.1%, concomitant with an increase in AEs reporting from 30.6% to 79.3%. The initial high complication rates were most likely due to lack of reporting on many patients with no AEs, thereby inflating the apparent prevalence.

The strategy of implementing mandatory reporting differed between TJUH and VUMC. On balance, making the QA entry mandatory (ie, a hard stop) is probably a more practical approach to achieve universal reporting, as otherwise, a fully implemented near real-time decision support system is likely necessary. This is beyond the technical capabilities of most hospitals to develop internally, and decision support is not well supported, currently, even within enterprise-wide electronic health records (eg, Epic and Cerner).

The mechanism through which mandatory AEs reporting could have potentially reduced the intraoperative complication rate cannot be determined from the available data. It is possible that the increasing use of advanced airway devices,6 reflecting a combination of their greater availability and acceptance in routine utilization, may have contributed to the reduction in documented airway AEs. Additionally, we conjecture that because providers must report QA data for every case, this focused attention on the reporting of complications may contribute to a culture of safety with an organization. Essentially, these systems could have resulted in a heightened awareness of factors that might lead to AEs, with subsequent mitigation of their occurrence. In the absence of mandatory reporting, a provider might choose to report or not report minor, self-resolving issues. However, an assertion that no complications occurred, when in fact the provider knew that something had happened, would represent a lapse in professionalism. At both institutions, reporting of intraoperative AEs is encouraged, even if the event is minor or possibly unrelated to the anesthesia care, and the process is completely nonpunitive. We think that not printing the AE as part of the anesthesia record is both appropriate (as this is truly a QA activity) and enhances reporting compliance. Additionally, at both institutions, trainees constitute a significant fraction of providers, and thus, providers are subject to substantial turnover. This mitigates the potential explanatory factor of reporting fatigue in the observed reduction of AEs.

An alternative and possibly additional strategy for collection of AEs is to utilize near real-time data to automatically flag cases for AEs.7 This approach could be used for a subset of events, such as hypoxia, hypotension, hypertension, bradycardia, tachycardia, and arrhythmias, and has been found to be more sensitive for these events compared to self-reporting.8 One difficulty with this approach is device artifact, such as a spuriously low noninvasive blood pressure reading due to a surgeon leaning on the cuff. Signal processing could be used to remove such artifacts as has been described by Sun et al9 for temperatures. A hybrid strategy could involve automated analysis of intraoperative AEs prepopulating a self-reporting form, which would give the anesthesia provider the opportunity to correct for device artifact while still flagging appropriate intraoperative AEs.

There are several limitations of this study. It is possible that AEs were already decreasing before implementation of mandatory reporting; however, due to the low-level of intraoperative AE reporting during the baseline period, this possibility is untestable. Although baseline AE data were not available at VUMC, Figure 4 reveals that the AE rate was relatively constant for the first 11 quarters after the institution of mandatory reporting and only subsequently decreased. Thus, we think it unlikely that our findings can be explained by a preexisting trend for improved safety before the interventions. Second, AEs from only 2 institutions were studied, both of which were large, tertiary care academic medical centers. Thus, our findings may not be generalizable to community hospitals or smaller institutions. An additional limitation is that although we used an AIMS to record AEs electronically, making them easily accessible to analysis, this is still a self-reporting system. Thus, the actual complication rates likely are underestimated. An alternative, potential explanation to our finding of a decrease in reported complications over time is that anesthesia providers had high utilization of these systems when mandatory reporting was instituted and that enthusiasm for event entry declined over time. However, arguing against this and in support of our finding, at TJUH, the quality and safety officer (D.M.G.) reports that he had not been contacted by the hospital quality officers over the study interval about any intraoperative AE about which he was not already aware of from the daily e-mail reports he receives by the TJUH system. Additionally, at VUMC, multiple cycles of residents and student-nurse anesthetists have come through the institution over the period of the decline in intraoperative AEs and would thus not be expected to experience event-entry fatigue. Thus, our mechanism for AE reporting is robust, as it is concordant with hospital-generated reports of serious anesthesia complications. Additionally, a substantial fraction of providers at both institutions are trainees; thus, there is substantial turnover, reducing the effect of any reporting fatigue, although potentially introducing variability in collective understanding of what should be reported. Furthermore, there is no basis to think that only reporting of minor complications would have been affected, since the effort to click an AE or the “no AE” item is the same. A third limitation is that the extensive duration of data collection at VUMC precluded individual event review, resulting in likely miscategorization of some events. However, since the focus of analysis at VUMC was to look at the prevalence of AEs after mandatory reporting, this would not affect the observed reduction in AEs. A fourth limitation of the study is that the complication rates between the 2 study hospitals cannot be directly compared. This is because many of the events captured (ie, on the list of common, reportable events) were different. Additionally, the process we describe only relates to intraoperative complications, not postoperative complications (eg, myocardial infarction, acute kidney injury, and reintubation) or postdischarge complications (eg, wound infection, readmission, and 30-day mortality).

In summary, after implementation of mandatory reporting of intraoperative AEs at 2 academic medical centers, there was a trend for a decrease in the prevalence of reported complications. If our findings are replicated at other institutions with corroboration through independent AE reporting systems, adding mandatory perioperative outcomes reporting in the AIMS may represent a simple method to improve the overall safety of anesthesia.

Back to Top | Article Outline

DISCLOSURES

Name: Jonathan P. Wanderer, MD, MPhil.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Name: David M. Gratch, DO.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Name: Paul St. Jacques, MD.

Contribution: This author helped conduct the study and critically revise the manuscript.

Name: Luis I. Rodriquez, MD.

Contribution: This author helped design the study, conduct the study, and write the manuscript.

Name: Richard H. Epstein, MD, CPHIMS.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

This manuscript was handled by: Richard C. Prielipp, MD.

Back to Top | Article Outline

FOOTNOTES

aIn the context of an AIMS, a “hard stop” refers to a documentation element that must be completed before the record can be closed. In some systems, this may preclude printing the record; in others, it may not allow submitting the case for billing.

Back to Top | Article Outline

REFERENCES

1. Agency for Healthcare Research and Quality. 2005 National Healthcare Quality Report. Complications of Anesthesia. Available at: http://archive.ahrq.gov/qual/nhqr05/safety/T2-8.htm. Accessed March 17, 2016.
2. Stol IS, Ehrenfeld JM, Epstein RH. Technology diffusion of anesthesia information management systems into academic anesthesia departments in the United States. Anesth Analg. 2014;118:644–650.
3. Schwann NM, Bretz KA, Eid S, et al. Point-of-care electronic prompts: an effective means of increasing compliance, demonstrating quality, and improving outcome. Anesth Analg. 2011;113:869–876.
4. Vakharia SB, Rinehart J. Using anesthesia AIMS data in quality management. Int Anesthesiol Clin. 2014;52:42–52.
5. Vigoda MM, Gencorelli F, Lubarsky DA. Changing medical group behaviors: increasing the rate of documentation of quality assurance events using an anesthesia information system. Anesth Analg. 2006;103:390–395.
6. Wanderer JP, Ehrenfeld JM, Sandberg WS, Epstein RH. The changing scope of difficult airway management. Can J Anaesth. 2013;60:1022–1024.
7. Sanborn KV, Castro J, Kuroda M, Thys DM. Detection of intraoperative incidents by electronic scanning of computerized anesthesia records. Comparison with voluntary reporting. Anesthesiology. 1996;85:977–987.
8. Benson M, Junger A, Fuchs C, et al. Using an anesthesia information management system to prove a deficit in voluntary reporting of adverse events in a quality assurance program. J Clin Monit Comput. 2000;16:211–217.
9. Sun Z, Honar H, Sessler DI, et al. Intraoperative core temperature patterns, transfusion requirement, and hospital duration in patients warmed with forced air. Anesthesiology. 2015;122:276–285.

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2017 International Anesthesia Research Society