Share this article on:

Quality Improvement Demands Quality Data

Epstein, Richard H. MD

doi: 10.1213/ANE.0000000000001012
Editorials: Editorial

From the Department of Anesthesiology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.

Accepted for publication August 24, 2015.

Funding: Departmental.

The author declares no conflicts of interest.

Reprints will not be available from the author.

Address correspondence to Richard H. Epstein, MD, Sidney Kimmel Medical College at Thomas Jefferson University, 111 S. 11th St., Suite 8490, Gibbon Bldg., Philadelphia, PA 19107. Address e-mail to richard.epstein@jefferson.edu.

In this issue of Anesthesia & Analgesia, Liau et al.1 provide a detailed description of how the Anesthesia Quality Institute (AQI) enrolls practices, manages the data collection process for the National Anesthesia Clinical Outcomes Registry (NACOR), ensures data security, meets current requirements of the Health Insurance Portability and Accountability Act of 1996,2 provides feedback to individual practices, and makes data available to researchers. The authors explain that the mission of the AQI is “to promote quality improvement in anesthesiology,” with its primary purpose related to “local practice quality improvement.”1 Benchmarking at the national level is provided for selected metrics. Data files are available to investigators in participating practices for specified projects, after local IRB approval and authorization through processes established by the AQI Data Use Committee. The authors indicate that use of the NACOR database for research purposes is a secondary objective, subject to a broad set of well-described limitations and caveats.1 These issues likely account for the fact that only 4 research studies have been published in peer-reviewed journals as of April 2015.3–6

When the AQI was chartered by the American Society of Anesthesiologists (ASA) in 2008, a major strategic decision was made to focus on the collection of data that were “readily available in digital form,” rather than specifying required data elements necessary to achieve specified, defined purposes.1 The latter approach is more typical of other quality improvement medical registries (e.g., the American College of Surgeons National Surgical Quality Improvement Program®).7 The “hands-off”1 AQI methodology has resulted in a low barrier to participation (i.e., >283 practices contributing to approximately 30,000,000 anesthesia records as of August 2015) and minimal effort and cost to participants (e.g., free to ASA members or $1000 per participant, otherwise; data transfer automated once configured; and no need for practices to hire staff to manage the process).1 However, there are important consequences of the AQI approach that currently limit the utility of the data being collected, both for individual practices and for health services and outcomes researchers.

The authors report that 32.5% of practices are reporting outcomes,1 but based on the examples provided in the manuscript and the outcome templates currently on the AQI Web site,8 these are nearly all adverse events occurring during the immediate perioperative period. Collection of postdischarge morbidity and mortality (e.g., at 30 days or longer) cannot be accomplished within the AQI data model, because there is not a systematic process that can identify automatically such events using a de-identified data set. For example, consider a patient who undergoes a procedure at one facility and is discharged without incident on postoperative day 1. The patient sustains a complication and is admitted to a different hospital on postoperative day 3. This generally would not be identified. Even more concerning, if the patient dies on postoperative day 23, this would be missed. Within NACOR, preoperative medical conditions are not being collected systematically, necessary for appropriate propensity matching of morbidity and mortality outcomes. For long-term postoperative outcomes to be studied, individual follow-up of every patient or randomly selected patients would need to be conducted at the local level (e.g., as done using the National Surgical Quality Improvement Program), a process inconsistent with the low-cost, low-effort approach embraced by the AQI and its participating facilities. Such follow-up is not a practical approach for many organizations. For perspective, see the recent article from the Cleveland Clinic describing increased mortality after intraoperative use of etomidate.9

Although long-term postoperative outcomes cannot be assessed using NACOR, it would be a realizable objective to measure immediate perioperative adverse events (i.e., while the patient is still in the hospital). This could be accomplished if the AQI were to require, as a condition of participation, that such events are reported; otherwise, there will be no quality component related to patient care. Many practices likely will need to implement such a process to capture adverse events and can refer to the work of Vigoda et al.10 for guidance.

From the perspective of individual practice feedback, the authors report that only 10% of participating practices are providing detailed data from their anesthesia information management systems; 90% just provide administrative data (e.g., from the practice’s professional fee billing system). This means that many practices do not report operating room (OR) entry and exit times, rather anesthesia billing start and end times. In addition, administrative data sets often lack the physical room where cases were performed. The absence of these data precludes the valid determination of most OR management metrics (e.g., turnover times,11–13 tardiness of first case starts,14–16 case durations,17–19 concurrency,20,21 utilization,22,23 and OR efficiency24–26). By definition, billing data only include performed procedures, not scheduled procedures, because the invoice is based on what was actually done. If scheduled procedures are not provided, this precludes valid comparison and prediction of case durations (e.g., for evaluating the accuracy of case scheduling27–29 and potential reductions in patient fasting30 and arrival times31). This is notwithstanding the fact that when multiple procedures are performed, only the procedure that crosswalks to the ASA code with the largest number of base units may be reported. Furthermore, according to the authors, some practices (15%) are not even reporting the Current Procedural Terminology® surgical codes, just anesthesia Current Procedural Terminology codes, which provide insufficient granularity for OR management purposes. There also is substantial ambiguity (25% of cases as of June 15, 2015)32 for practices providing services in multiple locations as to the type of facility (e.g., freestanding surgical center, hospital, and physician office) where individual cases were performed, preventing comparison of complication rates among various facility types. To correct this, the AQI should strongly encourage, if not require, practices to report for all cases the facility type, OR enter and exit times, scheduled procedures, and a de-identified actual OR location.

Given that the primary objective of the AQI is “local practice quality improvement,” the organization should influence its participants to provide what is needed to accomplish this laudable goal. Because presently there are potentially insurmountable limitations on tracking patient outcomes beyond the immediate postoperative period, it is unreasonable to expect the AQI to be able to provide information on important postoperative outcomes potentially influenced by anesthesia care (e.g., 30-day mortality, persistent cognitive decline, late cancer recurrence, or delayed surgical wound infection). Realistically, the AQI should put this option aside. Perioperative adverse events reasonably could be assessed, but first, a much higher percentage of hospitals need to report adverse events. This might be feasible within a few years. The greatest relatively short-term value to individual practices that are contributing data to NACOR is in the area of improving OR management.33 However, to realize this benefit, a few additional data elements need to be provided for each case (i.e., facility type, OR enter and exit times, scheduled procedure(s), and de-identified OR location) and the NACOR database schema modified accordingly. The addition of these data may enhance the ability of investigators to use NACOR data for research purposes.

Although the infrastructure created by the AQI to collect, protect, and manage massive amounts of data is excellent, it is time to address current lapses in the data that are being provided by participating practices. Quality Improvement demands Quality Data.

Back to Top | Article Outline

DISCLOSURES

Name: Richard H. Epstein, MD.

Contribution: This author wrote the manuscript.

Attestation: Richard H. Epstein approved the final manuscript.

This manuscript was handled by: Franklin Dexter, PhD, MD.

Back to Top | Article Outline

REFERENCES

1. Liau A, Havidich JE, Onega T, Dutton RP. The National Anesthesia Clinical Outcomes Registry. Anesth Analg. 2015;121:1604–10
2. Department of Health and Human Services. 45 CFR Parts 160 and 164. Available at: http://www.gpo.gov/fdsys/pkg/FR-2013-01-25/pdf/2013-01073.pdf. Accessed August 23, 2015
3. Nunnally ME, O’Connor MF, Kordylewski H, Westlake B, Dutton RP. The incidence and risk factors for perioperative cardiac arrest observed in the National Anesthesia Clinical Outcomes Registry. Anesth Analg. 2015;120:364–70
4. Deiner S, Westlake B, Dutton RP. Patterns of surgical care and complications in elderly adults. J Am Geriatr Soc. 2014;62:829–35
5. Shapiro FE, Jani SR, Liu X, Dutton RP, Urman RD. Initial results from the National Anesthesia Clinical Outcomes Registry and overview of office-based anesthesia. Anesthesiol Clin. 2014;32:431–44
6. Fleischut PM, Eskreis-Winkler JM, Gaber-Baylis LK, Giambrone GP, Faggiani SL, Dutton RP, Memtsoudis SG. Variability in anesthetic care for total knee arthroplasty: an analysis from the Anesthesia Quality Institute. Am J Med Qual. 2015;30:172–9
7. Birkmeyer JD, Shahian DM, Dimick JB, Finlayson SR, Flum DR, Ko CY, Hall BL. Blueprint for a new American College of Surgeons: National Surgical Quality Improvement Program. J Am Coll Surg. 2008;207:777–82
8. Anesthesia Quality Institute. Quality Measurement Tools. Available at: https://www.aqihq.org/qualitymeasurementtools.aspx. Accessed August 23, 2015
9. Komatsu R, You J, Mascha EJ, Sessler DI, Kasuya Y, Turan A. Anesthetic induction with etomidate, rather than propofol, is associated with increased 30-day mortality and cardiovascular morbidity after noncardiac surgery. Anesth Analg. 2013;117:1329–37
10. Vigoda MM, Gencorelli F, Lubarsky DA. Changing medical group behaviors: increasing the rate of documentation of quality assurance events using an anesthesia information system. Anesth Analg. 2006;103:390–5
11. Dexter F, Abouleish AE, Epstein RH, Whitten CW, Lubarsky DA. Use of operating room information system data to predict the impact of reducing turnover times on staffing costs. Anesth Analg. 2003;97:1119–26
12. Dexter F, Epstein RH, Marcon E, Ledolter J. Estimating the incidence of prolonged turnover times and delays by time of day. Anesthesiology. 2005;102:1242–8
13. Masursky D, Dexter F, Isaacson SA, Nussmeier NA. Surgeons’ and anesthesiologists’ perceptions of turnover times. Anesth Analg. 2011;112:440–4
14. Dexter F, Epstein RH. Typical savings from each minute reduction in tardy first case of the day starts. Anesth Analg. 2009;108:1262–7
15. Dexter EU, Dexter F, Masursky D, Garver MP, Nussmeier NA. Both bias and lack of knowledge influence organizational focus on first case of the day starts. Anesth Analg. 2009;108:1257–61
16. Wang J, Dexter F, Yang K. A behavioral study of daily mean turnover times and first case of the day start tardiness. Anesth Analg. 2013;116:1333–41
17. Dexter F, Macario A. Decrease in case duration required to complete an additional case during regularly scheduled hours in an operating room suite: a computer simulation study. Anesth Analg. 1999;88:72–6
18. Abouleish AE, Dexter F, Whitten CW, Zavaleta JR, Prough DS. Quantifying net staffing costs due to longer-than-average surgical case durations. Anesthesiology. 2004;100:403–12
19. Dexter F, Epstein RH, Bayman EO, Ledolter J. Estimating surgical case durations and making comparisons among facilities: identifying facilities with lower anesthesia professional fees. Anesth Analg. 2013;116:1103–15
20. Posner KL, Freund PR. Trends in quality of anesthesia care associated with changing staffing patterns, productivity, and concurrency of case supervision in a teaching hospital. Anesthesiology. 1999;91:839–47
21. Epstein RH, Dexter F. Influence of supervision ratios by anesthesiologists on first-case starts and critical portions of anesthetics. Anesthesiology. 2012;116:683–91
22. Abouleish AE, Hensley SL, Zornow MH, Prough DS. Inclusion of turnover time does not influence identification of surgical services that over- and underutilize allocated block time. Anesth Analg. 2003;96:813–8
23. Wachtel RE, Dexter F. Tactical increases in operating room block time for capacity planning should not be based on utilization. Anesth Analg. 2008;106:215–26
24. Strum DP, Vargas LG, May JH. Surgical subspecialty block utilization and capacity planning: a minimal cost analysis model. Anesthesiology. 1999;90:1176–85
25. McIntosh C, Dexter F, Epstein RH. The impact of service-specific staffing, case scheduling, turnovers, and first-case starts on anesthesia group and operating room productivity: a tutorial using data from an Australian hospital. Anesth Analg. 2006;103:1499–516
26. Dexter F, Epstein RHLongnecker D, Newman MF, Brown DL, Zapol WM. The economics of operating room anesthesia practice. In: Anesthesiology. 20122nd ed New York, NY McGraw-Hill:1631–48
27. Stepaniak PS, Heij C, de Vries G. Modeling and prediction of surgical procedure times. Statistica Neerlandica. 2010;64:1–18
28. Dexter F, Traub RD, Lebowitz P. Scheduling a delay between different surgeons’ cases in the same operating room on the same day using upper prediction bounds for case durations. Anesth Analg. 2001;92:943–6
29. Dexter F, Dexter EU, Ledolter J. Influence of procedure classification on process variability and parameter uncertainty of surgical case durations. Anesth Analg. 2010;110:1155–63
30. Smallman B, Dexter F. Optimizing the arrival, waiting, and NPO times of children on the day of pediatric endoscopy procedures. Anesth Analg. 2010;110:879–87
31. Dexter F, Epstein RH, Traub RD, Xiao Y. Making management decisions on the day of surgery based on operating room efficiency and patient waiting times. Anesthesiology. 2004;101:1444–53
32. Anesthesia Quality Institute. Available at: https://www.facebook.com/AnesthesiaQualityInstitute. Accessed August 23, 2015
33. Dexter F, Wachtel RE. Strategies for net cost reductions with the expanded role and expertise of anesthesiologists in the perioperative surgical home. Anesth Analg. 2014;118:1062–71
© 2015 International Anesthesia Research Society