A total of 69,042 surgical cases were observed in all of the post-POCEPs periods. Compliance rates remained consistently >95% throughout all post-POCEPs observation periods (at 97.9%, 96.4%, and 97.1% for periods B′, C, and D, respectively).
Figure 6 displays the SSI rates of the study periods corresponding to their approximate calendar year: A′ (2006), B′ (2007), C (2008), and D (2009). Analysis of variance and ad hoc analysis showed statistically significant differences in mean SSI rates between period A′ and all other periods (periods B′, C, and D: P = 0.004, P < 0.001, and P < 0.001, respectively) and may reflect the sustainability of the POCEPs' effect on SSI rates. No significant differences in compliance or SSI rates were observed between any consecutive time periods since POCEPs implementation.
Electronic prompts have been reported to increase compliance with a variety of best-practice recommendations including venous thromboembolism prophylaxis,9 asthma care,10 and medication prescribing patterns.11 POCEPs have also been shown to be “useful” for implementation of complex algorithms12 and to alert providers of impending danger or risk that would be difficult to discern by other means or easily neglected because of human oversight.13,14 Despite these encouraging reports, the systematic disregard of prompts by providers15 and alert fatigue16 have also been described.
In the operating room, the introduction of POCEPs has thus far had a consistent beneficial effect on compliance with evidence-based practices. Improvements in rates of adherence to clinical recommendations have been reported for antibiotic administration by 2 centers using POCEPs.17,18 Consistent with these reports, our data confirm the effectiveness of electronic prompts to increase rates of appropriate perioperative antibiotic administration, but go an important step further in demonstrating an association between increased antibiotic administration compliance rates and a decreased incidence of SSI.
Quality Indicators and Outcome
Although the investigators predicted an improvement in compliance rates with POCEPs implementation, the association with lower rates of SSI was less anticipated. Although unexpected, the findings are consistent with published reports demonstrating that antibiotic prophylaxis reduces SSIs in controlled settings.19,20 More importantly, these results suggest that improved adherence to process can measurably affect outcome in an unprotocolized clinical milieu.
Establishing an association between quality indicators and outcome outside the domain of the rigorously controlled environment of clinical trials has proven surprisingly challenging.21–24 A recent retrospective cohort study evaluating 405,720 patients between July 1, 2006 and March 31, 2008 from 398 hospitals in the United States for whom SCIP performance was recorded and submitted for public report on the Hospital Compare web site demonstrated that none of the individual infection prevention SCIP process measures was associated with a lower probability of infection.24 SCIP compliance rates from that published report were self-reported by hospitals and predominantly abstracted from nonelectronic and secondary administrative data sources. Although our sample size is smaller, our compliance data were derived from primary electronic clinical data and suggest that increasing compliance with a single process measure (SCIP 1) was associated with decreased incidence of infection.
Impact of SSI and AIMS
Conservative estimates suggest that the national economic burden of SSI exceeds $3 billion annually, and that SSI is a primary contributor to in-hospital mortality.25 Forty million surgical procedures are performed in the United States annually. From those, approximately 780,000 patients will develop a postoperative surgical wound infection,26 and 20,000 will die as a direct consequence.25 Evidence suggests that the current national surgical infection rate (2.6%) can be reduced by 40% to 60% annually by adherence to recommended SCIP 1 guidelines.27
Vis-à-vis the extant technological sophistication in health care today, poor adherence28,29 with effective low-cost, evidenced-based interventions seems inexplicable. Poor compliance with quality indicators may result from multifarious factors.30 Efforts aimed at changing provider behavior by relying on or promoting an increase in vigilance, safety, or education, have historically had limited success31 in producing the high compliance rates necessary to approximate expected industry standards (>95%).32 Our data suggest that computerized clinical reminders, presented at a point in time during which the process measure can be physically executed, are a highly effective modality in eliciting behavior change in a “real world” clinical setting.
Investment in AIMS can be substantial, ranging from $250,000 to >$2 million, commensurate with scale and complexity of the system.33 Demonstrating a return on investment in tangible dollars directly arising from the implementation of AIMS (via the creation of efficiencies or decreased cost of reporting, etc) has been challenging.34 If, however, economic incentives are realigned to encourage value, then the return on investment in AIMS becomes quantifiable. The data suggest that interactive prompts imbedded in such technologies can consistently affect human behavior and create value by improving quality while reducing health care cost. A calculation based on the presented data suggests that POCEPs implementation may have been responsible for the 26 fewer SSIs during the latter 6-month study period.
Weakness and Bias: Rates of Compliance
Factors other than POCEPs may have been related to the improved rates of compliance with SCIP 1 obtained in period B. Although no other organizational initiatives focused on SSI prevention or antibiotic compliance were conducted during the observation period, a Joint Commission site visit occurred during period A. It is possible that additional factors related, but not directly attributable, to POCEPs may have influenced provider behavior sufficiently to account for the increase in compliance obtained after POCEPs implementation.
Furthermore, the process of POCEPs implementation rather than the POCEPs themselves may have influenced providers. However, the sustainability of the high rates of compliance over the ensuing 3 years does not support this conclusion. Nevertheless, desensitization is a potential phenomenon in human factors, and extended evaluation periods will be necessary to fully evaluate the impact of electronic prompts on behavior and outcome over longer periods of time.
Weakness and Bias: SSI Outcome
Postdischarge surveillance to identify SSIs was limited to the review of positive microbiological culture results. A positive culture result in an outpatient with a history of a surgical procedure initiated further investigation by infection control professionals to identify criteria that matched the CDC/NHSN definition of an SSI. Alternatively, if an outpatient received empirical treatment in the physician's office and a culture was not submitted for microbiological testing, it may have been missed as a potential SSI. The absence of a means to identify SSI postdischarge when cultures were not collected may have resulted in the underreporting of SSIs.
Although observation period B had an increase in overall surgical volume, no new surgical service lines, changes in inpatient/outpatient surgical ratios, or assimilations of geographic populations were introduced during the study period. In addition, no change in preoperative risk factors or patient acuity occurred over the consecutive 12-month observation period in the primary analysis. Nevertheless, significant differences in known risk factors for SSI between the 2 evaluated groups may still exist and account for the decrease in SSI rates.
Furthermore, although a strong association between SCIP 1 compliance and SSI rate has been demonstrated, a cause and effect relationship cannot be unequivocally inferred from the data. It is conceivable that additional factors that have not been accounted for may be related to, or responsible for, the reduction in SSIs obtained in period B.
The assessment of behavior-modulating interactive electronic technology is in its infancy. Considering the relative scarcity and limited collective experience with POCEPs, broader clinical applicability of these “electronic coordinators” of care is unknown. The clinical application of POCEPs in this trial features a rigorously validated quality measure (antibiotic administration) related to outcome. Furthermore, the described POCEPs-coordinated intervention algorithm is very basic and linear. These factors coupled with the relatively homogeneous clinical setting of the operating room present an ideal environment for testing electronically driven algorithms. It is impossible to predict whether POCEPs will be effective when exported to other settings where the complexity of the encounters and clinical pathways increases. The current data also support collateral investigations of POCEPs as a potential tool to benchmark quality and demonstrate value.
From the *Department of Anesthesiology; §Department of Community Health, Health Studies and Education; ∥Infection Control; ¶Lehigh Valley Anesthesia Services; #Department of Information Technology; and **Department of Quality and Patient Safety, Lehigh Valley Health Network, Allentown; †Clinical Anesthesiology, Penn State College of Medicine, Hershey; ‡Allentown Anesthesia Associates, Inc., Allentown, Pennsylvania; and ††Division of Surgical Anesthesiology, University of South Florida College of Medicine, Tampa, Florida.
Name: Nanette M. Schwann, MD.
Contribution: Study concept and design, acquisition of data, analysis and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and statistical analysis.
Attestation: Dr. Schwann had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Karen Bretz, MD.
Contribution: Study concept and design, critical revision of the manuscript for important intellectual content, and statistical analysis.
Attestation: Dr. Bretz had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Sherrine Eid, MPH.
Contribution: Statistical analysis.
Attestation: Ms. Eid had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Terry Burger, RN, BSN, MBA.
Contribution: Acquisition of data and analysis and interpretation of data.
Attestation: Ms. Burger had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Deborah Fry, MT, MBA.
Contribution: Acquisition of data and analysis and interpretation of data.
Attestation: Ms. Fry had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Frederick Ackler, CRNA.
Contribution: Study concept and design.
Attestation: Mr. Ackler had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Paul Evans, CRNA.
Contribution: Study concept and design, acquisition of data, and data collection.
Attestation: Mr. Evans had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: David Romancheck, BS.
Contribution: Acquisition of data and data collection.
Attestation: Mr. Romancheck had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Michelle Beck, BS, MBA.
Contribution: Administrative, technical, or material support.
Attestation: Ms. Beck had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Anthony J. Ardire, MD, MPH.
Contribution: Administrative, technical, or material support.
Attestation: Dr. Ardire had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Harry Lukens, BS, MS.
Contribution: Administrative, technical, or material support.
Attestation: Mr. Lukens had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
Name: Thomas M. McLoughlin, MD.
Contribution: Study concept and design and critical revision of the manuscript for important intellectual content.
Attestation: Dr. McLoughlin had full access to all study data and takes responsibility for the data integrity and the accuracy of the data analysis.
a Identifying Healthcare-Associated Infections (HAI) in NHSN. Available at: http://www.cdc.gov/nhsn/PDFs/pscManual/2PSC_IdentifyingHAIs_NHSNcurrent.pdf. Accessed May 13, 2011.
1. Department of Health and Human Services, Centers for Medicare and Medicaid Services: 42 CFR Parts 422 and 480 (CMS-3239-P), RIN 0938-AQ55. Medicare Program: Hospital Inpatient Value-Based Purchasing Program. Proposed January 2011 as part of Section 3001(a) Patient Protection and Affordable Health Care Act (Pub. L. 111-148) enacted March 23, 2010
2. Bratzler DW, Houck PM, Richards C, Steele L, Dellinger EP, Fry DE, Wright C, Ma A, Carr K, Red L. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical Infection Prevention Project. Arch Surg 2005;140:174–82
3. Dajani AS, Taubert KA, Wilson W, Bolger AF, Bayer A, Ferrieri P, Gewitz MH, Shulman ST, Nouri S, Newburger JW, Hutto C, Pallasch TJ, Gage TW, Levison ME, Peter G, Zuccaro G. Prevention of bacterial endocarditis: recommendations by the American Heart Association. JAMA 1997;277:1794–801
4. Pavel A, Smith RL, Ballard A, Larsen IJ. Prophylactic antibiotics in clean orthopaedic surgery. J Bone Joint Surg Am 1974;56:777–82
5. Fry DE. The Surgical Infection Prevention Project: processes, outcomes, and future impact. Surg Infect 2006;7:s17–26
6. McCahill LE, Ahern JW, Gruppi LA, Limanek J, Dion GA, Sussman JA, McCaffrey CB, Leary DB, Lesage MB, Single RM. Enhancing compliance with Medicare guidelines for surgical infection prevention: experience with a cross-disciplinary quality improvement team. Arch Surg 2007;142:355–61
7. National Nosocomial Infections Surveillance (NNIS) System. Data summary report from January 1992 through June 2004, issued October 2004 [abstract]. Am J Infect Control 2004;32:470–85
8. Edwards JR, Peterson KD, Andrus ML, Dudeck MA, Pollock DA, Horan TC, Peterson KD, Andrus ML, Dudeck MA, Pollock DA, Horan TC; National Healthcare Safety Facilities. National Healthcare Safety Network (NHSN) report, data summary for 2006 through 2007, issued November 2008. Am J Infect Control 2008;36:609–26
9. Fiumara K, Piovella C, Hurwitz S, Piazza G, Niles CM, Fanikos J, Paterno M, Labreche M, Stevens LA, Baroletti S, Goldhaber SZ. Multi-screen electronic alerts to augment venous thromboembolism prophylaxis. Thromb Haemost 2010;103:312–7
10. Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, Stephens TB, Swietlik M, Guevara JP. Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics 2010;125:e770–7
11. Monane M, Matthias DM, Nagle BA, Kelly MA. Improving prescribing patterns for the elderly through an online drug utilization review intervention: a system linking the physician, pharmacist, and computer. JAMA 1998;280:1249–52
12. Colpaert K, Hoste E, Van Hoecke S, Vandijck D, Danneels C, Steurbaut K, De Turck F, Decruyenaere J. Implementation of a real-time electronic alert based on the RIFLE criteria for acute kidney injury in ICU patients. Acta Clin Belg Suppl 2007;2:322–5
13. Mashour GA, Esaki RK, Vandervest JC, Shanks A, Kheterpal S. A novel electronic algorithm for detecting potentially insufficient anesthesia: implications for the prevention of intraoperative awareness. J Clin Monit Comput 2009;23:273–7
14. Eden A, Pizov R, Toderis L, Kantor G, Perel A. The impact of an electronic reminder on the use of alarms after separation from cardiopulmonary bypass. Anesth Analg 2009;108:1203–8
15. Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ, Weingart SN. Overrides of medication alerts in ambulatory care. Arch Intern Med 2009;169:305–11
16. Kucher N, Puck M, Blaser J, Bucklar G, Eschmann E, Lüscher TF. Physician compliance with advanced electronic alerts for preventing venous thromboembolism among hospitalized medical patients. J Thromb Haemost 2009;7:1291–6
17. O'Reilly M, Talsma AN, VanRiper S, Kheterpal S, Burney R. An anesthesia information system designed to provide physician-specific feedback improves timely administration of prophylactic antibiotics. Anesth Analg 2006;103:908–12
18. Wax DB, Beilin Y, Levin M, Chadha N, Krol M, Reich DL. The effect of an interactive visual reminder in an anesthesia information management system on timeliness of prophylactic antibiotic administration. Anesth Analg 2007;104:1462–6
19. Bedouch P, Labarère J, Chirpaz E, Allenet B, Lepape A, Fourny M, Pavese P, Girardet P, Merloz P, Saragaglia D, Calop J, Francois P. Compliance with guidelines on antibiotic prophylaxis in total hip replacement surgery: results of a retrospective study of 416 patients in a teaching hospital. Infect Control Hosp Epidemiol 2004;25:302–7
20. Classen DC, Evans RS, Pestotnik SL, Horn SD, Menlove RL, Burke JP. The timing of prophylactic administration of antibiotics and the risk of surgical-wound infection. N Engl J Med 1992;326:281–6
21. Fonarow GC, Abraham WT, Albert NM, Stough WG, Gheorghiade M, Greenberg BH, O'Connor CM, Pieper K, Sun JL, Yancy C, Young JB; OPTIMIZE-HF Investigators and Hospitals. Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA 2007;297:61–70
22. Horn SD. Performance measures and clinical outcomes. JAMA 2006;296:2731–2
23. Ferrer R, Artigas A, Levy MM, Blanco J, Gonzalez-Diaz G, Garnacho-Montero J, Ibanez J, Palencia E, Quintana M, de la Torre-Prados MV; Edusepsis Study Group. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA 2008;299:2294–303
24. Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu PF, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA 2010;303:2479–85
25. Kirkl KB, Briggs JP, Trivette SL, Wilkinson WE, Sexton DJ. The impact of surgical-site infections in the 1990's: attributable mortality, excess length of hospitalization, and extra costs. Infect Control Hosp Epidemiol 1999;20:725–30
26. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for Prevention of Surgical Site Infection, 1999: Centers for Disease control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control 1999;27:97–132
27. Chang SW. Health information technology as a tool, not an end: comment on “an empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Arch Intern Med 2009;169:1474–5
28. Bratzler DW, Houck PM. Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project. Am J Surg 2005;189:395–404
29. Bratzler DW, Houck PM. Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project. Clin Infect Dis 2004;38:1706–15
30. Turnbull BR, Zoutman DE, Lam M. Evaluation of hospital and patient factors that influence the effective administration of surgical antimicrobial prophylaxis. Infect Control Hosp Epidemiol 2005;26:478–85
31. Dellinger EP, Hausmann SM, Bratzler DW, Johnson RM, Daniel DM, Bunt KM, Baumgardner GA, Sugarman JR. Hospitals collaborate to decrease surgical site infections. Am J Surg 2005;190:9–15
32. Senders JW, Moray N, North Atlantic Treaty Organization. Human Error: Cause, Prediction, and Reduction. Hillsdale, NJ: Lawrence Erlbaum Associates Publishing, 1991:270–1
33. Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR, Ferris TG, Shields A, Rosenbaum S, Blumenthal D. Use of electronic health records in U.S. hospitals. N Engl J Med 2009;360:1628–38
© 2011 International Anesthesia Research Society
34. Parente ST, McCullough JS. Health information technology and patient safety: evidence from panel data. Health Aff (Millwood) 2009;28:357–6