Maizlin, Redden, Beierle, Chen, and Russell (2017) report that the use of comparative outcome databases has become more prevalent in healthcare over time and the databases can now more accurately predict risk of surgical complications. The best systems provide risk-adjusted outcome reports that facilitate benchmarking with other institutions, motivate hospitals and teams to address gaps, and ultimately lead to improvements in outcomes. The American College of Surgeons National Surgical Quality Improvement Program–Pediatrics (ACS NSQIP-P) is a very useful program for hospitals to assess their outcomes and identify areas for improvement.
Etzioni et al. (2015) reported little change in outcomes for hospitals incorporating NSQIP-P compared with other facilities. However, we have found that risk-stratified data provide a solid base for quality improvement (QI) efforts. Since our organization joined NSQIP-P in 2010, multiple areas for improvement have been identified through the use of semiannual reports and pilot data. Our objective was to evaluate the impact of local QI efforts on both outcomes and cost. Robinson et al. (2018) describe the use of pediatric surgical homes combined with QI efforts as a robust model for improving operational and clinical effectiveness. Perioperative surgical homes are analogous to medical home models where the care team gets involved in some aspects of care management to ensure optimal outcomes and coordination of care. In both the Robinson et al. article and this article, we outline the skills of nurses to lead successful QI teams.
Our institution is a large academic teaching hospital that performs more than 21,000 pediatric surgical cases each year. We have Level III neonatal intensive care unit and a Level I trauma program. We train medical students, residents, and fellows. There are over 850 residents and fellows in our training programs. We also have a large School of Nursing with Registered Nurse, Nurse Practitioner, and Doctorate in Nursing Practice programs.
A small multidisciplinary team was formed in 2013 within the pediatric surgery program to review the first few semiannual reports. This team, known as the Pediatric Surgical Quality Improvement Group, was expanded to ensure all stakeholders were represented and now consists of more than 30 individuals. Figure 1 displays our team constituents. The team members include seven quality assurance (QA)/QI specialists, six pediatric surgeons, four critical care attending physicians, four infection prevention specialists, four Nurse Practitioners, four anesthesia or Emergency Department physicians, one data specialist, and one parent advisor.
The team used standard plan–do–study–act improvement methods as described by Schriefer and Leonard (2012). Smaller working groups were then developed to review literature and determine potentially better practices that could be implemented as part of each plan–do–study–act cycle, where each cycle lasted about 3 months. It was always our plan to work in subgroups. Each subgroup had a team leader who would report back to the larger work group.
Projects had measurable aims that allowed us to determine if our improvement efforts were resulting in meaningful change. The aims were all to ensure none of our complications was in the NSQIP high outlier range. We were able to achieve no outlier metrics in the 2 years of our project. The institutional review board was approached for review, and our effort was given an expedited review because we were not conducting research but rather using QIs and care bundles that were occurred to all patients.
After a collection of baseline NSQIP data in 2012, a comprehensive review of these data in 2013 suggested key areas for improvement including reduction of surgical site infections (SSIs), unplanned reintubations (RIs), length of stay (LOS), blood transfusions, and preoperative Computerized Tomography (CT) use for patients undergoing appendectomy. Our institution was either an outlier or in the tenth decile based on NSQIP Semi-Annual Reports from 2012 to 2013 in each of these key outcome measures. Active QI projects were developed and implemented beginning in 2014. Data for each metric were reported to the larger workgroup by our NSQIP surgical case reviewer.
Smaller workgroups met monthly to develop and test change ideas and track outcomes. The Pediatric Surgical Quality Improvement Group team also met monthly where each working group was responsible for apprising the team with progress and challenges faced. All team meetings had agendas sent out before the meeting via email, and at the conclusion of each team meeting, notes were created with action items and expected time frame for completion. In addition, a smaller group met monthly to ensure that projects remained on target. The smaller group keeps a project management spreadsheet for all the efforts, and this added in the progress and accountability.
The data collection methods followed the protocols set out by the ACS NSQIP-P program. Baseline data analysis was conducted with the assistance of a statistician to help our team interpret the ACS NSQIP-P reports and determine the areas on which to focus. The statistician was able to help us sort and analyze the data from many different angles. The primary statistical analyses were descriptive in nature. We decided to focus on measures where we were outliers in the NSQIP reports. We used all the standard NSQIP definitions for our measures.
Our initiatives included the development of pediatric orthopedic and pediatric general surgery SSI prevention bundles, reduction of unplanned RIs in neonatal intensive care unit patients undergoing abdominal surgery, reduction of LOS in orthopedic surgical patients, blood transfusion protocols to reduce unnecessary/excessive transfusions, and an appendectomy pathway and scoring tool to reduce CT scans. To assess the potential impact of our process changes, key measures were compared between two cohorts of pediatric general and pediatric orthopedic patients during two discrete phases: the baseline phase and the QI implementation phase. Our team meetings occurred once a month for 90 minutes. If we had obstacles to the implementation of bundle elements, we would discuss those obstacles with key stakeholders. For instance, use of the preoperative skin cleansing was not being documented in all cases so we added a field to the nursing flow sheet.
The pediatric orthopedic and pediatric general surgery SSI prevention bundles were developed and implemented in 2015. The bundles consisted of evidence-based practices shown to reduce SSIs: began identifying the specific patient populations at risk with our surgery patients, review of the literature on evidence-based practices for the reduction of SSIs, and issues specific to the at-risk populations. Schriefer et al. (2017) outline the SSI project and summarize the different aspects of each SSI prevention bundle element. This article provides specific information about our specific SSI prevention project.
Sparling et al. (2007) outline the financial impact of SSIs and the need for prevention bundles. Our bundle focused on creating preoperative order sets, warming the operating rooms before patient entry, and maintaining intraoperative normothermia as well as on day of surgery. Chlorhexidine gluconate baths for all surgical cases were some of the key elements included in the bundle. The addition of chlorhexidine gluconate baths the night before surgery and povidone-iodine nasal antiseptic swabs after induction were utilized for patients at the highest risk of developing an SSI.
As outlined by Roddy, Spaeder, Pastor, Stockwell, and Klugman (2015), efforts to reduce unplanned RIs in the pediatric intensive care unit included standardizing postoperative ventilation management to reduce hypoxemic/hypercarbic episodes that mimic endotracheal tube (ETT) obstruction, which historically resulted in ETT removal and replacement. Strategies to reduce unplanned ETT dislodgement were also implemented as part of our RI prevention initiative.
New blood transfusion guidelines were developed with representatives from pediatric hematology, pediatric surgery, blood bank, and intensive care providers that recommend 5–10 ml/kg for a transfusion. We used Shander, Hofmann, Gombotz, Theusinger, and Spahn (2007) article to share the evidence that these changes are associated with better outcomes and lower costs. After review of the data and identification of the need for improvement, blood transfusion guidelines were approved, circulated, and posted on key Web sites and education was provided to staff including nursing, residents, surgeons, and anesthesia providers.
Through a review of the NSQIP Appendectomy Pilot data, our organization was identified as a high utilizer of preoperative CTs in patients undergoing appendectomy. A multidisciplinary group with key stakeholders in the project met and developed an appendectomy pathway. Through a retrospective review of appendectomy cases, an ultrasound scoring tool was also developed and used in conjunction with the Pediatric Appendicitis Score to identify patients at risk for appendicitis. The team used interrater reliability testing of the score between surgeons, radiology, and emergency medicine providers. The components of the score included physical symptoms as well as ultrasound findings. Education was provided to Emergency Department providers and staff as well as pediatric surgeons and surgical residents before initiation of the scoring system.
As reported in McGuire et al. (2017), a combined score of 7 or greater had sensitivity = 0.972, specificity = 0.882, positive predictive value (PPV) = 0.946, and negative predictive value (NPV) = 0.938 for appendicitis. A total score of 13 or greater had sensitivity = 0.306, specificity = 1, PPV = 1, and NPV = 0.405. An ultrasound score of 2 or greater had sensitivity = 0.778, specificity = 0.647, PPV = 0.823, and NPV = 0.579. Analysis of CT utilization revealed a 3% decrease in the number of CT scans ordered since the implementation of the protocol. The decrease was not statically significant, p = .635
Cost estimates for each initiative were derived by utilizing published reports on the cost of various complications in the pediatric surgical patient population (see Table 1). The LOS costs were calculated using the published cost per day for general pediatric units in New York State. Once these estimates were obtained, the cost of each key measure was multiplied by the difference in the number of occurrences between the baseline and QI implementation phases to calculate an estimated cost savings for each measure. The estimates for each measure were then summed to provide a total estimated cost savings after implementation of QI efforts.
Key measures for each cohort are shown in Table 1. Compared with the baseline phase cohort, the number of patients in the QI implementation cohort experienced fewer SSIs (see Figure 2), which depicts the implementation of the high-risk-patient SSI prevention bundle in Q3 2014 and the implementation of the low-risk-patient SSI prevention bundle in Q2 2015. The left-hand side of figure to y axis is the number of SSIs per quarter, and the right-hand y axis is the rate of SSI per quarter.
The use of CT scans for patients with appendicitis (see Figure 3) and unplanned RI trends were also tracked quarterly (see Figure 4). In Figure 4, the “Interventions in place” are when the interventions to reduce RIs were put into place. In addition, we also calculated the median and average annual LOS for pediatric orthopedic patients in the baseline (n = 343) and QI implementation (n = 374) phases. Compared with baseline, the median LOS observed in the implementation phase was reduced by 1.0 day. The average annual LOS was reduced by 214 patient days when compared with the baseline (see Figure 5). We believe the LOS was reduced because of improvements such as better preoperative pulmonary and nutritional optimization with our perioperative surgical home effort as well as reduced SSIs that can prolong the LOS.
We also found experienced reduction in blood transfusions (1135 vs. 1115 ml as seen in Figure 6) and unplanned readmissions (36 vs. 23). All reductions were clinically important, and two reached statistical significance including CT use and RIs using chi-square test of change. On the basis of evidence-based estimates, the potential cost savings from our improvements is also shown in Table 1. Cumulatively, we estimate the savings in the QI implementation cohort (2015–2016) to be $1,655,194.
The limitation of this project was that we used a before-and-after method. We did determine that the average patient Anesthesia Severity Assessment score was similar between the baseline versus the QI implementation phase. Although not all differences in key outcome measures were statistically significant, we believe all represented clinically significant improvements. Cost estimates were used from published studies and thus may not directly reflect costs in our healthcare system. Although this study does not compare our results with those of other institutions who may have improved without using a data-driven approach, our change in percentiles seen in Table 1 supports its utility in driving changes for patient safety.
The most important finding of this project was that providing a multidisciplinary team with routine comparative and risk-adjusted reports can drive QI that leads to better outcomes. The team utilized monthly meetings with agendas and minutes and also celebrated successes along the way. We have been able to target high-yield areas to make the best use of our limited human and financial resources. Team members are consistent in attending all meetings and are dedicated to making progress each month toward improving processes and outcomes for pediatric surgical care.
Although some improvements have occurred, we have found sustaining improvement (“holding the gains”) has its own challenges, particularly in an academic institution with rotating residents and expected staff turnover. One of our primary lessons learned is that the plans are needed to refresh educational programs annually. We recognized the importance of continuing education as a critical component in sustaining changes and in creating new ideas for further improvement. One example was the development of a video that provided education on the appendectomy pathway and scoring tool to reduce preoperative CT utilization in appendectomy patients. This video has been created to provide education to community providers on best care practices. The goal is to provide a link to these educational materials on public Web sites in an effort to share our work and help community providers improve their pediatric surgical outcomes.
Our team has been successful in reducing pediatric surgery morbidities, maintaining successes, and lowering cost. Future directions include implementing the “As Low as Reasonably Achievable” principle in the use of imaging and radiation exposure for pediatric patients with trauma. We are also improving parent and family engagement in our surgery QI efforts.
This project can be used as a model for other hospitals working to improve quality of care, reduce morbidity, and lower costs. Our team found it very inspiring to assess the value of our efforts through evaluating the cost savings associated with reduction in various morbidities. In addition, it was rewarding to know that we not only impacted patient and family's lives in a positive way but that our efforts also contributed to a reduced cost burden for the local health system.
We are currently developing a pediatric perioperative surgical home to improve preparation preoperatively and follow-up after hospital discharge postoperatively. We feel our surgical home effort and our comprehensive QI program will continue to enhance the health of children requiring surgical care in our region.
The team would like to thank the entire group for their assistance: Anne Brayer, Robert Dorman, Jill Cholette, Susan Singer, Megan Gabel, Melina Zalewski, Derek Wakeman, Sean McMahon, Matt Miller, Steve Webster, Linda Prentice, Tim Stevens, Casey Calabria, Jeni Nayak, Rebecca Kanaley, Audra Webber, Ingrid Mikk, Jimena Cubillos, Sarah Ridley, Jeffrey Rubenstein, Emily Hermann, Jessica Axford, Anne Francis, Regina LoMaglio, Susan Nelson, Valerie Phillips, Diane Prinzing, John Post, Jeff Raines, Chris Smith, Brenda Tesini, and Erin Barker.
Etzioni D. A., Wasif N., Dueck A. C., Cima R. R., Hohmann S. F., Naessens J. M., … Habermann E. B. (2015). Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality. Journal of the American Medical Association
, 313(5), 505–511.
Maizlin I. I., Redden D. T., Beierle E. A., Chen M. K., & Russell R. T. (2017). Utilization of the NSQIP-Pediatric database in development and validation of a new predictive model of pediatric postoperative wound complications. Journal of the American College of Surgeons
, 224(4), 532–544.
McGuire A., Gitzelmann C. A., Davis C., Blickman J., Wolcott K., Pegoli W., & Sanders J. (2017). Prospective analysis of a novel ultrasound scoring system for pediatric appendicitis. Journal of the American College of Surgeons
, 225(4), e138.
Robinson R., Schriefer J., Gloff M., Michels J., Webber A., Hilt S., Sanders J., Hermann E. (2018). Building a pediatric perioperative surgical home: One patient, one team. Journal of Pediatric Surgical Nursing
, 7(2), 49–59.
Roddy D. J., Spaeder M. C., Pastor W., Stockwell D. C., & Klugman D. (2015). Unplanned extubations in children: Impact on hospital cost and length of stay. Pediatric Critical Care Medicine
, 16(6), 572–575.
Schriefer J., Hilt S., Sanders J., Michels J., Wolcott K., Ruddy C., & Hanson J. (2017). Implementation of a pediatric orthopaedic bundle to reduce surgical site infections. Orthopedic Nursing
, 36(1), 49–59.
Schriefer J., & Leonard M. S. (2012). Patient safety and quality improvement
: An overview of QI. Pediatrics in Review
, 33(8), 353–359.
Shander A., Hofmann A., Gombotz H., Theusinger O. M., & Spahn D. R. (2007). Estimating the cost of blood: Past, present, and future directions. Best Practice and Research Clinical Anesthesiology
, 21(2), 271–289.
Sparling K. W., Ryckman F. C., Schoettker P. J., Byczkowski T. L., Helpling A., Mandel K., … Kotagal U. R. (2007). Financial impact of failing to prevent surgical site infections. Quality Management in Health Care
, 16(3), 219–225.