Kin, Cindy M.D.1; Snyder, Karen R.N.2; Kiran, Ravi P. M.D.1; Remzi, Feza H. M.D.1; Vogel, Jon D. M.D.1
In 2003, the Agency for Healthcare Research and Quality (AHRQ) proposed a set of patient safety indicators (PSIs) intended to reflect the quality of care delivered in hospitals. Patient safety indicators are presumed to be preventable by changes at the provider or system level and include iatrogenic events such as accidental puncture or laceration (APL) during a procedure, iatrogenic pneumothorax, decubitus ulcer, transfusion reaction, and postoperative hemorrhage, thromboembolism, and wound dehiscence. Administrative audits of inpatient hospitalization records identify these PSIs. The Centers for Medicare and Medicaid Services (CMS) requires hospitals to report their PSI rates to receive reimbursement. Since 2005, the rates of 8 PSIs for all hospitals have been publicly available on the CMS Hospital Compare Web site. Several PSIs are included in the current CMS pay-for-performance plan, directly affecting reimbursement. CMS issued a proposal for a new hospital value-based purchasing program in January 2011 that tied payments to performance on quality measures, intended to provide financial incentives for improvement. Accidental puncture or laceration is among the 9 AHRQ measures to be adopted into the hospital value-based purchasing program for fiscal year 2014.1 Accidental puncture or laceration is defined by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 998.2 as an accidental perforation during a procedure on a blood vessel, nerve, or organ. Excluded from the definition of APL are iatrogenic pneumothorax, laceration due to implanted device, dural laceration during spine surgery, and any injury sustained during pregnancy, childbirth, and puerperium.2
The AHRQ originally intended for PSIs to serve as a inexpensive way for hospitals and health care organizations to find potential areas of improvement.3 Given inherent limitations associated with reliance on administrative data, these measures were meant to serve as a case-finding tool to aid internal patient safety improvement efforts, not for the purposes of comparison, purchasing decisions, sanctioning, or public reporting of individual institutions.4,5 Although the AHRQ’s internal review found the APL measure to be highly reliable in that APL rates identified by administrative audits seemed to accurately reflect actual APL events, it also found that risk adjustment was very important for interpretation of APL rates.6 Still, APL as a PSI has already been incorporated into the value-based purchasing plan for CMS, affecting reimbursement, and hospital rates of APL are already published as a “Serious Adverse Event” on the CMS consumer Web site.
Evaluation of other surgical PSIs including postoperative physiologic/metabolic derangement, respiratory failure, thromboembolic events, sepsis, and wound dehiscence has led to the incorporation of higher-sensitivity definitions into current PSI algorithms.7–9 Accidental puncture or laceration was found to have a positive predictive value of 85%, where 10% of cases had coding inaccuracies and 5% indicated injuries present on admission. Of the true APLs, 27% were minor injuries such as small serosal tears with no clinical significance.10 However, it has also been shown that different detection methods identify different adverse events with very little overlap, and that only 6.2% of hospitalizations with a PSI also had a provider-reported event, and only 10.5% of provider-reported events had a PSI.11 These results bring the extension of PSI from a screening tool to a pay-for-performance metric into question.
Patient safety indicators have been found to correlate to worse outcomes. A Veterans Health Administration study found that more than 11,000 PSI events occurred in the Veterans Health Administration nationwide in 2001, and hospitalizations with PSI events had longer lengths of stay, higher mortality, and higher costs than those without PSI events.12 Accidental puncture or laceration, in particular, was found to correlate with increases in 30-day mortality, length of stay, and cost.13
Missing from the discussion of using APL as a safety measure is an evaluation of whether they are clinically significant with respect to surgical outcomes, and whether they are even avoidable given the settings in which they occur.14,15 The American College of Surgeons in 2011 appealed to CMS regarding the need for field testing for PSIs, especially for APL, because of their inherent reliability and validity problems.16 The objective of this study is to advance the understanding of the clinical significance of APL by identifying risk factors associated with APL and to assess surgical outcomes for different types of APL.
This is a single-center retrospective study based on a prospectively collected database. Demographics, diagnoses, operative details, APL type, and outcomes including surgical complications (prolonged ileus, sepsis, surgical site infection, anastomotic leak, and reoperation), length of stay, and readmission were examined. Accidental puncture or lacerations include serosal tear, enterotomy, and injury to ureter, bladder, spleen, and blood vessels. Data sources included discharge summaries, progress notes, laboratory results, and operative reports. Surgical complications were coded based on American College of Surgeons National Surgical Quality Improvement Program definitions and, when necessary (eg, ileus, anastomotic leak), unit-based definitions. Multiple complications and APLs were coded for a single case if they occurred.
This study was conducted in the Department of Colorectal Surgery at the Cleveland Clinic from July 2010 to July 2011.
The study population comprised all operations in which an APL occurred. The APL cases were identified by an administrative database by using ICD-9-CM code 998.2. Four APL cases were identified by the administrative database that were found on subsequent chart review to have had no APLs occur; these cases were removed from the study. Cases in which an injury occurred once or multiple times were included in the same group. Cases in which both serosal tears and enterotomies occurred were categorized as an enterotomy. There were no cases in which both intestinal and extraintestinal injuries occurred.
A control group of 299 cases was selected from all cases during the 13-month study period in which an APL did not occur to allow accurate analysis of patient records for both the study and control groups. Random selection was performed by scrambling the list of non-APL cases, then every eighth or ninth case was used for the control group. Anorectal cases were excluded; cases involving small bowel or colon were categorized as “abdominal,” and those involving the rectum were categorized as “pelvic.” There were no cases in the control group that were subsequently found on chart review to have an APL.
Outcome measures included prolonged ileus, sepsis, surgical site infection, anastomotic leak, reoperation, enterocutaneous fistula, length of stay, and readmission.
Analysis of risk factors for APL as well as outcomes of patients with and without APL was performed. Subgroup analysis based on type of APL (serosal tear, enterotomy, and extraintestinal APL) was performed, because these groups had differences in operative characteristics and postoperative outcomes. The numbers of patients with different types of extraintestinal APL were too few to be useful for comparison, so these injuries were grouped together. Comparisons of patient groups were performed with the Fisher exact test or a χ2 test for categorical variables, and a Wilcoxon rank sum test or Kruskal-Wallis test for quantitative variables. Multiple pairwise comparisons among APL subtypes were performed, although for such comparisons, individual significance levels of 0.017 should be considered to achieve overall 0.05 significance levels by Bonferroni.
Of the 2897 colorectal operations performed during the study period, 269 had APLs (9.2%); this group was compared with a representative subset of 299 of the 2628 patients without APL. Demographic data revealed that patients in the APL group were older than those in the non-APL group (53 years vs 49 years, p = 0.005). Also, patients in the APL group were more likely to have the preoperative diagnosis of enterocutaneous fistula (12% vs 2%, p < 0.001) and less likely to have the preoperative diagnosis of ulcerative colitis (20% vs 31%, p < 0.001) in comparison with the patients in the non-APL group. Diagnoses of neoplasm, Crohn’s disease, diverticulitis, and other diagnoses (including ischemia, prolapse, GI hemorrhage, and motility disorder) did not differ significantly between the 2 groups (Table 1).
A large proportion of APLs were intestinal injuries, with serosal tears accounting for 47% and enterotomies accounting for 38% (Table 2). The remaining 15% were extraintestinal injuries, including injuries to the spleen (4.8%), bladder (4.5%), ureter (3.3%), major blood vessels (1.9%), and vagina (0.4%). All but 6 (2%) of the 269 APLs documented in the study period were recognized intraoperatively. The majority of serosal tears and enterotomies were repaired primarily (87% and 73%), a smaller proportion resulted in small-bowel resection (12% and 25%), and a few resulted in proximal diversion (1% and 2%). Review of these operative reports reveals that, in the opinion of the operating surgeon, the severity of the bowel injury, in combination with the underlying disease, resulted in the decision to perform a resection or diversion. The 2 delayed diagnoses of enterotomy required reoperation. Of the 13 splenic injuries, 11 were managed with packing, coagulation, or hemostatic agents, one was managed with splenorrhaphy, and one required splenectomy. There were 12 bladder injuries, 58% of which were repaired primarily by the primary colorectal surgeon, whereas, in 25%, intraoperative urological surgery consultation was obtained for repair. The 2 delayed diagnoses of urine leak from the bladder were managed with percutaneous drainage of a urinoma and continued Foley catheter drainage. Of the 9 ureteral injuries, 78% were diagnosed intraoperatively and repaired by a consulting urological surgeon, whereas 22% were recognized postoperatively and managed with percutaneous nephrostomy drainage. Three of the 5 injuries to major blood vessels were repaired by consulting vascular surgeons, whereas the other 2 were repaired by the primary colorectal surgeon. The 1 vaginal injury was repaired by a consulting gynecological surgeon.
Operative factors were examined (Table 3). Patients with APLs were significantly more likely to have undergone a previous operation than those without APLs (91% vs 62%, p < 0.001). The OR of having an APL was 6.4 in patients who had a previous operation in comparison with those who had not. However, subgroup analysis demonstrated that those with extraintestinal APL were no more likely to have had a previous operation than those without any APL occurring in their operations (60% vs 62%, p = 0.8), whereas the occurrence of serosal tears and enterotomies were both significantly associated with previous operations (95% and 99%). Similarly, there was a significantly higher proportion of open cases in the APL group than in the group without APL (96% vs 77%, p = 0.001); this difference was seen in all types of APL. Operative time was significantly longer in operations in which APL occurred (189 vs 146 minutes, p = 0.001), although subgroup analysis demonstrated that serosal tears had no difference in operative time in comparison with the group without APL (138 vs 146 minutes, p = 0.08), whereas all other APL types were associated with longer operative times. Operations were divided into patients having abdominal operations and pelvic operations. In comparison with the control group and other APL groups, the extraintestinal APL group had a significantly greater proportion of patients undergoing pelvic operations (52%).
Postoperative outcomes are shown in Table 4. Overall, patients with APL had longer length of stay than those without APL (10.2 vs 7.8 days, p = 0.002) (Fig. 1). Subgroup analysis revealed that serosal tears made no difference in length of stay in comparison with patients who had no APLs (7.9 vs 7.8 days, not significant). However, enterotomies and extraintestinal APLs were significantly associated with longer lengths of stay (11.9 and 13.2 days) in comparison with patients without APLs and patients with serosal tears.
Analysis of surgical outcomes such as reoperation, sepsis, abscess, leak, and readmission show a similar trend (Fig. 2). Although the APL and non-APL groups had similar rates of reoperation overall (6.7% vs 7.1%, not significant), patients with serosal tears had the lowest rate of reoperation (3.7%), whereas those with extraintestinal APLs had the highest rate (12.5%), which was significant difference in comparison with patients without APL. A similar effect was seen in the rate of sepsis; there was no difference between the APL and non-APL groups overall, but the rate of sepsis in patients with serosal tears (0%) was significantly lower than in those with enterotomies (7.8%) or extraintestinal APLs (12.5%). The difference in sepsis rates between patients with no APL and those with extraintestinal APL was also significant (3.3% vs 12.5%, p = 0.02). Serosal tears in comparison with extraintestinal APL also had a significantly lower rate of intra-abdominal abscess (1.6% vs 10%, p = 0.03). The group with serosal tears also had the lowest rate of anastomotic leak (0.8%), which was significant in comparison with patients with no APLs (6%, p = 0.04) and patients with enterotomy (6.9%, p = 0.02). Readmission rates were actually higher in patients without APLs than in patients with APLs (10% vs 5.2%, p = 0.04). Serosal tears had the lowest rate of readmission (3.9%). There was no difference among all groups in the rates of prolonged ileus, superficial or deep surgical site infection, small-bowel obstruction, or death.
There are several preoperative and operative characteristics that appear to be related to the occurrence of APL, such as the diagnosis of enterocutaneous fistula and reoperative settings. However, this elevated risk was seen only in patients who had intestinal APL and not in patients who had extraintestinal APL. This latter group was no more likely to have had undergone a previous abdominal operation than those patients who had not had an APL. A larger majority of APL cases were performed open, probably because of a difficult or reoperative field in which a laparoscopic approach was not possible. Operative time, another indicator of surgical complexity, was significantly longer in cases involving APL, with the exception of serosal tears. Extraintestinal APLs occurred more often in cases involving pelvic dissection. Thus, the occurrence of APL is correlated with technical difficulty. The subsequent outcomes, therefore, can also be attributed to the underlying complexity of the patients’ condition, not just the occurrence of the APL itself.
The AHRQ’s definition of APL, in fact, equates it with “technical difficulty,” which is incongruent because certainly technical difficulty cannot be synonymous with a deficit in patient safety.6,12 Previous studies have found that independent predictors of inadvertent enterotomy are patient age, 3 or more previous laparotomies, and pelvic dissection.17 Extraintestinal APLs such as bladder injury also occur in cases of technical difficulty such as colon cancer, diverticulitis, and IBD.18 The high rate of reoperative cases and diagnoses of enterocutaneous fistula in the APL group is more a reflection of the inherent technical difficulty of the case than of a preventable patient safety issue.
Postoperative outcomes of the APL group compared with the non-APL group revealed longer length of stay and higher rates of sepsis and reoperation in the APL group, but this effect was not seen in patients with serosal tears. APLs had no association with ileus, surgical site infections, readmission rates, and mortality. Abscesses were more common in patients with enterotomies, but not significantly different than the non-APL group.
A multicenter study conducted in response to a report in the print media detailing bile duct injuries during cholecystectomies at Canadian hospitals based on administrative audits of hospital discharge data with ICD-9-CM code 998.2 found that only 10% of these injuries were clinically relevant. The large majority of these injuries referred to a small tear in the gallbladder on removal of the organ, rather than to actual bile duct injuries causing serious morbidity, reflecting the lack of specificity of ICD-9-CM code 998.2.15
Just as the nonspecificity of ICD-9-CM code 998.2 led to misleading and exaggerated reports of bile duct injuries during cholecystectomies, so too has it led to a misrepresentation of surgical mishaps occurring in colorectal surgery. Previous studies have suggested that a significant proportion of APL may not be clinically relevant and that they occur at an acceptable rate relative to the procedure being performed.14 Our data demonstrate that serosal tears have no association with negative outcomes, whereas extraintestinal APLs and enterotomies are associated with measurably worse outcomes, highlighting the importance of considering clinical relevance in the interpretation of administrative data.
A limitation of our study is that the subjects are operative cases within a highly specialized colorectal unit, most of which are complex reoperative cases. Thus, our results may not be generalizable to the majority of other surgical units. Another limitation is the loss of sensitivity from analyzing all extraintestinal APLs as 1 group, because these injuries are quite disparate with regard to severity and potential repercussions. However, given the small number of these complications, grouping them together was the only way of including them in the analysis. Finally, because the control group was a subset of all non-APL cases, the differences between the control and study groups may be affected by sampling error.
On the CMS Hospital Compare Web site, tertiary and quaternary care centers consistently have higher APL rates in comparison with the national average and with smaller community hospitals in their respective regions. Accidental puncture or laceration in the absence of risk adjustment serves more as a marker of higher complexity than of poor surgical quality. This is demonstrated also by the incongruity between APL rates and other measures of hospital quality. For example, although PSIs such as postoperative thromboembolic events and obstetrical trauma were associated with less favorable accreditation by Joint Commission on Accreditation of Healthcare Organizations, there is an inverse relationship between APL rates and favorable Joint Commission on Accreditation of Healthcare Organizations accreditation.19,20 Accurate interpretation of administrative data is critical not only for consumer education, but now also for equitable reimbursement and to prevent the creation a disincentive for hospitals and providers to care for the sickest patients.21
The results of this study have timely policy implications. If the collective mission of health care providers, hospitals, and the federal government is to improve the quality and safety of health care, then it is important to ensure that our measures accurately reflect quality and safety. Because consumers demand more transparency in their consumption of health care, it is very important that public reporting is also accurate.
Accidental puncture or laceration lacks specificity and clinical relevance as a quality indicator, because it can be both a marker for surgical blunders as well as a marker for surgical complexity. To refine APL into a meaningful quality indicator, serosal tears, accounting for almost half of all APL, should be excluded, because they have no effect on surgical outcomes. In addition, risk adjustment based on the results of this study may be helpful to surgeons and administrators who use the patient safety indicators to measure quality in surgery. Until these revisions are considered, APL in colorectal surgery should be used for department- or hospital-level quality improvement, not for public reporting and pay-for-performance.
3. Johantgen M, Elixhauser A, Bali JK, Goldfarb M, Harris DR. Quality indicators using hospital discharge data: state and national applications. Jt Comm J Qual Improv. 1998;24:88–105
4. Miller MR, Elixhauser A, Zhan C, Meyer GS. Patient Safety Indicators: using administrative data to identify potential patient safety concerns. Health Serv Res. 2001;36(6 pt 2):110–132
5. Iezzoni LI, Daley J, Heeren T, et al. Identifying complications of care using administrative data. Med Care. 1994;32:700–715
7. Rosen AK, Itani KM, Cevasco M, et al. Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events? Med Care. 2012;50:74–85
8. Kaafarani HM, Rosen AK. Using administrative data to identify surgical adverse events: an introduction to the Patient Safety Indicators. Am J Surg. 2009;198(5 suppl):S63–S68
9. Romano PS, Mull HJ, Rivard PE, et al. Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data. Health Serv Res. 2009;44:182–204
10. Kaafarani HM, Borzecki AM, Itani KM, et al. Validity of selected Patient Safety Indicators: opportunities and concerns. J Am Coll Surg. 2011;212:924–934
11. Naessens JM, Campbell CR, Huddleston JM, et al. A comparison of hospital adverse events identified by three widely used detection methods. Int J Qual Health Care. 2009;21:301–307
12. Rosen AK, Rivard P, Zhao S, et al. Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data? Med Care. 2005;43:873–884
13. Rivard PE, Luther SL, Christiansen CL, et al. Using patient safety indicators to estimate the impact of potential adverse events on outcomes. Med Care Res Rev. 2008;65:67–87
14. Utter GH, Zrelak PA, Baron R, et al. Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator. Ann Surg. 2009;250:1041–1045
15. Taylor B. Common bile duct injury during laparoscopic cholecystectomy in Ontario: does ICD-9 coding indicate true incidence? CMAJ. 1998;158:481–485
16. . American College of Surgeons letter to Marilyn Tavenner (Acting Administrator of Centers for Medicare & Medicaid Services) from ACS Executive Director David Hoyt. June 25, 2012:10–11 . Available at: http://www.facs.org/ahp/fy13-comments.pdf
. Accessed September 26, 2012
17. Van Der Krabben AA, Dijkstra FR, Nieuwenhuijzen M, Reijnen MM, Schaapveld M, Van Goor H. Morbidity and mortality of inadvertent enterotomy during adhesiotomy. Br J Surg. 2000;87:467–471
18. Armenakas NA, Pareek G, Fracchia JA. Iatrogenic bladder perforations: longterm followup of 65 patients. J Am Coll Surg. 2004;198:78–82
19. Miller MR, Pronovost P, Donithan M, et al. Relationship between performance measurement and accreditation: implications for quality of care and patient safety. Am J Med Qual. 2005;20:239–252
20. Isaac T, Jha AK. Are patient safety indicators related to widely used measures of hospital quality? J Gen Intern Med. 2008;23:1373–1378
21. Merkow RP, Bilimoria KY, Cohen ME, Richards K, Ko CY, Hall BL. Variability in reoperation rates at 182 hospitals: a potential target for quality improvement. J Am Coll Surg. 2009;209:557–564
Patient safety; Accidental puncture or laceration