Studies of surgical case cancellations for more than 20 years have published rates based on the number of cases cancelled divided by the number of cases scheduled.1–4 Recent studies looking at the effect of cancellations on operating room (OR) (anesthesia group) workload considered the scheduled hours of cases.5–7 The financial impact of cancellations on surgeons’ productivity also relates to the duration of the cancelled cases.7 For example, consider a patient scheduled for a 30-minute bronchoscopy during a thoracic surgeon’s clinic who arrives with symptoms compatible with influenza. Cancellation of that patient has a negligible impact on productivity versus cancellation on the morning of surgery of a patient scheduled for a 6-hour video-assisted thoracoscopic surgery (VATS) esophagectomy (VATS).
Notwithstanding this anecdotal example for the thoracic surgeon, it is not currently known whether it makes a substantive difference whether cancellation rates are calculated based on the number of cases cancelled (i.e., a “numeric cancellation rate”) or the number of minutes cancelled (i.e., a “duration cancellation rate”). There are historical precedents for studying cancellations based on numbers of cases. If the mean duration of cancelled cases were the same as the mean duration of all cases done by a service, then the 2 methods would provide the same results. However, if average durations of cases that are cancelled differ from the mean, the numeric cancellation rate could misrepresent the financial and operational impact of cancellations from the perspective of surgeons and surgical groups. Although annual trends in surgeons’ workloads are interchangeable whether calculated by cases or by hours,8 it is not necessarily true with respect to cancellations.
Because not all case cancellations are preventable (e.g., the patient experiences new symptoms on the day of surgery requiring further workup before surgery can safely proceed), hospitals benchmark cancellations against established norms. Schuster et al.4 recently showed that comparing cancellation rates among hospitals should account for different mixes of cases because services have different intrinsic cancellation rates. Another source of variability among services could be differences in the durations of cases that are cancelled. Anesthesiologists involved in OR management need to be knowledgeable about these potential sources of variability to assist surgeons when cancellation rates are discussed at OR committee meetings.
The purpose of this study was to determine whether there is a systematic difference (bias) between cancellation rate calculations based on the number of cases or minutes of surgery cancelled, and to evaluate heterogeneity among surgical services with respect to the bias.
The study was approved by the Thomas Jefferson University and Vanderbilt University IRBs without requirement for written patient consent. Retrospective data from all cases scheduled to be performed were retrieved from the OR case scheduling system between December 2010 and November 2012 at Hospital A, and between January 2011 and December 2012 at Hospital B. The following fields were extracted from the respective OR scheduling system databases: case identifier; scheduled date of surgery; scheduled case duration; scheduled procedure code(s) for the case; primary surgeon/service; time of OR entry (used to determine that the case was performed); and timestamps when cases were changed (scheduled, rescheduled, or cancelled). Due to institutional concerns, the names of the surgical services were de-identified.
The study was limited to elective OR cases performed on non-holiday workdays. Anesthetics administered in non-OR locations such as the gastrointestinal endoscopy and magnetic resonance imaging suites were excluded. Urgent cases also were excluded, defined for the purpose of the study as having the ASA Physical Status “E” modifier and having been entered into the OR scheduling system <5 hours before the time that the case entered the OR.
An unbiased Bayesian estimated duration was calculated for each case using the estimated duration entered by the OR scheduler and the historical data of cases of the same scheduled procedure code(s) and surgeon, as previously described.9,10 For each date, the regular working day before the scheduled date of surgery was determined (e.g., the previous Friday for Monday cases, the previous Wednesday for the day after Thanksgiving). A case was considered “cancelled” if the cancellation or scheduled date change took place after 7:00 AM or after 7:00 PM on the working day before the then current scheduled date of surgery. For example, if a case scheduled for a Tuesday were cancelled on the preceding Monday at 10:00 AM, it would be counted as having been cancelled after 7:00 AM but not after 7:00 PM on the working day before surgery.
Data were binned into 8 sequential periods, each exactly 13 weeks in duration to allow for accurate calculations of the expected small percentages of cancelled cases.11 The counts and sums of the scheduled hours of performed and cancelled cases were determined for each service during each period. The mean ± SEM was then computed among the 8 periods for each service based on counts
, where CR represents cancellation rate. The bias and 95% prediction limit (using the inverse Student t distribution to adjust for our small sample size of N = 8 periods) of the differences between CRnumeric and CRduration were then calculated and plotted versus CRduration following the method of Bland and Altman.12 For example, 1% ± 2% denotes that the bias (i.e., mean difference) equaled 1% and the 95% prediction interval equaled −1% to +3%. A positive bias indicates that the average duration of cancelled cases is less than the average duration of all cases, whereas a negative bias indicates that longer than average duration cases are being cancelled. The implication is that services with positive biases in cancellation rates based on case counts will have a smaller financial impact on surgeons than services with negative biases.
The analysis was repeated by combining all surgeons into 1 large service to obtain the overall cancellation rate at each hospital.
Cancellation rates were calculated after Freeman–Tukey double arcsine transformation, as previously described,11,13 and are reported as the mean and the 95% confidence interval (CI), weighted by the harmonic mean (HM) of the number of cases in each of the 8 periods.
Overall cancellation rates among services at hospital A were 11.6% (CI, 10.6% to 12.6%; HM = 12,395 cases per period) and 9.2% (CI, 8.6% to 9.8%; HM = 12,060) after 7:00 AM and after 7:00 PM on the working day before surgery, respectively. Corresponding values at hospital B were 10.7% (CI, 10.4% to 11.0%; HM = 14,216) and 7.5% (CI, 7.3% to 7.6%; HM = 13,716), respectively. These values are comparable to the overall cancellation rate of 12.2% among German academic hospitals, as reported by Schuster et al.4
The overall bias for all services’ cancellations after 7:00 AM on the working day before the scheduled date of surgery was −0.78% ± 1.20% at hospital A (Table 1) and 0.53% ± 0.94% at hospital B (Table 2). These biases are the pairwise differences between the numeric cancellation rates and duration cancellation rates. There was no significant correlation in the cancellation rate biases between matching services (e.g., Urology versus Urologic Surgery) at the 2 hospitals (P = 0.74, Fig. 1). There also were no significant correlations between the mean biases and the means and standard deviations of the services’ scheduled durations (P = 0.61 and P = 0.13, Tables 1 and 2).
At both hospitals, there was considerable heterogeneity among services with respect to the bias and 95% prediction limits of the estimated value of the cancellation rate using the simple ratio of cancelled to scheduled cases compared with the cancellation rate based on the ratio of minutes of cancelled cases to minutes of scheduled cases (Fig. 2). At Hospital A, the range among individual services was from −1.16% ± 1.34% to 1.93% ± 3.01% (Table 1). At Hospital B, the range was −1.08% ± 2.76% to 3.05% ± 1.89%. The prediction limits are the values relevant to the productivity of individual surgeons and their groups. A consequence of these findings is that if a hospital chooses not to repeat the analysis comparing numerical and duration cancellation rates, they should assume for any given service that the duration cancellation rate is between 3.05% lower and 1.16% larger than the numeric cancellation rate. However, for any given 13-week period, this would be a conservative estimate, as the range would be the widest precision interval among all services (±7.56%, Table 1, service G).
Not only was there large heterogeneity in the numeric cancellation rates among services and between hospitals, but there were also considerable differences among prediction intervals for the duration cancellation rates (Fig. 3). For example, consider the same surgical service “E” at the 2 hospitals (bottom 2 bars in Fig. 3). Although the services had similar observed numeric cancellation rates (7.7% and 5.0%), the 95% prediction intervals for the duration cancellation rates differed markedly. Even larger variation in the prediction intervals was demonstrated for service “C” (3rd and 4th bar from the top in Fig. 3). These wide ranges in the prediction limits arise primarily from considerable heterogeneity in the cancellation rates among 13-week intervals. For hospitals using briefer periods for analysis, the prediction intervals would be even wider.
Our study demonstrates that despite the long history of reporting cancellation rates based on case counts, this metric may not accurately represent the financial impact on surgeons’ productivity.5–7 Two services with the same numeric cancellation rate may have quite different consequences on surgeons’ productivity6 depending on the magnitude of the difference from their cancellation rates based on the total scheduled minutes of cancelled cases. From Figure 3, using data from any an 8-week period, the differences can be astoundingly large. From the perspective of the surgeon, it is the bias in cancellation rates for his or her service that matters, not the overall bias among all services.
A related and important conclusion is that hospital analysts should not naively compare cancellation rates based on counts of cases among services within a hospital and especially among hospitals, because of wide variations in bias relative to the more operationally and financially relevant cancellation rate based on minutes (Fig. 1). Bias in cancellation rate methodology needs to be added to the concern raised by Schuster et al.4 regarding comparison of overall cancellation rates among hospitals related to different contributions by individual services to the overall case mix. Together, comparing overall numeric cancellation rates among hospitals is misleading. Cancellation rates (by either methodology) can be monitored usefully by service and individual hospital for trends over time using appropriate quality control chart methods.11
Accurate determination of duration cancellation rates requires that a hospital uses unbiased estimates of surgical case durations. Simply using the scheduled case durations from the OR scheduling system without checking for systematic differences from the cases’ actual durations will influence the calculations. We used one Bayesian method that accounts for uncommon procedures, historical durations, and scheduled durations to perform unbiased calculations validly. This method has been fully described and is not difficult to implement.9,10 Other Bayesian methods for case duration prediction and for assessing cancellation rates would function equally well.14,15
There are 2 principal limitations of our study. First, the implications for how case cancellations are measured are of less financial importance to anesthesia groups than to surgeons or surgical groups, as variability in OR workload is not affected by cancellations.5,6 In other words, cancellations principally affect patients and individual surgeons, not anesthesiologists, since there is usually another case that replaces the cancelled case (e.g., an add-on case that otherwise would have been done later in the day). Second, if a hospital or anesthesia group wishes to focus on patient satisfaction issues in preference to the financial impact of cancelled cases, then continuing to use numeric cancellation rates would be appropriate. An organization might choose this approach because cancellations can have adverse psychological effects on patients.16,17 From the perspective of many patients, the duration of the case is not relevant to the impact of a last minute cancellation, since they have likely already taken time off from work, arranged for family members or friends to do likewise to take them to and from the hospital, and made other arrangements.
In conclusion, we recommend that hospitals examine their data for the presence of systematic differences (bias) between cancellations measured based on numbers of cases versus estimated durations of cases, do this by service, and thereby determine whether the more complex calculation of cancellation rates using unbiased estimates of scheduled minutes needs to be implemented.
Franklin Dexter is the section Editor of Economics, Education, and Policy for the Journal. The manuscript was handled by Dr. Steven L. Shafer, Editor-in-Chief, and Dr. Dexter was not involved in any way with the editorial process or decision.
Name: Jesse M. Ehrenfeld, MD, MPH.
Contribution: This author helped design and conduct the study and write the manuscript.
Attestation: Jesse M. Ehrenfeld has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Name: Franklin Dexter, MD, PhD.
Contribution: This author helped design and conduct the study, analyze the data, and write the manuscript.
Attestation: Franklin Dexter has reviewed the analysis of the data and approved the final manuscript.
Name: Brian S. Rothman, MD.
Contribution: This author helped write the manuscript.
Attestation: Brian S. Rothman has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Name: Adrienne M. Johnson, MA, MBA.
Contribution: This author helped conduct the study and write the manuscript.
Attestation: Adrienne M. Johnson has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Name: Richard H. Epstein, MD, CPHIMS.
Contribution: This author helped design and conduct the study, analyze the data, and write the manuscript.
Attestation: Richard H. Epstein has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.
1. Hand R, Levin P, Stanziola A. The causes of cancelled elective surgery. Am J Med Qual. 1990;5:2–6
2. van Klei WA, Moons KG, Rutten CL, Schuurhuis A, Knape JT, Kalkman CJ, Grobbee DE. The effect of outpatient preoperative evaluation of hospital inpatients on cancellation of surgery and length of hospital stay. Anesth Analg. 2002;94:644–9
3. Lau HK, Chen TH, Liou CM, Chou MC, Hung WT. Retrospective analysis of surgery postponed or cancelled in the operating room. J Clin Anesth. 2010;22:237–40
4. Schuster M, Neumann C, Neumann K, Braun J, Geldner G, Martin J, Spies C, Bauer MCASCAES Study Group. . The effect of hospital size and surgical service on case cancellation in elective surgery: results from a prospective multicenter study. Anesth Analg. 2011;113:578–85
5. Dexter F, Shi P, Epstein RH. Descriptive study of case scheduling and cancellations within 1 week of the day of surgery. Anesth Analg. 2012;115:1188–95
6. Epstein RH, Dexter FD. Rescheduling of previously cancelled surgical cases does not increase variability in operating room workload when cases are scheduled based on maximizing operating room efficiency. (In press)
7. Tung A, Dexter F, Jakubczyk S, Glick DB. The limited value of sequencing cases based on their probability of cancellation. Anesth Analg. 2010;111:749–56
8. Dexter F, Masursky D, Ledolter J, Wachtel RE, Smallman B. Monitoring changes in individual surgeon’s workloads using anesthesia data. Can J Anaesth. 2012;59:571–7
9. Dexter F, Ledolter J. Bayesian prediction bounds and comparisons of operating room times even for procedures with few or no historic data. Anesthesiology. 2005;103:1259–167
10. Dexter F, Epstein RH, Lee JD, Ledolter J. Automatic updating of times remaining in surgical cases using bayesian analysis of historical case duration data and “instant messaging” updates from anesthesia providers. Anesth Analg. 2009;108:929–40
11. Dexter F, Marcon E, Epstein RH, Ledolter J. Validation of statistical methods to compare cancellation rates on the day of surgery. Anesth Analg. 2005;101:465–73
12. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10
13. Miller JJ. The inverse of the Freeman-Tukey double arcsine transformation. Am Stat. 1978;32:38
14. Stepaniak PS, Heij C, Mannaerts GH, de Quelerij M, de Vries G. Modeling procedure and surgical times for current procedural terminology-anesthesia-surgeon combinations and evaluation in terms of case-duration prediction and operating room efficiency: a multicenter study. Anesth Analg. 2009;109:1232–45
15. Eijkemans MJ, van Houdenhoven M, Nguyen T, Boersma E, Steyerberg EW, Kazemier G. Predicting the unpredictable: a new prediction model for operating room times using individual characteristics and the surgeon’s estimate. Anesthesiology. 2010;112:41–9
16. Tait AR, Voepel-Lewis T, Munro HM, Gutstein HB, Reynolds PI. Cancellation of pediatric outpatient surgery: economic and emotional implications for patients and their families. J Clin Anesth. 1997;9:213–9
17. Ivarsson B, Kimblad PO, Sjöberg T, Larsson S. Patient reactions to cancelled or postponed heart operations. J Nurs Manag. 2002;10:75–81