In the current issue of Anesthesia & Analgesia, Kynes and colleagues1 describe patient complaints about anesthesiologists reported during postoperative phone calls. Among pediatric patients, tardiness from scheduled start times accounted for more complaints than any other factor under our control. Among adult patients, the use of regional anesthesia or IV sedation (as opposed to general anesthesia) resulted in more complaints than other factors under our control.1 There were large differences in the rate of complaints among the anesthesiologists.1 However, these differences were not evident once the analysis adjusted for risk factors such as the use of general anesthesia.1
Anesthesiologists differ in their clinical performance.2–4 Anesthesiology residents can assess differences among supervising faculty anesthesiologists in the quality of their operating room supervision.3 Similarly, anesthesiologists can assess differences among residents in the quality of their clinical care.4 In both cases, the assessments are reliable and valid.4,5
The same is not true regarding management skills. For example, neither anesthesiologists nor anesthesiology residents can accurately assess even their own turnover times.5 As shown in Kynes’ study and others, waiting on the day of surgery matters to patients.1,6 However, there do not appear to be differences among anesthesiologists in the minutes of patients’ waiting past scheduled start times, frequency and minutes of add-on patients waiting following submission of the request for surgery, or complaints about the care provided.1,7–8 Complaints about intraoperative anesthesia care are principally a managerial outcome, not clinical, because the incidence of satisfaction with anesthetic care is just as high among the population of patients with adverse anesthetic events as patients without such events.9 Complaints are principally related to waiting.9
The reason anesthesiologists have similar waiting from scheduled start times, waiting for add-on cases, and patient satisfaction is that they make managerial decisions using common rules-of-thumb (i.e., “heuristics”). For example, multiple studies have shown that anesthesiologists make decisions that increase their own clinical work during the hours to which they are assigned.8–11 When working in a single room (e.g., as an anesthesiology resident), this good work ethic reduces the hours worked late and the minutes that cases start late, both of which are good.12–13 Such activity can be sustained both by other physicians’ perceptions of efficiency and of (positive) team activity.5,14–15 The paper by Wang et al.11 in this issue also reaches this conclusion.
Anesthesiologists should make managerial decisions to run operating rooms as efficiently as possible. Yet, neither monitoring patient complaints,1 first case starts,7 nor patient waiting8 is useful. What should an institution monitor to assess managerial performance?
- Institutions should monitor anesthesiologists’ scheduled hours (shifts) and attendance. When anesthesiologists can work fewer or more hours for lesser or larger compensation, they make different decisions about how many hours they will work.16 For such flexibility to be provided, by definition scheduled hours must be monitored, and since it is the scheduled hours so too would attendance be recorded (i.e., showing up that day). Compensation can be a fixed payment for signing up and working a shift or based on hours actually worked during the shift. Provided the shifts are appropriately calculated statistically,17–22 individuals’ annual compensation will average out over the year to be virtually the same. This is necessarily the case for the group, because the “pot” of money earned for providing anesthesia services is the same regardless of whether the work is provided on a shift or an hourly reimbursement model.
- The reason for monitoring scheduled hours (shifts) is drawbacks in linking compensation to hours actually worked. First, many surgeons perceive that anesthesiologists are less diligent at working quickly when paid hourly for working late,5,23 even though research has shown no such relationship with compensation.10,11 Second, even though compensation is psychologically important when setting up schedules in advance, on the day of surgery anesthesiologists ignore the compensation.23 Not a single surveyed anesthesiologist defined “working late” as working beyond the shift to the time when incremental compensation started.23 Intuitively this makes sense because there are no decisions to be made since an anesthesiologist has to remain with an anesthetized patient.23
- Institutions should monitor anesthesiologist’s use of evidence-based management when making decisions at the operating room control desk and/or scheduling office the working day before surgery (e.g., following an appropriate checklist).13,20–21,24–27 One example is cases waiting to start at night. Night cases should not wait unless the anesthesiologist(s) on-call has all operating rooms in use8,13,24 for which staffing was planned.20,21 A second example is that when assigning add-on cases to operating rooms the day before surgery, rooms should be fully filled when possible.20–21,24,26–27
- Institutions should monitor anesthesiologists’ compliance with evidence-based management when performing staff scheduling (e.g., following an appropriate checklist).28 For each team (e.g., regional anesthesia) considered during staff assignment the day before surgery, staff scheduling should be based on the actual number of rooms usually run on that day of the week and for the actual number of hours each of those rooms usually runs, including all add-on cases.28 However obvious this example may seem, without a checklist, it does not tend to happen in practice because of psychological biases.29 Many groups tend, when doing staff scheduling, to plan the same numbers of rooms for each team for each day of the week; to ignore some teams entirely and assume that this can be addressed when case assignments are made; and/or to consider only “rooms” per day and not heterogeneity among rooms in the durations of the workday.
The paper by Kynes and colleagues1 is an outstanding example of the risks of comparing anesthesiologists’ performance without knowing whether the differences among individuals are associations or causal. Monitoring performance among health care providers is useful when there are differences, such as in clinical care2–4 or when performance is influenced by psychological biases.10,11,13,14,24,29
Franklin Dexter is the Statistical Editor and Section Editor for Economics, Education, and Policy for Anesthesia & Analgesia. This manuscript was handled by Dr. Steven L. Shafer, Editor-in-Chief, and Dr. Dexter was not involved in any way with the editorial process or decision.
Name: Pieter S. Stepaniak, PhD.
Contribution: This author helped write the manuscript.
Attestation: This author approved the final manuscript.
Conflicts of Interest: This author has no conflicts of interest to declare.
Name: Franklin Dexter, MD, PhD.
Contribution: This author helped write the manuscript.
Attestation: Franklin Dexter has approved the final manuscript.
Conflicts of Interest: Franklin Dexter reports no conflicts of interest other than being an author of Ref. 11.
1. Kynes JM, Schildcrout JS, Hickson GB, Pichert JW, Han X, Ehrenfeld JM, Westlake MW, Catron T, St Jacques P. An analysis of risk factors for patient complaints about ambulatory anesthesiology care. Anesth Analg. 2013;116:1325–32
2. Henrichs BM, Avidan MS, Murray DJ, Boulet JR, Kras J, Krause B, Snider R, Evers AS. Performance of certified registered nurse anesthetists and anesthesiologists in a simulation-based skills assessment. Anesth Analg. 2009;108:255–62
3. de Oliveira Filho GR, Dal Mago AJ, Garcia JH, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg. 2008;107:1316–22
4. Baker K. Determining resident clinical performance: getting beyond the noise. Anesthesiology. 2011;115:862–78
5. Masursky D, Dexter F, Isaacson SA, Nussmeier NA. Surgeons’ and anesthesiologists’ perceptions of turnover times. Anesth Analg. 2011;112:440–4
6. Dexter F, Birchansky L, Bernstein JM, Wachtel RE. Case scheduling preferences of one Surgeon’s cataract surgery patients. Anesth Analg. 2009;108:579–82
7. Dexter F, Epstein RH. Typical savings from each minute reduction in tardy first case of the day starts. Anesth Analg. 2009;108:1262–7
8. Ledolter J, Dexter F, Wachtel RE. Control chart monitoring of the numbers of cases waiting when anesthesiologists do not bring in members of call team. Anesth Analg. 2010;111:196–203
9. Brown DL, Warner ME, Schroeder DR, Offord KP. Effect of intraoperative anesthetic events on postoperative patient satisfaction. Mayo Clin Proc. 1997;72:20–5
10. Dexter F, Lee JD, Dow AJ, Lubarsky DA. A psychological basis for anesthesiologists’ operating room managerial decision-making on the day of surgery. Anesth Analg. 2007;105:430–4
11. Wang J, Dexter F, Yang K. Behavioral study of daily mean turnover times and first case. Anesth Analg. 2013;116:1333–41
12. Dexter F, Epstein RH, Traub RD, Xiao Y. Making management decisions on the day of surgery based on operating room efficiency and patient waiting times. Anesthesiology. 2004;101:1444–53
13. Dexter F, Willemsen-Dunlap A, Lee JD. Operating room managerial decision-making on the day of surgery with and without computer recommendations and status displays. Anesth Analg. 2007;105:419–29
14. Dexter EU, Dexter F, Masursky D, Garver MP, Nussmeier NA. Both bias and lack of knowledge influence organizational focus on first case of the day starts. Anesth Analg. 2009;108:1257–61
15. Shapiro LE, Alfille PH, Sandberg WS. Robots with a social memory. Anesth Analg. 2010;111:19–20
16. Miller RD, Cohen NH. The impact of productivity-based incentives on faculty salary-based compensation. Anesth Analg. 2005;101:195–9
17. Dexter F, O’Neill L. Weekend operating room on-call staffing requirements: schedule each staff member in-house or on standby from home? AORN J. 2001;74:666–71
18. Dexter F, Epstein RH, Marsh HM. Costs and risks of weekend anesthesia staffing at 6 independently managed surgical suites. AANA J. 2002;70:377–81
19. Dexter F, Epstein RH. Holiday and weekend operating room on-call staffing requirements. Anesth Analg. 2006;103:1494–8
20. McIntosh C, Dexter F, Epstein RH. The impact of service-specific staffing, case scheduling, turnovers, and first-case starts on anesthesia group and operating room productivity: a tutorial using data from an Australian hospital. Anesth Analg. 2006;103:1499–516
21. Pandit JJ, Dexter F. Lack of sensitivity of staffing for 8-hour sessions to standard deviation in daily actual hours of operating room time used for surgeons with long queues. Anesth Analg. 2009;108:1910–5
22. van Oostrum JM, Van Houdenhoven M, Vrielink MM, Klein J, Hans EW, Klimek M, Wullink G, Steyerberg EW, Kazemier G. A simulation model for determining the optimal size of emergency teams on call in the operating room at night. Anesth Analg. 2008;107:1655–62
23. Masursky D, Dexter F, Garver MP, Nussmeier NA. Incentive payments to academic anesthesiologists for late afternoon work did not influence turnover times. Anesth Analg. 2009;108:1622–6
24. Stepaniak PS, Mannaerts GH, de Quelerij M, de Vries G. The effect of the Operating Room Coordinator’s risk appreciation on operating room efficiency. Anesth Analg. 2009;108:1249–56
25. Dexter F, Shi P, Epstein RH. Descriptive study of case scheduling and cancellations within 1 week of the day of surgery. AnesthAnalg. 2012;15:1188–95
26. Dexter F, Macario A, Traub RD. Which algorithm for scheduling add-on elective cases maximizes operating room utilization? Use of bin packing algorithms and fuzzy constraints in operating room management. Anesthesiology. 1999;91:1491–500
27. Dexter F, Traub RD. How to schedule elective surgical cases into specific operating rooms to maximize the efficiency of use of operating room time. Anesth Analg. 2002;94:933–42
28. Dexter F, Wachtel RE, Epstein RH, Ledolter J, Todd MM. Analysis of operating room allocations to optimize scheduling of specialty rotations for anesthesia trainees. Anesth Analg. 2010;111:520–4
29. Wachtel RE, Dexter F. Review article: review of behavioral operations experimental studies of newsvendor problems for operating room management. Anesth Analg. 2010;110:1698–710