Miller, Ronald D. SECTION EDITOR; Freund, Peter R. MD*†; Posner, Karen L. PhD*‡
Departments of *Anesthesiology and
‡Anthropology, University of Washington; and
†Anesthesia Clinical Services, University of Washington Medical Center, Seattle, Washington
Supported by the Department of Anesthesiology, University of Washington, Seattle, WA.
All opinions expressed herein are those of the authors.
Presented in part at the annual meeting of the Association of Anesthesia Clinical Directors, New Orleans, LA, October 14, 2001.
December 10, 2002.
Address correspondence and reprint requests to Peter R. Freund, MD, Department of Anesthesiology, University of Washington, 1959 N.E. Pacific St., Box 356540, Seattle, WA 98195. Address e-mail to firstname.lastname@example.org.
The late 1990s saw a continuing national shortage of anesthesia personnel. We previously reported on the trends in the quality of anesthesia care in a university-based anesthesia practice as it balanced the demand for increased service, decreasing reimbursement, and availability of qualified staff in the early 1990s (1). From 1992 to 1997, anesthesia productivity increased, with a significant decrease in patient injury. It was unknown whether our levels of quality could be sustained with further increases in workload and the continued pressures generated by the continuing shortage of qualified anesthesia personnel.
In this article, we report on our productivity and quality measures from 1998 through 2000 to assess whether the previous productivity and quality gains were sustained during the latter time period. We hypothesized that quality would decline as productivity was maintained or increased over a prolonged time period.
This study was conducted at the University of Washington Medical Center (Seattle, WA), a tertiary care teaching hospital with 350 beds. At the start of the study, there were 17 operating rooms (ORs) and 10 other anesthetizing locations (labor and delivery, cystoscopy, diagnostic radiology, radiation oncology, gastrointestinal endoscopy, intensive care unit, pain service, and dental clinics). At the completion of the study, there were 18 ORs and no change in other anesthetizing locations. All anesthetizing locations were contained within a single building. All ORs were on a single floor; other anesthetizing locations were on various floors of the hospital.
Productivity, caseload, and outcome (quality) data were abstracted from departmental administrative and quality-improvement reports. Our previous report analyzed increasing productivity trends between 1992 and 1997. For this analysis of trends in quality during periods of sustained productivity levels, we excluded the lowest-productivity years (1992–1993). We also excluded the initial period of highest productivity (1997) to focus on the time period when sustained high productivity might result in declines in quality. This report compares the 3-yr time period of moderate productivity (1994–1996) with a 3-yr period of high productivity (1998–2000), excluding 1997 as a transition year.
Productivity was measured as the mean monthly surgical anesthesia hours billed per attending anesthesiologist per full-time equivalent (FTE) clinical day and calculated by dividing the total anesthesia hours billed each month by the total FTE clinical days credited to all attending anesthesiologists that month. Total anesthesia hours were calculated as time units × 4 (excluding base units), as obtained from billing data. FTE clinical days were derived from the daily roster, with partial clinical day credits provided for assignment to late attending and out-of-house calls (e.g., cardiac). Acute pain service was excluded from productivity calculations. Caseload was measured as the mean monthly number of anesthetics administered (e.g., cases), derived by counting the number of anesthesia records in the departmental administrative database. Concurrency was measured as a weighted monthly mean of individual case concurrencies. Our administrative database assigned each case (excluding labor analgesia) a concurrency value in whole integers (from 1 to 4). Concurrent direction of two anesthetizing locations per attending anesthesiologist was the goal in our teaching practice. Any concurrent direction of anesthesia administration in multiple locations by an attending anesthesiologist, regardless of length of time, was considered concurrent case direction for determination of the individual case concurrency value. Concurrent direction of multiple locations included multiple residents or certified registered nurse anesthetists (CRNAs) or any combination of residents and CRNAs administering anesthesia under the direction of the attending anesthesiologist. A simple mean calculation of concurrency across all cases would result in overweighting of cases with higher concurrency. To adjust for this, we calculated an adjusted weighted mean of individual case concurrencies. Weights were chosen to eliminate the effect of duplicate counting of concurrency values for cases with concurrency >1. For example, cases with a concurrency of 2 were weighted by 0.5 in calculation of the mean monthly concurrency. Cases with a concurrency of 3 were weighted by 0.3, cases with a concurrency of 4 were weighted by 0.25, and so on. Hours worked per FTE clinical day was measured for each month by dividing productivity by concurrency (weighted means as described above). Mean productivity, caseload, concurrency, and hours worked per FTE clinical day were compared by Student’s t-test between the two study time periods (1994–1996 versus 1998–2000).
The quality of anesthesia care was measured as monthly rates of anesthesia-related events, outcomes, and human errors, as recorded in a previously described continuous quality improvement (CQI) program database (1,2). Briefly, adverse events and outcomes were reported primarily by anesthesiologists, residents, and CRNAs by placing a Y on the billing copy of the anesthesia record. Reported events and outcomes were investigated by the CQI nurse coordinator and evaluated by weekly peer review by following a structured evaluation format. Evaluation of human error was conducted through this peer review. For this analysis, events and outcomes assessed by peer review to be related to anesthesia management were aggregated into broad indicator categories: critical incidents (adverse events with no adverse outcome) or adverse outcomes (patient injury, escalation of care, and operational inefficiencies). Human errors were assessed as an additional quality indicator. An overall CQI report rate was calculated by dividing the number of records with any quality indicator by the total number of anesthetic records. Rates for each indicator were calculated by dividing the number of anesthesia records with that indicator by the total number of anesthesia records. Rates for the two study time periods were compared by a two-sided Kolmogorov-Smirnov test with Monte Carlo calculation of P values to determine statistical significance (3).
Productivity, as measured by hours billed per FTE clinical day, increased during 1998–2000 compared with 1994–1996 (P ≤ 0.01;Fig. 1). The increased productivity was associated with an increase in caseload (anesthetics administered per month), as well as with hours worked per FTE clinical day (Table 1). Concurrency of case direction remained unchanged, at 1.75 cases per attending anesthesiologist (Table 1).
The overall CQI report rate decreased during 1998–2000 compared with 1994–1996 (P ≤ 0.01;Table 2). Rates of critical incidents, patient injuries, and escalation of care declined, whereas rates of operational inefficiencies and human error showed no statistically significant differences (Table 2). To assess whether these decreases in specific quality measures were an artifact of the declining CQI report rate, we conducted a post hoc analysis of adjusted quality measures. Adjustment consisted of dividing the quality rates by the CQI report rate, in effect calculating a rate per CQI case rather than the unadjusted rate per anesthetic. Adjusted rates are shown in Table 3. After adjusting the quality rates for the declining CQI report rate, the operational inefficiency and human error rates increased during 1998–2000 compared with the earlier time period. Other adjusted quality measures showed no statistically significant change between the two time periods.
During this three-year study period (1998–2000), productivity was increased beyond the historically high levels set during 1994–1996, without any significant decrease in measurable quality (Fig. 1, Tables 2 and 3). Although there were increases in operational inefficiencies and human error rates in the more recent time period, patient injury rates did not change (Table 3). Rates of critical incidents, suggested as an indicator of operating closer to the margin of safety, were also unchanged (Table 3).
The key to our previously reported increased productivity (1) was achieving our goal of a staffing ratio of two anesthetizing locations per attending. This was maintained during the 1998–2000 study period (Table 1). We maintained 1:1 ratios where educational or patient acuity dictated. This smaller ratio was offset by a more than 2:1 ratio under certain clinical conditions and at the end of the clinical day, when stable longer cases could be assigned to the on-call attending. Therefore, the increased workload did not require the addition of more call days for the attending anesthesiologists. The on-call attending was used more efficiently by assignment to cases that extended into late afternoon and evening hours.
Since our initial publication (1), several articles have reported different measures of anesthesia productivity and have attempted to define a useful metric for anesthesia productivity that is applicable in both private and academic practice (4–6). Unlike these other reports, which focus strictly on productivity, our study links productivity to quality. Our metric of billable hours per clinical day has similar characteristics to these recent reports and also has some unique features. Our definition of productivity is derived from the standard equation used in operations management (7):
Input is the faculty resource (FTE clinical days), and output is the product (billable hours of service) produced by the input. In essence, we are measuring labor productivity, rather than the productivity of capital resources such as OR sites. Because productivity is defined as a relative measure, for it to be meaningful it must be comparable to like measures in similar operations. Like Abouleish et al. (4), we focused on the clinical day for input, and like both Feiner and Miller (6) and Abouleish et al. (5), we used time and billable hours from the anesthesia record for output. In our latest three-year epoch, the average number of hours worked per clinical day per FTE faculty (mean productivity/mean concurrency) was 8.5 hours, which is more than the 7.5-hour benchmark used by Feiner and Miller (6) and less than the 8.7, 9.9 (8), and 11.6 (5) hours calculated from data presented for academic groups reported by Abouleish et al. (5) and Abouleish (8).
Although our productivity measure has these common features, our conceptual use of productivity has important differences. We did not define our measurement from an individual perspective, but from use of a finite group resource. The group measure offered by Abouleish et al. (5) is based on the OR site as input, thereby measuring the productivity of capital. Our input was the attending anesthesiologist, so we measured labor productivity. This measure is applicable to our management planning, because capital is fixed (in the short term) and because the immediate management need is organization of personnel resources to meet the demand for services. This is not unlike the management issues addressed by Dexter et al. (9) in attempting to maximize labor productivity while minimizing staffing costs. In our practice, we calculate how many FTEs are needed per clinical day to staff the anesthesia clinical service for the year. This includes all aspects of the anesthesia service, with the exception of the chronic pain service, which is managed separately. The daily roster is managed like a business ledger by the charge anesthesiologist so that all FTEs are accounted for. FTEs are dynamically deployed on a daily basis to cover the anesthesia service needs with a minimal amount of manpower while maintaining a concurrency ratio of 2:1. The workforce is balanced to the demand, including the non-OR-based anesthesia personnel (obstetrics) who cover out-of-OR anesthesia (radiology and gastrointestinal endoscopy) along with their primary responsibilities. The OR coordinator, rather than the individual anesthesia team, is responsible for the daily inefficiencies in the schedule. A daily worksheet documents the actual FTEs used and the problems encountered. Regular meetings between the coordinators and the service chief review the operational problems and generate their solutions.
Increases in total output can be achieved by increasing FTEs, productivity, or capacity utilization (the proportion of time that the resources are actually engaged in providing service) (7). Examples of strategies to increase capacity utilization include reducing case turnover time or running ORs later each day (up to 24 hours a day, 7 days a week). During this time period (1998–2000), the shortage of qualified anesthesia personnel limited full-capacity utilization, because there were not enough personnel available to use the ORs for increasingly longer periods of time. Shortage of personnel also prevented increases in FTEs, so labor productivity increases were used to meet continued demands for service (increased output). Productivity increases could be obtained by increases in concurrency, shorter turnover time, and/or longer workdays per FTE. In this study, concurrency did not change significantly between the early and late time periods (Table 1), so increases in productivity cannot be attributed to increases in concurrency. Average hours worked per clinical day (productivity/concurrency) increased from 8.0 in 1994–1996 to 8.5 in 1998–2000. This increase, based on billed hours, could have resulted from a longer workday, shorter turnover times, or some combination of those factors. We have not measured the contributions of different capacity-utilization strategies (turnover time versus longer workdays) to increase output.
In our previous study (1), rates of critical incidents increased at higher productivity levels, raising concerns about whether these high productivity levels could be safely maintained or whether incidents would begin evolving into accidents and patient injuries under sustained production pressure. We found in this study that quality was maintained throughout an extended period of high productivity (Tables 2 and 3). The decline in the CQI report rate in 1998–2000 raises questions that cannot be answered in this report. We do not know the precision of our CQI report system, so we cannot quantify the rate of underreporting. We do not know the extent that decreasing reports reflect fewer problems versus changes in reporting behavior on the part of the anesthesia staff. In our previous report (1), we speculated that more frequent rates of critical incidents (events with no bad outcome) at higher productivity levels might reflect increased safety risk, increased compliance with the CQI program, or the use of CQI reports to express dissatisfaction with increased productivity demands. If the last explanation were correct, we could further speculate in this report that declining critical incident and CQI report rates might reflect a reduction in the use of the CQI program to express such dissatisfaction (or a reduction in dissatisfaction).
Because we cannot differentiate changes in quality from changes in CQI reporting behavior, we adjusted our quality measures in this report. Our adjusted measures of quality were conservative, removing any increase in quality that might result simply from declining report rates. We were also conservative in our analysis by excluding the initial high-productivity year (1997). It might be expected that declines in quality would lag behind increases in productivity if staff could temporarily compensate for increased caseload. To address the question of whether quality could be maintained during sustained high productivity, we excluded the initial high-productivity year and focused the analysis on the following three years of continually high productivity. The extended time period (1994–2000) and retrospective analysis make the results less likely to be attributable to a Hawthorne effect (improvement due to observation rather than to changes in work processes) than our previous report. All data analysis was performed retrospectively, eliminating any observer influence during the study periods.
The Association of American Medical Colleges, in their report on the assessment of faculty and departmental contributions to the clinical mission of medical schools’ teaching hospitals, emphasized that clinical productivity measures do not capture quality of service and urged that quality measures be integrated into clinical performance measurement (10). Anesthesiology as a specialty has a strong tradition to “do no harm” and has a clinical culture built on the primacy of patient safety. Although other medical specialties have hailed the impressive patient safety leadership that the specialty of anesthesia has provided, we are still struggling with development of appropriate benchmarking criteria for patient safety and quality in anesthesia. As the specialty deals with labor shortages, declining reimbursements, and changes in the organization and delivery of health care services, continued efforts to define productivity and quality and establish any critical linkage between work processes and care outcomes will be vital to maintaining the high profile of safety in anesthesia care.
The authors wish to thank Lynn Akerlund for her excellent assistance in the preparation of the manuscript. The authors also thank Evan Kharasch, MD, PhD, for his review and commentary.
1. Posner KL, Freund PR. Trends in quality of anesthesia care associated with changing staffing patterns, productivity, and concurrency of case supervision in a teaching hospital. Anesthesiology 1999; 91: 839–47.
2. Posner KL, Kendall-Gallagher D, Wright IH, et al. Linking process and outcome of care in a continuous quality improvement program for anesthesia services. Am J Med Qual 1994; 9: 129–37.
3. Mehta CR, Patel NR. SPSS Exact Tests 7.0 for Windows. Chicago: SPSS Inc, 1996.
4. Abouleish AE, Zornow MH, Levy RS, et al. Measurement of individual clinical productivity in an academic anesthesiology department. Anesthesiology 2000; 93: 1509–16.
5. Abouleish AE, Prough DS, Zornow MH, et al. Designing meaningful industry metrics for clinical productivity for anesthesiology departments. Anesth Analg 2001; 93: 309–12.
6. Feiner JR, Miller RD. Productivity versus availability as a measure of faculty clinical responsibility. Anesth Analg 2001; 93: 313–8.
7. Davis MM, Aquilano NJ, Chase RB. Fundamentals of operations management. Boston: Irwin McGraw-Hill, 1999: 108–9.
8. Abouleish A. Measuring clinical productivity for anesthesiology. In: Schwartz AJ, ed. ASA refresher courses in anesthesiology. Vol. 30, chapt. 1. Philadelphia: Lippincott Williams & Wilkins, 2002.
9. Dexter F, Epstein RH, Marsh HM. A statistical analysis of weekday operating room anesthesia group staffing costs at nine independently managed surgical suites. Anesth Analg 2001; 92: 1493–8.
10. D’Alessandri RM, Albertson P, Atkinson BF, et al. Measuring contributions to the clinical mission of medical schools and teaching hospitals. Acad Med 2000; 75: 1232–7.