Benchmarking is defined as comparing performance by using specified definitions with other groups, or against industry-wide data, to assess operational effectiveness and productivity of an individual group (1). Benchmarking the performance of anesthesiology groups can be particularly challenging because of the differences in staffing ratios (i.e., concurrency) between groups (2). Because of this, using traditional “per full-time-equivalent (FTE)” measurements may lead to inaccurate conclusions because a group in which physicians personally administer most anesthetics will appear to produce fewer units per FTE than one in which anesthesiologists concurrently supervise more than one nurse anesthetist, anesthesia assistant, or resident (3).
In contrast, productivity measurements based on “per anesthetizing operating room (OR) site” and “per case” are not influenced by differences in concurrency and thus permit more meaningful comparisons among small samples of both academic and private-practice anesthesiology groups (3,4). Such comparisons suggest that academic anesthesiology groups generate fewer American Society of Anesthesiologists (ASA) units per hour than private-practice groups because academic groups deal with surgical procedures of longer durations. The comparisons also suggest that a larger sample size would permit more focused benchmarking of anesthesiology groups by allowing for additional groupings to be made. In the present study, we used such grouping variables as types and sizes of hospitals and types of surgical staff to compare clinical productivity in a broad survey of academic anesthesiology programs.
We requested billing and staffing data as previously defined (3,4) for calendar or fiscal year 2001 from the membership of the Society of Academic Anesthesiology Chairs and the Association of Anesthesiology Programs Directors. Billing data included total ASA units billed (tASA), total 15-min time units billed (TU), and the total cases billed. Staffing data included the average daily number of anesthetizing sites (OR sites), which included remote sites. All types of anesthesia care billed using ASA units were included except for obstetrical anesthesia care. Pain management and critical care were excluded because these services are billed with resource-based relative-value system units. Any clinical activity that was not billed (e.g., preoperative outpatient assessment clinic) was also excluded. Departments that provided care in more than one hospital were asked to report hospitals separately when possible.
Descriptive data for each hospital were collected for analysis by different grouping variables (i.e., type and size of hospital, type of staff). The types of surgical staffs were academic (i.e., surgical residents involved in all or almost all surgical procedures), private-practice (i.e., no surgical residents involved), and mixed (i.e., surgical residents involved in some surgical procedures). The types of hospitals were ambulatory surgical centers (ASCs), county/city indigent care hospitals (a primary hospital for indigent care in an area that could be the primary teaching hospital for the medical school), academic medical centers (AMCs) (not a primary indigent care hospital, but still a primary teaching hospital for the medical school), and community hospitals (all other hospitals). The final variable used for analysis was based on the number of ORs at each of the hospitals (defined as small [1–9 OR sites], medium [10–19], or large [20 or more]).
The productivity measurements calculated from these data were the same as those previously described (3,4). Measurements included total ASA units per OR site (tASA/OR site), billed hours per OR site per day (h/OR/d), billed hours per case (h/case), total ASA units billed per hour of anesthesia care (tASA/h), and base units per case (base/case).
Median productivity measurements were reported and compared by using the different grouping variables. Productivity data, listed by quartiles, were also determined for all categories within each grouping variable. The number of OR sites and hospital types (four categories for each variable) were assessed for the five productivity measurements by using the Kruskal-Wallis test. Multiple comparisons were conducted by using Fisher’s least significant difference procedure computed on the ranks with a Bonferroni adjustment for the number of comparisons. Surgical staffs (academic, private practice, and mixed) were assessed by using the Wilcoxon’s two-sample test. Spearman’s correlation coefficient was used for analysis of correlation among the five productivity measurements. All tests (except multiple comparisons) were assessed at the 0.05 level of significance.
To illustrate that the overall median data were less effective for benchmarking than the detailed quartile data arranged by grouping variables, we selected three groups for comparisons of productivity measurements as previously described (4).
Of 137 groups/departments that were listed as members of the Society of Academic Anesthesiology Chairs and Association of Anesthesiology Programs Directors, 42 returned surveys (31% response rate) describing 69 different hospitals. Of the submitted surveys, 37 groups representing 58 hospitals submitted complete data that were verified as valid. Data were excluded if the survey was incomplete (e.g., no TU reported).
The number of hospitals and median values of the productivity measurements for each grouping variable are shown in Table 1. All of the ASCs reported fewer than nine OR sites. In comparing ASCs with non-ASC hospitals of the same size (small) or different size (medium or large), ASCs had significantly less tASA/OR site, fewer h/OR/d, and fewer h/case than non-ASC hospitals. Although ASCs had a larger tASA/h and a smaller base/case, these values were not significantly different from those of non-ASC hospitals (Figs. 1 and 2).
When comparing anesthesiology groups in non-ASC hospitals of different sizes, the productivity measurements were not significantly different, although smaller hospitals had smaller tASA/OR site and h/OR/d.
The type of hospital significantly influenced the productivity measurements. As noted above, ASCs were distinctly different than the other three types of hospitals. For non-ASC hospitals, tASA/OR did not differ significantly. For h/OR/d, community hospitals had a significantly shorter values than AMCs or indigent hospitals. Hourly productivity (tASA/h) and surgical duration (h/case) were not significantly different between AMCs and community hospitals, but were significantly different from indigent hospitals. Community hospitals were almost always staffed by mixed or private-practice surgical staffs (92%), whereas academic surgeons staffed 78% of the AMCs and 100% of the indigent hospitals. Although indigent hospitals and ASCs trended to have a smaller base/case, there was no significant difference among the four different hospital types.
In comparing hospitals by surgical staff, the hospitals with an academic surgical staff had a significantly longer h/case and a smaller tASA/h. These hospitals also trended to have a longer h/OR/d and a similar tASA/OR site. Because base/case was not significantly different, we concluded that the smaller tASA/h was attributable to the longer surgical durations. Therefore, anesthesiology groups in academic hospitals trended to work more billed hours than the mixed/private-practice hospitals to produce the same tASA/OR. Both tASA/OR site and billed h/OR/d were highly correlated (r = 0.86), whereas tASA/h and h/case had a large negative correlation (r = −0.68).
In Tables 2–4, the clinical productivity measurements of three groups (Group A = ASC, Group B = AMC, and Group C = community) were benchmarked by using quartile data arranged by group variable and category. In Appendix 1, the results of the productivity measurements in terms of quartiles (0, 25th, 50th, 75th, and 100th) for each category within each grouping variable (Table 5 = all groups, Tables 6–9 = by OR sites number, Tables 10–13 = by type of hospital, and Tables 14 and 15 = by type of surgical staff) are shown.
For example, Group A (see Table 2) is an ASC with an average of 2.0 OR sites per day in which both academic and private-practice surgeons perform surgery. In comparison to all groups, Group A is in the lowest quartile (0–25th percentile) for overall productivity (tASA/OR), billed hours per day (h/OR/d), and surgical duration (h/case) but has a large number of base units per case. The combination of large base units per case and short surgical duration results in large hourly clinical productivity (tASA/h). Although Group A’s h/OR/d is only at the median, overall productivity (tASA/OR) is more than the ASC’s median because hourly productivity is larger. In the second example, Group B (Table 3) provides anesthesia in a medium hospital (17.2 ORs) AMC in which only academic surgeons perform surgery. Group B’s overall productivity (tASA/OR) is above the median for all groups, medium hospitals, and hospitals with academic hospitals. On the other hand, compared to other AMCs, Group B is below the median for overall productivity. Although the group’s h/case is at the median for all groups (which includes ASCs), the group’s h/case is among the shortest when compared to similar hospitals (e.g., AMCs).
Group C (Table 4) is a small (6.0 ORs) community hospital with a mixed surgical staff. As with Group A, benchmarking against all groups is less useful than more focused benchmarking against other small hospitals (1–9 non-ASC) or community hospitals.
Differences in staffing ratios (i.e., concurrency) confound previously published “per FTE” analyses (5) of clinical productivity of anesthesiology groups. Anesthesiologists who supervise more rooms simultaneously generate larger numbers of tASA per hour and therefore per FTE (6). When comparing the clinical productivity of anesthesiology groups, the differences in concurrencies can result in inaccurate conclusions (3). However, previous comparisons using the measurements in the present study have facilitated meaningful comparisons of clinical productivity between a sample of private-practice and academic anesthesiology groups, demonstrating that longer duration of surgery significantly reduced tASA/h in academic anesthesiology groups when compared with private-practice groups (4). These comparisons also suggested that a larger sample of anesthesiology groups would permit better benchmarking by allowing grouping variables other than type of practice (academic or private-practice) to be analyzed. In this investigation, several additional grouping variables (types and sizes of hospitals and types of surgical staff) were studied. The results demonstrate that the type of clinical setting was associated with distinct differences in productivity metrics. Recognizing these differences allowed for more complete comparisons than simply collating all the results into one group (i.e., “all academic departments”).
Whether analyzed by type of hospital or by the number of ORs, ASCs differed significantly from other facilities. ASCs function distinctly differently from their non-ASC counterparts. First, ASCs have fixed hours of operations (i.e., the recovery room) and close at a specific time, thereby limiting the hours the ORs are scheduled for operation and the maximal expected duration of surgery. In addition, by definition, the types of surgical procedures are limited to outpatient or ambulatory surgery; these hospitals are also limited in scope because of the facility’s capabilities (e.g., equipment, space, or staff). Because of these differences, the categories within the hospital size grouping variable were separated into ASC and non-ASC facilities.
The productivity measurements illustrate the results of this pattern of utilization (Table 1). ASCs produced the fewest h/OR/d. The median value for h/OR/d was only 3.8 hours for ASCs, although unbilled hours, including turnover time and room set-up time, extend the amount of time the OR is used each day. In contrast, non-ASC facilities usually are not limited by fixed hours. When comparing non-ASC facilities with a small number of ORs to ASCs, the small, non-ASC facilities generated 75% more billed hours per OR (Table 1 and Fig. 1). The fewer billed h/OR/d of ASCs also results in lower tASA/OR. Furthermore, surgical duration (h/case) is also shortest in ASC facilities (Fig. 2).
In a previous study comparing private-practice and academic groups, longer surgical duration in academic groups significantly reduced hourly billing productivity (tASA/h) (4), presumably because groups with shorter surgical durations were able to perform more surgical procedures, and, hence, generate more base units per unit time. This relationship was also shown when comparing academic groups to Medicare averages (7). In this previous study, the billing data, including anesthesia time, of four academic anesthesiology departments were compared with Medicare average times for the same anesthesia codes. Each of the four departments had surgical durations exceeding the Medicare average and thus generated fewer tASA/h than average. This negative correlation is consistent with the results in the present study, in which tASA/h and h/case were negatively correlated similar to the previous study (r = −0.60 for the present study versus r = −0.68 for the previous study).
In the current study, the relationship between tASA/h and surgical duration is evident among ASCs. The overall productivity (tASA/OR site) is smallest in ASCs. Because the surgical duration is shortest among ASCs, these types of hospitals had the most hourly billing productivity (tASA/h) despite having the smallest base/case. Only the much lower number of h/OR/d limits the ASC’s productivity. A similar distinction can be seen when using the grouping variable of type of surgical staff (academic surgical staff versus mixed or private-practice surgical staff). The groups were analyzed separately because of the assumption that participation of surgical residents in surgery would be associated with a longer surgical duration. As expected, the hospitals where residents were involved in all or almost all surgical procedures had the longest surgical durations.
The methodology and limitations of the survey technique and the productivity measurements used to compare anesthesiology groups have been previously discussed (3,4). As in previous work, the current study provides a basis for external comparisons but does not suggest the best way for a group to measure clinical productivity. Unlike the previous study, the data reported (Tables 5–15 in Appendix 1) allow for benchmarking of an academic anesthesiology group. The response rate in this survey is equal to or more than that found in currently accepted benchmarking databases of the Medical Group Management Association (8,9).
A major implication of the results of this study can be seen in the examples of benchmarking that are provided (Tables 2–4). When comparing anesthesiology groups or the hospitals in which they provide care, it is important to compare similar circumstances. For instance, comparing Group A (an ASC) to the overall productivity measures of all groups could prompt the conclusion that Group A is not productive because they work few hours (Table 2). However, in comparison to other ASCs, Group A has high overall productivity and median billed hours. Similarly, in Group B, overall productivity (tASA/OR) is above the median in comparison to all groups or to hospitals with similar number of ORs (10–19) but is below the median when compared with AMCs (Table 3). The surgical duration for Group B is among the shortest in comparison to AMCs, facilities with 10–19 ORs, and those in which academic surgeons perform most surgery, but, when compared with all groups (including ASCs), the duration is at the median. For Group C, a comparison with all groups shows that this group’s productivity and billed hours worked are among the smallest (0–25th percentile) (Table 4). However, in comparison to other community hospitals, the group’s productivity is in the 2nd quartile (25th–50th percentile).
In addition to benchmarking against outside groups, the results are pertinent to anesthesiology departments that provide care at more than one hospital. For example, if Groups A, B, and C were three hospitals at which one academic anesthesiology department provided care, clinical productivity would be difficult to compare among hospitals, even within the same group of anesthesiologists, because of the different organizational factors. In comparing productivity among facilities, the department should consider organizational factors that influence productivity (i.e., type of hospital, number of OR sites, and surgical staff) and should set individual productivity goals for each facility based on benchmarking and past performance. The same methodology can be used to compare productivity in hospitals in which operational factors are similar but in which revenue generation is different because of differences in payer mix.
From the current study and previous works (3,4), an algorithm of clinical productivity measurements for an anesthesiology group has been developed (Fig. 3). Overall productivity is measured by using tASA/OR. Two major factors determine tASA/h—the number of units billed per billed hour (hourly productivity, or tASA/h) and the number of billed hours per OR site (hours per OR per day, or h/OR/d). Hourly productivity is most strongly influenced by surgical duration (h/case) and base units per case (base/case). Billed hours per OR site are determined by workload (numbers of cases and case duration) and utilization of resource hours (case scheduling, delays, cancellations, block time allocation, number of OR sites).
In addition to comparing overall clinical productivity, an anesthesiology department chair or group leader can use comparisons based on the algorithm to help in evaluating the operations of the group and the hospital’s OR. Although clinical benchmarking is often used to help determine personnel requirements, the staffing needs of anesthesiology groups cannot be directly determined by productivity benchmarking (10) because of the confounding influences of number of clinical sites, staffing ratios, and second-shift needs (including call and post-call) on the numbers of personnel required. However, productivity measurements can help support or refute internal evaluations of the number of OR sites. If h/OR/d is large in comparison to other similar hospitals, then there are more surgeries being performed on evenings, nights, or weekends. In this case, increasing the number of OR sites would reduce h/OR/d if the late or weekend surgeries could be done during the day. In contrast, small h/OR/d is not as easy to interpret. A small h/OR/d may indicate small actual hours worked, but it might also reflect inefficient OR scheduling with prolonged intervals in which rooms are not used, but during which anesthesia personnel must be available. The relationship between billed hours and actual hours worked (or available) is similar to the raw utilization of OR time (i.e., the time that the patient is in the OR divided by available OR time). Thus, in instances in which h/OR/d is small, factors that may influence utilization (e.g., allocation of block time, number of OR sites, case scheduling, delays, and cancellations) must be evaluated to determine why this is the case.
The algorithm also illustrates the negative effect of surgical duration on hourly billing productivity. When surgical duration is short, more short cases than long cases can be done per TU, which results in more base units that can be billed in a set time period, which in turn increases tASA/h. As seen in this study and a previous study (7), for academic groups that provide anesthetic care for academic surgical programs, surgical durations are longer than average and tASA/h is smaller. Therefore, to have the same overall productivity (tASA/OR), longer hours must be worked (i.e., larger h/OR/d), resulting in increased staffing costs. If, at a relatively small tASA/h, insufficient numbers of hours are worked to offset the small hourly productivity, both tASA/OR and revenue would be smaller. In either situation, the data provide evidence that academic anesthesiology departments require subsidization to offset the increased expense to those departments supporting surgical residency programs.
Finally, despite evidence indicating that reducing reasonable length turnover time (<60 min) will not allow for an additional surgery to be performed (11), OR managers, nurses, surgeons, anesthesiologists, and hospital administrators spend time and energy fo-cused on turnover time reduction. Because turnover time is non-billable time, a change in turnover time would not be reflected in any comparison of clinical productivity using measurements based on ASA units (Fig. 3). Therefore, unless an additional case were to be performed in an OR, differences in turnover time would not affect the measurements. It is important to note that, although actual hours worked would be affected, only billable time would be compared.
In conclusion, facilities in which academic anesthesiology departments provide care have highly variable operations that strongly influence clinical productivity. By grouping facilities based on type of hospital, number of OR sites, and type of surgical staff, academic anesthesiology departments (and hospitals) can be better compared by using clinical productivity measurements based on “per OR site” and “per case” measurements (tASA/OR, billed h/OR/d, h/case, tASA/h, and base/case). Reporting quartile data by focused grouping variables allows an anesthesiology group to compare its clinical productivity with groups practicing in similar clinical settings.
The authors thank Jordan Kicklighter, BA (editor) and Christy Perry (editor) in the editorial office of the Department of Anesthesiology at the University of Texas Medical Branch, Galveston, TX, for preparing and editing the manuscript.