Secondary Logo

Journal Logo

Evaluating Intraoperative Therapeutic and Diagnostic Interventions

Klein, Nava, BA, RN; Weissman, Charles, MD

doi: 10.1097/00000539-200211000-00050
TECHNOLOGY, COMPUTING, AND SIMULATION: Research Report
Free

A cost-conscious health care system requires detailed measures of its activities, including measurements of care provided to perioperative patients. Because there are no scoring systems that quantify the extent of intraoperative care interventions, we developed an intraoperative therapeutic intensity score (I-TIS). Physiological/biochemical monitoring and therapeutic interventions were assigned one to four points on the basis of the resource utilization and/or intensity of care they each reflect. Scoring was performed on actual patients, and the results were compared with ASA classification and surgical complexity. A 78-item scoring system was developed and assessed by using two patient groups. Group 1 (n = 307) entered the postanesthesia care unit (PACU) for short postoperative stays and had an I-TIS of 7.3 ± 5.0; Group 2 patients (n = 443) were either admitted to the surgical, cardiothoracic, or neurosurgical intensive care units or had extended PACU stays, and they had an I-TIS of 25.2 ± 12.4 (P < 0.001 versus Group 1). The correlation of I-TIS with the surgical complexity classification was r = 0.77, with ASA base relative value units was rs = 0.75, and with the ASA physical status classification was rs = 0.49. The score correlated well with surgical complexity and was able to differentiate between the intensity of care during various surgical procedures.

Department of Anesthesiology and Critical Care Medicine, Hebrew University, Hadassah School of Medicine, Jerusalem, Israel

Supported by a grant from The Israel National Institute for Health Policy and Health Service Research.

June 13, 2002.

Address correspondence and reprint requests to Charles Weissman, MD, Department of Anesthesiology and Critical Care Medicine, Hadassah University Hospital, Kiryat Hadassah, POB 12000, Jerusalem, Israel 91120. Address e-mail to Charles@ hadassah.org.il.

A cost-conscious health care system invites close assessment of medical care. This has spawned a need for detailed measurements of various health care activities, including perioperative care. Methods are currently available to evaluate operative risk (e.g., ASA physical status classification and Goldman cardiac risk score) and to assess surgical complexity (1). However, while we designed a study of the reasons patients are admitted to intensive or intermediate care units after surgery, it became evident that there are no measures of intraoperative therapeutic intensity similar to those available to assess the intensity of intensive care unit (ICU) care. The concept of evaluating the intensity of therapeutic interventions was introduced by Cullen et al. (2) and Keene and Cullen (3), who developed the therapeutic intervention score system (TISS) for ICU patients. This scoring system quantifies the amount of intensive care provided to patients by measuring nursing activities, monitoring techniques, resuscitation procedures, and the use of advanced technologies. Although TISS was found to adequately quantify ICU care intensity and also to reflect resource utilization and costs (4), it is not appropriate for evaluating intraoperative care because intraoperative care is also rendered to noncritically ill patients. Therefore, an intraoperative therapeutic intervention score (I-TIS) was developed and evaluated by using a scoring scheme similar to that used by the TISS system.

The development objectives for the I-TIS were for an easily used scoring system to quantify the extent of physiological monitoring, therapeutic or other (nonsurgical) interventions, and biochemical diagnostic testing performed during surgery. The score was to be applicable to all types of surgical patients and procedures, as well as to have the capability to be used across institutions. There was no intention to assess operative risk or the intricacies of surgical or anesthesia technique. The intended use of the score was as a health services research tool to quantify intraoperative resource utilization as it relates to medical (especially anesthesia) care. This article details the proposed scoring system and examines its validity by comparing it with existing scoring systems. In addition, a post hoc analysis was performed to determine whether the score differentiates between patients having short postoperative postanesthesia care unit (PACU) stays and those receiving more intense postoperative care (extended PACU stay or ICU admission).

Back to Top | Article Outline

Methods

The scoring scheme used in the TISS system (2) was the framework for designing the I-TIS. Interventions were given one to four points on the basis of the resource utilization and/or the intensity of care they each reflect. Criteria for including specific items in the score were that they be interventions that reflect intraoperative physiological monitoring (e.g., hemodynamic and neurological monitoring), biochemical testing (e.g., arterial blood gas determinations), therapeutic (e.g., vasoactive drug and blood product administration), or other (e.g., prone positioning) interventions. The development process involved the authors developing a draft score that was based, in part, on items included in the TISS score. However, each TISS item was assessed anew as to its relative value in the intraoperative environment and, if necessary, its scoring value was readjusted. The draft scoring system was then reviewed by a group of attending anesthesiologists, surgeons, and intensivists who commented on the choice of interventions and their point values. To help in their assessment, they were asked to consider the scores for a group of commonly performed operations such as appendectomy, abdominal hysterectomy, cholecystectomy, coronary artery bypass, open repair of a hip fracture, and femoral-popliteal artery bypass. Disagreements were discussed in group meetings so as to reach a consensus. Preliminary drafts were then piloted with actual patients, and the scores were reviewed by the group. A number of draft versions were necessary before the group was satisfied that the score met the development objectives outlined previously.

After approval from the IRB, data were gathered from patients upon their leaving the operating room by querying the anesthesiologist and reviewing the anesthetic record and operative notes. Two groups of patients were studied. Group 1 included random patients entering the PACU (recovery room) for short postoperative stays on randomly chosen days. Group 2 included patients admitted to the surgical, cardiothoracic, and neurosurgical ICUs or those with extended (>12 h) PACU stays (the hospital does not have a surgical intermediate care unit, so such patients remain in the PACU for 12–24 h, at which time they are transferred to either the floor or the ICU). Decisions as to ICU or extended PACU admission were made by the responsible anesthesiologist, independently of the study, in consultation with the responsible surgeon. Extended PACU admissions were for patients deemed to require additional monitoring and care for the first postoperative night. Arterial line and central venous pressure, but not pulmonary artery (PA) catheter, monitoring were permitted in the PACU. Mechanically ventilated patients were accommodated in the PACU if the initial plan was to attempt extubation by the morning after surgery. Patients receiving continuous IV infusions of vasoactive drugs were admitted directly to the ICU.

The data collected about each patient included ASA physical status classification, age, surgical diagnosis, operative procedure, and surgical complexity (Table 1). The I-TIS score was also calculated. ASA anesthesia base relative value units alone and modified for ASA physical status and emergency surgery were also assigned (5). The mean I-TIS scores for each surgical service were compared with published anesthesia drug and supply cost data (6).

Table 1

Table 1

Correlations between the I-TIS (continuous variable) scores and ASA physical status and relative value units (categorical/ordinal variables) were examined with Spearman correlation coefficients. The correlation between the I-TIS and the surgical complexity classification (logarithmic variable) was determined by using a logarithmic correlation coefficient. Comparisons between subgroups of the I-TIS data were performed by using analysis of variance with the Tukey post hoc test. The variation of the I-TIS scores for various operations was assessed by calculating the coefficient of variation (sd/mean). The mean I-TIS scores for each surgical service were compared with published anesthesia drug and supply cost data (6) by using Pearson correlation coefficients. Interrater reliability was assessed by using κ statistics.

Additional analysis was performed to determine whether the score differentiated between patients having short postoperative PACU stays and those receiving more intense postoperative care (extended PACU stay or ICU admission). The odds ratio for the I-TIS score’s ability to predict postoperative ICU or extended PACU admission was calculated by using a 2 × 2 table after graphical analysis of the data. Logistic regression models were used to confirm this graphical and odds ratio analysis. One model examined the I-TIS as a continuous variable, whereas another examined it as a binary categorical variable (I-TIS <20 and ≥20).

Back to Top | Article Outline

Results

Seventy-eight interventions were incorporated into the I-TIS (Table 2). Each intervention received a value of 1–4 points, with a maximum possible score of 202.

Table 2

Table 2

Table 2

Table 2

It was recognized that in contemporary practice, all patients undergoing surgery are physiologically monitored and usually undergo some therapeutic or other interventions (i.e., no patient has zero interventions). Therefore, the I-TIS scores basic monitoring and interventions. For example, the insertion of a single IV catheter and the routine monitoring combination of electrocardiogram, noninvasive blood pressure, temperature, and arterial oxygen saturation were each given a scoring value of 1.

Approximately a quarter of the I-TIS interventions were adapted from the TISS score. Often the I-TIS scoring value differed from the corresponding TISS value. For example, stat blood tests were upgraded from one to two or three points because their intraoperative use represents a more intense intervention than their almost routine use during ICU care. Alternately, arterial line (TISS, three points) and PA (TISS, four points) were downgraded by the developers of the score to two and three points, respectively, because their use in the operating room represents less intense use than in an ICU; i.e., in the ICU, arterial lines and PA catheters are usually inserted because of existing critical illness, whereas in the operating room they are usually used to monitor for potential problems. However, platelet transfusion remained at four points because in the operating room, the transfusion of platelets is almost always associated with massive hemorrhage and blood transfusion.

I-TIS scoring was performed on 750 patients. It was observed that an experienced individual was able to score a patient in <5 min. Twenty randomly selected patients were scored by the same three individuals, resulting in a κ statistic of 0.94. Group 1 included 307 patients (age, 43 ± 23 yr [mean ± sd]; range, 0.1–89 yr) who had brief PACU stays. One-hundred-thirty-six (44%) of these patients underwent operations classified as having a surgical complexity score of 1 (Table 1), and 77 (25%) operations had a score of 2. The mean I-TIS score of Group 1 was 7.3 ± 5.0 (median, 5). Group 2 consisted of 443 patients (age, 56 ± 21 yr; range, 0.04–95 yr) who had extended (>12-h) PACU stays or were admitted to the surgical, neurosurgical, pediatric, or cardiothoracic ICUs. One-hundred-eighty-eight patients (46%) had operative complexity scores of 8, and 157 (38%) had complexity scores of 4. The mean I-TIS of Group 2 was 25.2 ± 12.4 (median, 25;P < 0.001 versus Group 1). Figure 1 shows a substantial difference between the distribution patterns of the I-TIS scores of the two groups. The I-TIS scores for patients who had short PACU stays were lower than those of patients with extended PACU stays and those who were admitted to the surgical ICU (Table 3).

Figure 1

Figure 1

Table 3

Table 3

The I-TIS scores for specific surgical procedures are found in Table 4. The highest scores were with liver transplantation. The variability of the scores within operations was moderate, with coefficients of variation of approximately 30%.

Table 4

Table 4

The association of the I-TIS score with other scoring systems was examined. The correlation coefficient of the ASA physical status classification with the I-TIS was rs = 0.46 (P < 0.01;Table 4). However, the correlation between the surgical complexity score and the I-TIS was r = 0.77 (P < 0.001;Table 5), whereas that between the I-TIS and base relative value units was rs = 0.75 (P < 0.001;Fig. 2). When the base relative value units were modified for ASA classification and emergency surgery, the correlation was rs = 0.79 (P < 0.001;Fig. 3). The mean I-TIS scores for each surgical service were compared with published anesthesia drug and supply cost data (6), and the correlation was r = 0.91 (P < 0.001).

Table 5

Table 5

Figure 2

Figure 2

Figure 3

Figure 3

An I-TIS score of >25 was a better indicator (odds ratio, 187) than a score of >15 (odds ratio, 32) of patients receiving postoperative ICU or extended PACU care. Logistic regression modeling (ρ2m = 0.32) confirmed these results, with an odds ratio of 35.5 (95% confidence limits, 20.4–61.7) for an I-TISS of ≥20. The model developed by using logistical regression predicted that 97.5% of patients with an I-TIS score of >28 would be receiving postoperative ICU or extended PACU care (Fig. 4).

Figure 4

Figure 4

Back to Top | Article Outline

Discussion

The I-TIS was developed to quantify the extent of physiological monitoring, biochemical diagnostic testing, and therapeutic and other (nonsurgical) interventions performed during surgery. Evaluation of the I-TIS in the clinical arena demonstrated that it met these requirements. I-TIS correlated well with surgical complexity and was able to gauge the intensity of care during various surgical procedures (Table 4). There are no equivalent scoring systems to quantify the extent of intraoperative interventions. The I-TIS scoring system is thus unique in that it fills an empty niche in the evaluation of perioperative care by providing a quantitative system with which to gauge the intensity of intraoperative resource utilization.

There are two major types of scoring systems. Categorical or qualitative systems use predetermined values to classify patients into categories, e.g., the ASA physical status classification. The other type of system is designed to quantify what actually occurs to individual patients. Both the TISS and I-TIS systems are examples of the latter type of scores. The concept of quantitative therapeutic intervention scoring was developed by Cullen et al. (2) for assessing the intensity of ICU care. This scoring idea has proven to be useful for a variety of reasons. For example, the TISS score is a good reflection of costs incurred in caring for ICU patients (3,7,8).

Scoring systems used in medicine have a number of functions, among them qualifying and quantifying issues such as the severity of illness, the extent of care, cancer spread, and the risks of complications. The I-TIS score, like the TISS score, was designed to be used in health services research to quantify the intensity of the nonsurgical portion of intraoperative care. Therefore, an integral feature of the I-TIS score’s design is an ability to examine intraoperative resource utilization. Practically, I-TIS can aid in cost assessment because many of the components of the I-TIS (e.g., arterial line and PA catheter) involve the use of supplies whose costs can be captured. Some components involve the use of capital equipment (e.g., transesophageal echocardiography and somatosensory evoked potential monitoring). However, because of the nature of anesthesiology practice (one anesthesiologist per operating room), variability in personnel costs are less of a factor than in ICUs. However, cases with high I-TIS scores likely could benefit from anesthesiology technician support, and very complicated situations may require the presence of additional anesthesiologists. The presence of additional anesthesiologists is a variable of the I-TIS. The ability of the I-TIS to reflect anesthesia drug and supply costs was examined by comparing the I-TIS score per surgical specialty with published cost data from a single United States (US) institution (6). The correlation was 0.91, showing the overall ability of the I-TISS score to reflect intraoperative anesthesia costs. Other comparisons of I-TIS scores with cost data should be performed to determine whether this correlation is reproducible.

The validity of the I-TIS was examined in a variety of ways. The initial evaluation was to compare it with measures of surgical complexity. This permitted testing of the hypothesis that the complexity of the operation correlates closely with intraoperative therapeutic intensity. This hypothesis was proven to be true, with a correlation coefficient of 0.77. This level of correlation can be attributed to a number of factors. Because of the dearth of surgical complexity scores and the need to chose one that was simple to apply, we used a surgical complexity score that had only four levels of surgical complexity. This resulted in our comparing a 4-level with an 80-item score or, more precisely, a categorical with a quantitative score. This undoubtedly influenced the exactness of the correlation coefficient. Another reason for the correlation coefficient of 0.77 is that for each operation, a range of I-TIS scores was observed. This is not unexpected because the need to monitor and contend with preexisting diseases and intraoperative complications increases the intensity and number of interventions performed during surgery.

The I-TIS scores were also compared with the ASA physical status classification. This was originally designed to facilitate interinstitutional comparisons of mortality and morbidity but was subsequently found to also reflect perioperative risk. This comparison facilitated the testing of the hypothesis that preoperative risk scoring does not correlate well with intraoperative therapeutic intensity. This is what was in fact observed with our data set and can be explained by the fact that the ASA classification measures the severity of preexisting and underlying diseases. Not infrequently, patients with minimal or no preexisting and underlying diseases underwent complex operations. Prime examples were healthy teenage patients (ASA I) undergoing spinal fusion for scoliosis or orthognathic surgery for prognathia (Table 4). Alternately, patients with high ASA classifications (ASA III or IV) underwent minor surgical procedures. For instance, a patient with acute leukemia (ASA IV) underwent Hickman catheter insertion for chemotherapy administration.

A further evaluation of the I-TIS was to determine how well it correlated with another measure of intraoperative nonsurgical intervention. The ASA Relative Value Guide (a guide for anesthesia values) is a group of codes that reflect the relative complexity (“value”) of anesthesia care. These codes are used in calculating the bill for anesthesia services in the US. There was a correlation coefficient of 0.75 between the base relative value units (RVU) and the I-TIS (Fig. 2), which further increased when the base units were modified for ASA classification and emergency surgery (r = 0.79;Fig. 3). These levels of correlation were not unexpected, because the RVU is designed to measure anesthetic care intensity. A better correlation was not achieved probably because RVUs are a fixed, predetermined value, whereas the I-TIS is more variable because it quantifies actual patient care (Fig. 5).

Figure 5

Figure 5

The data set used to evaluate the I-TIS scores included a group of patients undergoing relatively simple operations and another group undergoing more complex operations, who received special care (extended recovery room stay or ICU admission) after surgery. Figure 1 shows that there was a distinct difference in the distributions of I-TIS scores between the groups. Furthermore, there were significant differences between the I-TIS scores depending on the location of postoperative care (Table 3). These results were expected because it is intuitive that patients tracked to receive postoperative ICU or prolonged PACU care are those receiving intensive intervention during surgery and thus will have the higher I-TIS scores. Similarly, the highest I-TIS scores were seen among liver transplantations followed by cardiac procedures. The variability of the scores for a given procedures was also examined, and many were found to have coefficients of variation of <30% (Table 5), demonstrating that at least in one institution, there was no undue variability. This was not unanticipated, because there is often “institutional” tradition as how to care for patients. However, this was not always true. For example, there was wider variability among patients undergoing radical prostatectomy, because it was performed under general and/or regional anesthesia.

Another use for the I-TIS scoring system that was examined was its ability to differentiate between patients having short postoperative PACU stays and those receiving more intense postoperative care (extended PACU stay or ICU admission). Although this was not the primary reason for developing the I-TIS, a post hoc analysis was performed to determine whether I-TIS could differentiate between the various postoperative care venues. It is the nature of medicine that the higher scores of many scoring systems are able to predict greater intensity of care. This was also true of the I-TIS score, with scores >15 and especially those >25 being highly predictive of the postoperative use of intensive or intermediate care beds. This is intuitive, because it is expected that most patients who receive intensive care during surgery will continue to require a high level of care postsurgery.

A major limitation of this evaluation of the I-TIS score was that it was performed in a single institution. However, it should be realized that the I-TIS facilitates interinstitutional comparisons. For instance, the scoring for coronary artery bypass surgery may vary among institutions. In some institutions, the routine is to place only a central venous catheter (as at our institution), whereas in others PA catheters and transesophageal echocardiography are the routine. This degree of monitoring would add an additional four points to the I-TIS score, thus reflecting more monitoring.

In summary, the I-TIS is able to measure the intensity of physiological/biochemical monitoring interventions and therapeutic support patients receive during surgery. It is a useful adjunct to surgical complexity scores because it reflects the actual physiological/biochemical interventions and monitoring provided to patients by anesthesiologists and others. These variables are not measured in a true quantitative fashion by currently available scores. The I-TIS should prove to be a useful tool to compare the intensity of intraoperative care afforded to individuals and groups of patients.

Back to Top | Article Outline

References

1. Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg 1991; 78: 355–60.
2. Cullen DJ, Civetta JM, Briggs BA, Ferrara LC. Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Crit Care Med 1974; 2: 57–60.
3. Keene AR, Cullen DJ. Therapeutic intervention scoring system: update 1983. Crit Care Med 1983; 11: 1–3.
4. Clermont G, Angus DC, Linde-Zwirble WT, et al. Measuring resource use in the ICU with computerized therapeutic intervention scoring system-based data. Chest 1998; 113: 434–42.
5. Relative value guide. Park Ridge, IL: American Society of Anesthesiologists, 2000.
6. Dexter F, Lubarsky DA, Gilbert BC, Thompson C. A method to compare costs of drugs and supplies among anesthesia providers. Anesthesiology 1998; 88: 1350–6.
7. Alzola C, Lynn J, Wagner D, Wu AW. Length of stay and therapeutic intervention allow estimation of in-hospital resource use independent of site and inflation. J Am Geriatr Soc 2000; 48: S162–7.
8. Dickie H, Vendio A, Dundas R, et al. Relationship between TISS and ICU cost. Intensive Care Med 1998; 24: 1009–17.
© 2002 International Anesthesia Research Society