Share this article on:

Multicenter Assessment of the Iowa Satisfaction with Anesthesia Scale, an Instrument that Measures Patient Satisfaction with Monitored Anesthesia Care

Dexter, Franklin MD, PhD*; Candiotti, Keith A. MD

doi: 10.1213/ANE.0b013e318217f804
Economics, Education, and Policy: Research Reports

BACKGROUND: The Iowa Satisfaction with Anesthesia Scale (ISAS) is a questionnaire that measures patient satisfaction with monitored anesthesia care. Previous assessments of the reliability and validity of this tool have been conducted in 2 separate single-center studies. Recently, the questionnaire was used in a 24-center, 315-patient, placebo-controlled trial of dexmedetomidine. We analyzed the data from these patients to provide anesthesiologists and statisticians designing multicenter clinical trials the information needed to use the ISAS as a primary study end point.

METHODS: Trial variables were ISAS score, treatment group, center, time of administration of the instrument, and potential covariates of age, race, gender, type of surgery, and ASA physical status.

RESULTS: The ISAS could be scored for 98% of patients (95% confidence interval [CI], >97%). Whereas 73% of patients responded with the minimum of −3 or the maximum of +3 for the single question “I was satisfied with my anesthetic care,” only 14% did so for the ISAS (i.e., the instrument differentiated among patients). The internal consistency (Cronbach α) equaled 0.84 (95% CI, 0.79–0.87). Because there was no correlation between the ISAS score and time of completion of the questionnaire the next day (Kendall τb, 0.01; 95% CI, −0.06 to 0.09), the ISAS likely is valid when administered by telephone the next day, not only when administered in person upon discharge as shown in its original development. Effect sizes of 0.48 have been detected in several trials, suggesting that typically appropriate sample sizes are 70 to 93 patients per group. There was significant heterogeneity in ISAS scores among study centers (P < 0.0001), making stratification by center very important, and planned analysis with center as a covariate reasonable. Type of surgery might be an additional appropriate covariate depending on the study.

CONCLUSIONS: The ISAS is reliable, valid, and useful over the conditions suitable for use as a primary study end point in multicenter clinical trials.

Published ahead of print April 25, 2011 Supplemental Digital Content is available in the text.

From the *Department of Anesthesia, University of Iowa, Iowa City, Iowa; and Department of Anesthesiology, Perioperative Medicine, and Pain Management, University of Miami, Miami, Florida.

Supported by departmental funds.

Address correspondence and reprint requests to Franklin Dexter, MD, PhD, Department of Anesthesia, University of Iowa, Anesthesia 6JCP, Iowa City, IA 52242. Address e-mail to Franklin-Dexter@UIowa.edu or www.FranklinDexter.net.

Accepted February 24, 2011

Published ahead of print April 25, 2011

The Iowa Satisfaction with Anesthesia Scale (ISAS) is a questionnaire that measures patient satisfaction with monitored anesthesia care (MAC) (Table 1).1 Completion of the 11 items takes 4 to 5 minutes.1 The ISAS literally measures satisfaction with the anesthetic, unlike assessments of satisfaction with the perioperative period that are the suitable options for patients who are unconscious for some or all of the period that the patient considers to be the anesthetic (i.e., for patients undergoing general anesthesia).1

Table 1

Table 1

The ISAS score is the mean of responses to all 11 questions. The score can range from −3 to +3 and is quantitative (i.e., not for ranked analysis).2 Scores of −3 and +3 correspond to “disagree very much” and “agree very much,” respectively (Table 2). The mean was 2.1 U among 80 patients at the University of Iowa's Ambulatory Surgery Center.1 We refer to these1 as the “original” results.

There has been some additional assessment of the reliability, validity, and usefulness of the ISAS since its original1 publication. Three reviews evaluated the process of its development and concluded that it is psychometrically valid.3 5 Fung et al.6 confirmed its reliability and validity for cataract surgery patients undergoing MAC at a community hospital. Subsequent randomized trials used the ISAS to show greater patient satisfaction with sub-Tenon's block for cataract surgery relative to topical anesthesia7,8 and retrobulbar block.8 In addition, an observational study of 306 patients found that a surgeon's use of intraoperative pilocarpine drops caused reduced satisfaction (P < 0.001) because of pain.9

Recently, Candiotti et al.10 used the ISAS in a study of 310 patients undergoing MAC for multiple procedures at 24 centers. Patients receiving dexmedetomidine, 0.5 μg/kg or 1.0 μg/kg loading plus infusion, had significantly greater patient satisfaction than patients receiving placebo (P = 0.001). The mean ± SD of the ISAS scores was 2.0 ± 1.0 for the dexmedetomidine groups versus 1.4 ± 1.4 for the placebo group. The mean difference between groups was 0.6 U relative to a pooled SD of 1.3 U (i.e., effect size was 0.48).a In this subsequent study, we analyzed the data from these patients to evaluate the reliability and validity of the ISAS for use as a primary study end point for multicenter clinical trials.

Back to Top | Article Outline

METHODS

Key methodological features of the clinical trial are repeated with nearly identical wording from that of the original article (see Conflicts of Interest for explanation).10 The trial was a randomized, double-blind, placebo-controlled trial of dexmedetomidine at 24 investigational sites in the United States. The protocol was approved by the IRB of each center. All 315 patients provided written informed consent. Patients 18 years or older scheduled for elective surgeries and procedures performed in an operating room or procedure room under MAC plus a local anesthetic block were eligible for enrollment. Patients were randomized in a 2:2:1 ratio to dexmedetomidine 0.5 μg/kg, 1.0 μg/kg, or saline placebo loading dose administered over 10 minutes. Patients in the dexmedetomidine groups subsequently received a maintenance infusion beginning at a rate of 0.6 μg/kg/h, whereas patients in the placebo arm received a volumetrically equivalent infusion of saline. Infusions were titrated to achieve the desired sedation level. Patients with an Observer's Assessment of Alertness/Sedation Scale score >4 and not responding to infusion titration received 0.5 mg IV midazolam, repeated as necessary. Approximately 24 hours after the end of the procedure (see Results), the ISAS was administered by telephone unless the patient had been admitted in which event the patient was interviewed in person.

The ISAS was a secondary study end point (i.e., not the basis for study sample size). Analyses for ISAS validation are described sequentially in the Results. Reported 1-sided 95% confidence limits (CLs) for percentages were calculated using the Blyth-Still-Casella method (StatXact-9; Cytel Software Corp., Cambridge, MA). Confidence intervals (CIs) for Kendall τb are asymptotic (StatXact-9). The exact P value for differences among centers in the distribution of sample sizes among treatment groups was tested using the χ2 test (StatXact-9). CIs for Cronbach α were calculated using 2000 bootstrap samples interpreted using the percentile method (Systat-12; Systat Software, Inc., San Jose, CA). Variance components analysis was performed using restricted maximum likelihood estimation (Systat-12).

Back to Top | Article Outline

RESULTS

The ISAS score is calculated only for completed questionnaires.1 All questions were answered by 98% of patients (n = 310 of 315; lower 95% CL, 97%). Among the 5 patients not completing the instrument, 1 to 3 of the 11 questions were skipped.

The ISAS would be useful only if its responses differ among patients. Before scoring is done, responses for “negative” statements are reversed (e.g., a response of −3 for “I itched” is changed to +3) (Table 2). Once reversed, a +3 is the most satisfied response for all questions, and −3 is the least. Each patient's ISAS score is the mean response of the patient's responses to the 11 questions. The 310 patients had 47 different scores. No patient had an ISAS score of −3, matching the original findings from Iowa.1 The most common score was +3, which was obtained for 14% of patients, nearly the same as the 15% rate found originally.1 In comparison, the single question “I was satisfied with my anesthetic care” was not as useful because it did not adequately differentiate among patients, because 73% of patients responded with −3 or +3.

The ISAS would be reliable only if it has a high internal consistency because it is measuring a singular construct, satisfaction with the anesthetic.b Cronbach αb ranges from 0 to 1, with an acceptable value being at least 0.6, but ideally greater. The Cronbach α equaled 0.84 (95% CI, 0.79–0.87), not significantly different from the 0.80 value obtained originally.1

The ISAS would be unnecessarily long if similar questions had nearly identical responses. Originally, 42% of patients gave different responses to the similar questions: “I felt pain” and “I felt pain during surgery.”1 These questions again measured different aspects of patient satisfaction, based on 39% of patients giving different responses to the 2 questions. Among the 120 patients giving different responses, 61% had responses for the 2 questions differing by ≥2 U. Most patients reported more “pain” than “pain during surgery” (62%; 95% CL, 54%; 2-sided P = 0.014). In addition, responses to the question “I hurt” differed for 52% of patients from their responses to “I felt pain” or “I felt pain during surgery.” These findings show that all 3 questions should be included as they are measuring different contributions to the common dimension of satisfaction with the anesthetic.

The validity of the ISAS depends in part on its positive correlation to other measures of satisfaction. Each patient's mean response to 10 of the 11 questions was compared with the answer for the remaining question: “I was satisfied with my anesthetic care.” The Kendall τb correlation was 0.47 (95% CI, 0.40–0.53; P < 0.0001), showing evidence of convergent validity. The correlation is similar to the 0.41 measured originally.1

Investigators would have more flexibility in using the ISAS if it does not have to be administered in person, but can instead be by telephone. In the original study, the ISAS was administered in person when the patient was ready to leave the phase II postanesthesia care unit.1 In contrast, during the dexmedetomidine trial, the ISAS was completed by telephone (see Methods) 25 ± 5 hours after the end of the study infusion, dexmedetomidine or saline. The 25th and 75th percentiles were 23 and 26 hours, respectively. There was no correlation between the ISAS score during the dexmedetomidine trial and the hour of completion of the questionnaire in the postoperative period (Kendall τb, 0.01; 95% CI, −0.06 to 0.09; 2-sided P = 0.66). Thus, the ISAS is reliable and valid when administered by telephone the day after surgery, not only in person upon discharge, as shown originally.1

Multicenter trial design depends on knowing whether stratification by center is needed. Because the anesthetic is influenced by the anesthesia providers and the providers differ among centers, heterogeneity among centers was expected and observed. Analysis of variance was performed among the 3 treatment groups comparing ISAS scores. The F ratio equaled 8.56, corresponding to an overall P = 0.0002. Next, the variable “center” was added to the statistical model as a random intercept (i.e., each center's mean ISAS score differs). There was significant heterogeneity in ISAS scores among centers (P < 0.0001). Controlling for the center increased the F ratio for treatment effect by 14%. These results show that when designing a multicenter trial using the ISAS, stratifying by center can either modestly reduce the necessary sample size or result in smaller P values. The results also show the major importance of stratifying by center, as was done in the dexmedetomidine trial,10 which had no significant unbalance in the (sample size) distribution of treatment groups among centers (χ2 P = 0.98).

Measuring patient satisfaction with anesthesia would be useful if the magnitude of differences in satisfaction among centers is comparable to differences modifiable by choice of the anesthetic. The ISAS detected centers with unusually low or high satisfaction. The total sample sizes at each of the 24 centers varied substantially (range, 2–42 patients; coefficient of variation, 97%). Consequently, although the standard deviation of the random intercept equaled 0.4 U, suggesting that there may be different mean satisfactions among multiple centers, the standard error was relatively large (0.3 U) resulting in P < 0.01 for 3 of the 24 centers and P > 0.10 for the remainder. The 3 significant mean differences from the overall mean were −0.7, −0.7, and +0.5. The effect sizes, calculated by taking the ratio of the mean differences to the square root of the residual error variance, were 0.74, 0.71, and 0.49, respectively. The first 2 exceed and the third is comparable to the 0.48 effect size for the difference between placebo and dexmedetomidine groups (see Introductiona).

Clinical trial design relies on the statistical power analysis, which can be based on the 0.48 effect size. Using a type I error rate of 0.05, 80% power is achieved with a sample size of 70 patients per group and 90% power with 93 patients per group.

Sample sizes could potentially be reduced if important covariates could be controlled. As above, a linear model was fit to ISAS scores using “treatment group” as a fixed effect and “center” as a random effect. The incremental reduction in sum of square residuals was tested with the addition of the available covariates of age, race, gender, type of surgery,10 and the ASA physical status. The largest F ratio and smallest P value were for type of surgery (P = 0.0003), and once added the other covariates were nonsignificant (P > 0.10). These results are consistent with the narrow pooled standard deviation of 0.22 U and relatively large mean ISAS scores of 0.33 U of Fung et al.9 when studying only cataract surgery patients (see Introduction). However, whereas “center” is prespecified and known without error, the effect of adding type of procedure was highly sensitive to the classification method. Because of that sensitivity, we used the classification of the 8 types of procedures in the tables of the dexmedetomidine trial article (e.g., “ophthalmology” and “breast biopsies”).10 Because heterogeneity of ISAS scores is dependent on pain (Table 1),9 attempts to achieve some balance in procedures among centers based on pain is likely to be warranted.

Back to Top | Article Outline

DISCUSSION

The ISAS has previously not been appropriate for use as a primary study end point for multicenter trials, because prior assessment has been for single centers. In our study, we demonstrated that the ISAS is reliable, valid, and useful over a far wider range of situations than evaluated originally at the University of Iowa's Ambulatory Surgery Center. We showed that trials and surveys should generally have 70 to 95 patients per group. However, because of heterogeneity of sample size and type of procedures among centers, we could not definitively determine whether type of procedure should be planned as a covariate. This limitation resulted from our study not being a planned analysis of the dexmedetomidine trial data (see Conflicts of Interest).

Performing clinical trials10 and observational surveys9 to increase patient satisfaction with MAC is important because patients value the factors that contribute to their satisfaction. The outcomes from the ISAS are amenable to pharmacological therapy (e.g., “I felt pain” and “I threw up or felt like throwing up”). The high internal consistencyb of the ISAS (0.84) shows that these measures of outcome are an integral part of the single dimension that patients consider to compose the anesthetic. These factors are different from those that reflect patient safety. In the closed claims analysis of MAC, several factors affecting patient satisfaction (itching, nausea, and pain) were not mentioned as a source of claims.12 Vomiting was included once in a list of different causes of medication-associated claims (i.e., accounted for <1% of claims). Some of the factors that lower the ISAS score (i.e., pain and vomiting) are outcomes that patients are willing to pay to avoid: US $35 for analgesics and $17 for antiemetics,13 $50 for analgesics and $33 for antiemetics,14 €65 or €67 for antiemetics,15 and $56 for antiemetics.16 Willingness to pay is larger among patients who have had surgery and have experienced nausea and vomiting.15,16 Patients are also willing to travel farther to obtain their desired characteristics of (cataract) surgery under MAC.17 Thus, anesthesia groups should apply c the trial results and thereby increase their patients' satisfaction.

The ISAS is not suitable for assessing how patients perceive an ambulatory surgery center, just the anesthetic itself. Deliberately excluded from the questionnaire were dimensions of perioperative care that are important to patients but unrelated to choice of anesthetic agent.1 For example, questions about communication19 were not included because the anesthetic agents cause some patients to have amnesia for parts of their procedure. The ISAS also does not assess perception of anesthesia services.20 The initial set of questions considered for the ISAS included questions for the dimension of “respect for patients' values, preferences, and expressed needs,” including (a) “I felt alone,” (b) “No one was helping me,” (c) “No one cared about me,” and (d) “I liked my anesthetist” (see Table 1 of the original article).1 This dimension of care contributed little to distinguishing the patients' satisfaction with their anesthetic, based on >90% of patients disagreeing very much with questions a to c and agreeing very much with question d.1

Back to Top | Article Outline

RECUSE NOTE

Franklin Dexter is the Statistical Editor and Section Editor for Economics, Education, and Policy for the Journal. This manuscript was handled by Steve Shafer, Editor-in-Chief, and Dr. Dexter was not involved in any way with the editorial process or decision.

Back to Top | Article Outline

DISCLOSURE

Name: Franklin Dexter, MD, PhD.

Role: This author helped design the study, conduct the study, and analyze the data.

Conflicts: Franklin Dexter works for University of Iowa. See the subsequent Acknowledgments and Conflicts of Interest section.

Attestation: Franklin Dexter has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Name: Keith A. Candiotti, MD.

Role: This author helped design the study, conduct the study, and write the manuscript.

Conflicts: Keith A. Candiotti consults for Hospira. See the subsequent Acknowledgments and Conflicts of Interest section.

Attestation: Keith A. Candiotti has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Back to Top | Article Outline

ACKNOWLEDGMENTS AND CONFLICTS OF INTEREST

This study was performed in an arrangement between Hospira and the University of Iowa prompted by Hospira's use10 of the ISAS and publication10 of the instrument. The authors appreciate Hospira and the University of Iowa Research Foundation negotiating the agreement, including the efforts of Brian R. Woodworth, Vice President, Patents and Trademarks, Hospira, Inc.; Robert N. Bilkovski, Associate Medical Director, Global Clinical Research & Development, Hospira, Inc.; and Shannon Sheehan, Senior Licensing Associate, University of Iowa Research Foundation. Hospira, Inc. provided the data for our study, but otherwise the work was supported by the authors' departments. Hospira physicians and scientists made recommendations about the study design as part of Hospira and the University of Iowa's agreement and reviewed the manuscript once written. They were not involved in the conduct of the study, analysis or interpretation of the data, or preparation of the manuscript. Dr. Candiotti previously performed research funded by Hospira and is currently a paid consultant. The ISAS is copyright Franklin Dexter and the University of Iowa Research Foundation. Dr. Dexter receives no funds personally other than his salary from the University of Iowa, including no travel expenses or honoraria, and has tenure with no incentive program.

a A negative study with the same effect size of 48 was also published.11 For third molar extraction with target-controlled propofol infusion for conscious sedation, n = 24 patients received boluses of fentanyl and n = 16 received alfentanil. ISAS scores were fentanyl 1.9 ± 0.6 versus alfentanil 2.2 ± 0.7. The difference between groups was 0.3 U relative to a pooled standard deviation of 0.7 U. The statistical power was low, ≅30%.
Cited Here...

b Cronbach α involves comparison of the variance of the sum of all responses (total score) with the variances of the responses to individual questions. Mathematically, when responses to questions are not correlated, the variance of the total score will be equal to the sum of variances for each question that comprises the total score. As the responses to questions become more and more correlated, the variance of the total score will increase.1.
Cited Here...

Back to Top | Article Outline

REFERENCES

1. Dexter F, Aker J, Wright WA. Development of a measure of patient satisfaction with monitored anesthesia care: the Iowa Satisfaction with Anesthesia Scale. Anesthesiology 1997;87:865–73
2. Dexter F, Chestnut D. Analysis of statistical tests to compare visual analog scale measurements among groups. Anesthesiology 1995;82:896–902
3. Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: a review of current methodology. Anesth Analg 1998;87:1089–98
4. Bell DM, Halliburton JR, Preston JC. An evaluation of anesthesia patient satisfaction instruments. AANA J 2004;72:211–7
5. Chanthong P, Abrishami A, Wong J, Herrera F, Chung F. Systematic review of questionnaires measuring patient satisfaction in ambulatory anesthesia. Anesthesiology 2009;110:1061–7
6. Fung D, Cohen M, Stewart S, Davies A. Can the Iowa Satisfaction with Anesthesia Scale be used to measure patient satisfaction with cataract care under topical local anesthesia and monitored sedation at a community hospital? Anesth Analg 2005;100:1637–43
7. Rüschen H, Celaschi D, Bunce C, Carr C. Randomised controlled trial of sub-Tenon's block versus topical anaesthesia for cataract surgery: a comparison of patient satisfaction. Br J Ophthalmol 2005;89:291–3
8. Ryu JH, Kim M, Bahk JH, Do SH, Cheong IY, Kim YC. A comparison of retrobulbar block, sub-Tenon block, and topical anesthesia during cataract surgery. Eur J Ophthalmol 2009;19:240–6
9. Fung D, Cohen MM, Stewart S, Davies A. What determines patient satisfaction with cataract care under topical local anesthesia and monitored sedation in a community hospital setting? Anesth Analg 2005;100:1644–50
10. Candiotti KA, Bergese SD, Bokesch PM, Feldman MA, Wisemandle W, Bekker AY MAC Study Group. Monitored anesthesia care with dexmedetomidine: a prospective, randomized, double-blind, multicenter trial. Anesth Analg 2010;110:47–56
11. Kwak HJ, Kim JY, Kwak YL, Park WS, Lee KC. Comparison of a bolus of fentanyl with an infusion of alfentanil during target-controlled propofol infusion in third molar extraction under conscious sedation. J Oral Maxillofac Surg 2006;64:1577–82
    12. Bhananker SM, Posner KL, Cheney FW, Caplan RA, Lee LA, Domino KB. Injury and liability associated with monitored anesthesia care: a closed claims analysis. Anesthesiology 2006;104:228–34
    13. van den Bosch JE, Bonsel GJ, Moons KG, Kalkman CJ. Effect of postoperative experiences on willingness to pay to avoid postoperative pain, nausea, and vomiting. Anesthesiology 2006;104:1033–9
    14. Gan TJ, Ing RJ, de L Dear G, Wright D, El-Moalem HE, Lubarsky DA. How much are patients willing to pay to avoid intraoperative awareness? J Clin Anesth 2003;15:108–12
    15. Kerger H, Turan A, Kredel M, Stuckert U, Alsip N, Gan TJ, Apfel CC. Patients' willingness to pay for anti-emetic treatment. Acta Anaesthesiol Scand 2007;51:38–43
    16. Gan TJ, Sloan F, de L Dear G, El-Moalem HE, Lubarsky DA. How much are patients willing to pay to avoid postoperative nausea and vomiting? Anesth Analg 2001;92:393–400
    17. Dexter F, Birchansky L, Bernstein JM, Wachtel RE. Case scheduling preferences of one surgeon's cataract surgery patients. Anesth Analg 2009;108:579–82
    18. Dexter F. Application of cost-utility and quality-adjusted life years analyses to monitored anesthesia care for sedation only. J Clin Anesth 1996;8:286–8
      19. Capuzzo M, Gilli G, Paparella L, Gritti G, Gambi D, Bianconi M, Giunta F, Buccoliero C, Alvisi R. Factors predictive of patient satisfaction with anesthesia. Anesth Analg 2007;105:435–42
      20. Le May S, Hardy JF, Taillefer MC, Dupuis G. Patient satisfaction with anesthesia services. Can J Anaesth 2001;48:153–61
      © 2011 International Anesthesia Research Society