To the Editor:
I would like to clarify a number of points in response to the recent article by Higgins and Purvis (March 2002 1) concerning the usefulness of the BASIS-32® in evaluating program level outcomes.
- BASIS-32® is copyrighted by McLean Hospital, which instituted an annual site license fee in 1998. This decision was made by the hospital, not by the author of the instrument. These licensing fees are used to cover the costs of printing, mailing and responding to requests for information about the instrument
- Higgins and Purvis compare BASIS-32® scores for 16,909 outpatients with those for 37 long-term inpatients. However, they do not indicate how long the 37 inpatients had been hospitalized before they were assessed. If these individuals had been hospitalized for several years and were living within the sheltered environment of the hospital, they may well have been experiencing less acute symptom and problem difficulty than outpatients experiencing an acute episode.
- The authors acknowledge that the protocol used to administer the instrument was not consistent with the standard, recommended outcomes protocol that assesses clients at the beginning of a treatment episode and at a later time point. Given the methodology used, they are not justified in concluding that the instrument is not useful for program outcome assessment.Numerous mental health treatment programs throughout the United States, as well as programs in other areas of the world, have found the BASIS-® useful for outcome assessment. Additionally, at least five JCAHO-approved performance measurement systems use the BASIS-32® as a quality indicator for JCAHO’s ORYX quality measurement initiative. Combined, these performance measurement systems have reported BASIS-32® results for more than 100 inpatient mental health programs during the past 2 years. The McLean BASIS-32 plus Performance Measurement System alone has a database of 15,226 cases with BASIS-32® scores at admission and discharge from 26 mental health facilities over the last 2 years. Results from these cases show significant improvement from admission to discharge with an average change score of approximately one full point for the overall mean score.
- The authors note on p. 127 that it was unrealistic to implement a clinician-completed instrument during the first year of the performance measurement effort. However, on p.126 the authors report that clinicians provide a Global Assessment of Functioning (GAF) score. The authors note briefly on p.130 that the GAF score has also not proven to be useful; it would be interesting to know why not. If the GAF has also been shown to be insensitive to change, one may question how much change actually occurs in this population with the protocol that was implemented and over the time period assessed. (Results obtained for the Quality of Life measure would also be of interest in this regard.)
- The authors cite earlier research comparing intake and follow-up BASIS® scores indicating that these studies showed statistically significant but small differences over time, and suggesting that there may be floor effects. However, what they consider a small difference is questionable. Eisen et al. 2 reported effect sizes for improvement among outpatients ranging from 0.31 to 0.53 (effect size = 0.53 for relation to self/others, depression/anxiety and mean BASIS® score). Larger effect sizes were reported for change among inpatients (up to 0.84). Cohen 3 suggested that an effect size of 0.50 (medium) is “large enough to be visible to the naked eye.” An effect size of 0.80 is considered by Cohen to be large. Thus, the research cited by Higgins and Purvis as showing small differences over time, has shown medium to large effect sizes as well.The authors’ comment that “floor effects” may be the reason for the small amounts of change is also implausible. BASIS® results obtained for a representative U.S. population sample show normative scores ranging from 0.20 to 0.70 with an overall average of 0.49. 4 These scores indicate lower levels of symptom and problem difficulty than have been reported for any of the clinical samples studied, as would be expected in an untreated population. In addition, they indicate that there is further room for improvement.
We agree that self-report measures have limitations, as do clinician-reported measures and other outcome data sources. Discussion of these limitations is beyond the scope of this letter, but they are well documented in the literature. Consequently, we recommend that comprehensive outcome measurement systems include multiple outcome measures and perspectives.
Susan V. Eisen, PhD
Alexander Speredelozzi, MBA
1. Higgins J, Purvis K. Usefulness of the BASIS-32 in evaluating program level outcomes. Journal of Psychiatric Practice 2002; 8:125–30.
2. Eisen SV, Culhane MA. Behavior and Symptom Identification Scale (BASIS-32). In: Maruish ME, ed. The use of psychological testing for treatment planning and outcomes assessment. 2nd ed. Mahwah, NJ: Lawrence Erlbaum; 1999.
3. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.
4. Eisen SV. (with special contributions by Prince-Embury S. and Weiss L.) BASIS-32 Application Guide: Community Norms and Clinical Benchmarks. Belmont, MA: The McLean Hospital Corporation and The Psychological Corporation, a Harcourt Assessment Company, 2000.