Journal Logo

Page Ten

Outcome measures

In health care today, you can't afford not to do them

Abrams, Harvey

doi: 10.1097/01.HJ.0000294451.23908.a6
  • Free

In Brief

If you're a regular reader of Page Ten, you know that just about every year we try to include something related to outcome measures. We started with a review of some self-assessment scales by Patricia McCarthy back in 1994, and the following year Fred Bess described several clinical applications of the HHIE. In 1998 Patti came back to tell us about some “HAPI” and “COSI” experiences, no doubt consulting with Catherine Palmer, who in the same year introduced us to a new “PAL.” And last year, Stuart Gatehouse visited Page Ten to tell us all about the Glasgow inventory. So what is there left to say? A lot!

Most of us are probably familiar with outcome measures that relate to the person's impairment, since conducting aided soundfield thresholds or insertion gain measures is fairly common. There are also the hearing aid outcome measures that relate to the patient's disability and handicap, such as standardized speech-in-noise tests and self-assessment inventories like the APHAB and the HHIE. And, of course, we usually assess the patient's satisfaction with hearing aids. Recently, however, managed care providers, insurers, and state agencies have become more interested in health-related quality-of-life measures. That is, how does the fitting of hearing aids improve the person's overall well being. Additionally, healthcare economists and policy makers have their own set of outcome measures and agendas. How do all these pieces fit together?

Explaining that is this month's guest author, Harvey Abrams, PhD, who is chief of audiology and speech pathology at the VA Medical Center in Bay Pines, FL. He is also on the faculty at the University of South Florida in Tampa. Dr. Abrams has conducted numerous workshops and seminars on outcome measures, and recently published two book chapters on this topic. Like the notable Captain Carhart, who also wrote about hearing aid outcome measures, Harvey got his audiologic start in the U.S. Army. It was probably this “be-all-you-can-be” experience that led to his current avocation of distance running, which gives him plenty of time to think about how standard gambles and time trade-offs predict HQL, and how they can be used with VAS within the QUA to compute the CBA in an effort to maximize the QALY.


But, this is all important stuff and very likely will affect our profession. For those of you who have more immediate concerns, Dr. Abrams also provides a practical protocol that you can put into use Monday morning. As Harvey concludes, outcome measures are simply something that you can't afford not to do.

Gus Mueller

Editor, Page Ten

1 I've been hearing and reading quite a bit about outcome measures. What are they and why are we measuring them?

In our current healthcare environment, an outcome refers to any form of benefit resulting from a particular intervention. I like to think of outcomes as evidence that we've made a difference in our patients' lives. The reason outcome assessment has taken on a level of importance is that there has been increasing pressure placed upon providers to demonstrate that their interventions are making a difference. Interventions include surgery, therapy, medication, or devices such as hearing aids.

2Who exactly is creating this pressure?

Much of the pressure comes from insurers, the state and federal government through the Medicaid and Medicare programs, and managed care organizations. In the case of insurers and the government, they want evidence that their money is well spent. The managed care organizations want to ensure that their enrollees are benefiting and are satisfied with their care.

As the sophistication and cost of our instruments increase, we need to be able to demonstrate the superiority of those devices in addressing and resolving our patients' complaints. Family members are also stakeholders, particularly the parents of children with hearing impairment.

3 Why can't we simply ask our patients if they're satisfied with their hearing aids and leave it at that?

We could, but, as we'll get to later, satisfaction is but one outcome among many that we may wish to measure. By simply asking a yes or no question, we miss the opportunity to discover the reasons our fitting is a success or failure. Measuring the outcome of our care can also provide the dispenser with an effective marketing tool. Imagine being able to advertise that 90% of your patients report improved speech understanding following treatment at your practice. This type of data is not only important for advertising in your community but for marketing your practice to compete in today's managed care environment.

4 You mentioned that there are other measures beside satisfaction. What are they?

The most common are measures of impairment and what we used to refer to as disability and handicap.

5 Why do you say “used to?”

The World Health Organization (WHO) is revising the terminology of its International Classification of Impairment, Disability, and Handicap or ICIDH. You're probably wondering what WHO has to do with outcome assessments.

The ICIDH is an international coding system developed by WHO. Unlike the familiar International Classification of Diseases (ICD) classification system, the ICIDH classifies the effects of diseases on functioning. For example, the ICD code for sensorineural hearing loss is 389.10. Knowing that, however, tells us nothing about how that hearing loss impacts on communication, activities of daily living, or interaction with others in the community.

The original ICIDH system published in 1980 contained codes for disability and handicap as well as for impairment. Over the years, these words have accrued increasingly negative connotations. As a result, WHO published a draft revision of the ICIDH in 1997 and substituted the term “activity limitations” for “disability” and “participation restriction” for “handicap” and their associated negative effects. The draft, currently undergoing world-wide review, is entitled “ICIDH-2: International Classification of Impairments, Activities and Participation: A Manual of Dimensions of Disablement and Functioning.”

6 What are the differences among impairment, activity limitations, and participation restriction?

We can think of impairment as the effect of a disorder at the level of the body, activity at the level of the person, and participation at the level of society.

When a hearing disorder results in changes in the auditory anatomy such as a loss of sensory structures, a hearing impairment is present.

Presence of a hearing impairment, then, can result in limitations of daily activities such as hearing and understanding speech that may adversely affect conversing at a restaurant, using the telephone, or watching television. These activity limitations might restrict participation, such as participating in family gatherings. The individual might also have difficulty in aspects of job performance requiring the use of a telephone that may ultimately lead to resignation or termination.

The domain of participation restriction is highly influenced by environmental factors such as the presence of federal and local regulations pertaining to the disabled, the availability of assistive devices, community support for the rights of the disabled, architectural considerations, workplace accommodations, and transportation alternatives. When an enlightened community or place of business provides the appropriate accommodations, the impact on the individual's participation in society is minimized despite the presence of an impairment.

7 Activity limitations and participation restrictions sound a lot like quality-of-life issues. Are they?

Indeed they are. When an impairment leads to significant activity limitations and participation restrictions, then an individual's overall physical, mental, and social well being may also be adversely affected. When this occurs, the person is said to experience a reduction in his or her health-related quality of life (HQL).

While health-related quality of life is not a specific domain within the WHO classification system, it is understood that HQL represents the sum of the effects of each impairment, activity limitation, and participation restriction. In fact, there are specific outcome measures unique to health-related quality of life.

I believe that satisfaction is a special case. Most of us have treated patients who we just know have benefited from the hearing aids we've selected for them yet are not satisfied with the outcome of treatment. Satisfaction does not always correspond to significant or quantifiable changes in impairment, activity limitations, participation, or health-related quality of life.

In addition to improvements in communication and real-world functioning, the domain of satisfaction involves the patient's relationship with service providers, the ease of access to services, as well as the influence of factors such as cosmetics, comfort, expectations, and perceived value. It is a domain that requires a separate assessment.

8 This is starting to get complicated—impairment, activity limitation, participation restriction, health-related quality of life, and satisfaction. How do you know which domain to measure and what measures do you use?

The particular outcome domain is far less important than the fact that outcome has been assessed in at least one domain. But let's take them one at a time, starting with impairment.

Remember that we need to match our outcome measure to the treatment goal. The primary goal of treatment in the impairment domain is to improve our patient's ability to detect sounds, since reduction of this ability is the main effect of conductive or sensory structure damage. The change in audiometric thresholds following stapedectomy is a measure of surgical outcome. The functional or real-ear insertion gain achieved following hearing aid fitting is a measure of audiologic outcome. An indirect outcome measure in the impairment domain is the aided articulation index (AI). The AI calculates the amount of acoustic speech information available to the individual. The higher the aided AI compared to the unaided AI, the more speech information is presumably available.

9 But don't we know that audibility doesn't always mean improved speech perception and communication functioning, particularly in noisy environments?

You're absolutely correct, and that's where our measures in the activity and participation domains come in. The goal of our treatment in the activity domain is to improve speech understanding and communication functioning. We measure the effectiveness of our treatment in this domain primarily through aided vs. unaided speech-recognition tests, including monosyllable tests (W-22, NU-6, CCT, etc.) and more recently developed sentence-length materials such as the Hearing in Noise Test (HINT) and Speech in Noise (SIN) tests.

The treatment goal in the participation domain is to improve communication functioning in specific situations and with specific partners. This is where we would use the familiar questionnaires such as the Abbreviated Profile of Hearing Aid Benefit (APHAB) and Hearing Handicap Inventory for the Elderly (HHIE). The questionnaires are administered and scored before and after intervention and the changes in scores represent the degree of improvement or outcome.

The Client Oriented Scale of Improvement (COSI) is a particularly effective outcome measure, in my opinion. Unlike the APHAB and HHIE which have a predetermined list of questions, some of which may not be relevant or important to the patient, the items on the COSI are actually created by the patient with the clinician acting as facilitator. The patient is encouraged to list up to five situations that he or she considers the most impacted by the hearing impairment and would like corrected through the intervention process. Following intervention, the patient judges the extent to which the stated problems have been resolved.

The beauty of this approach is that it is patient-centered. That is, the patient is an active participant in the development of treatment goals. It is particularly important, however, for the clinician to direct the stated problems in as specific language as possible. For instance, it is not good enough to simply state that “I want to hear better in noise.” The clinician needs to probe to determine what kind of noise, in what situations, and with whom, which are precisely the kinds of patient-specific problems we need to identify in the participation domain.

For clinicians who like the features of both the APHAB and the COSI, the Glasgow Hearing Aid Benefit Profile (GHABP) identifies four pre-specified situations for outcome assessment and also asks the patient to identify up to four additional situations in which he or she experiences the most difficulty.

10 I'd like to get back to the idea of satisfaction. You mentioned before that measuring satisfaction requires separate kinds of assessment. Do we have any specific scales or questionnaires like the APHAB or HHIE that measure satisfaction?

Yes, we do. Robyn Cox and Genevieve Alexander have developed the Satisfaction with Amplification in Daily Life (SADL). The SADL is a 15-item questionnaire that attempts to determine the patient's level of satisfaction among several hearing aid-related dimensions, including perceived positive and negative effects of amplification, service and cost, and perceived effect on personal image (for review check out

There has been a considerable amount of research examining consumer satisfaction with hearing aids, most notably the Knowles MarkeTrak survey series conducted by Sergei Kochkin. The last few surveys indicated higher levels of consumer satisfaction with high-performance and multiple-microphone hearing aids than with conventional and omnidirectional instruments.1,2

I find that many clinics have designed their own satisfaction questionnaire. Keep in mind that the goal of treatment in the satisfaction domain is to meet or exceed the patient's expectations with your product and service. As long as the questionnaire measures these features, you'll have an effective instrument. The limitation of using your own questionnaire, however, is that you will not be able to compare your results to those of other clinics. Third-party payers and quality review organizations, in particular, are looking for standardized and widely accepted measures that can be compared across clinics and providers.

11 We haven't discussed how to measure health-related quality of life. What can you tell me about that?

There is increased interest in examining the impact of hearing impairment in terms of HQL. The goal of treatment in the HQL domain is to improve your patient's overall perception of well being.

An assessment of an individual's HQL involves more global considerations than those normally associated with impairment, activity limitations, or participation restrictions, although each of these necessarily impacts on an individual's perceived HQL. While specific clinical disciplines such as psychiatry, rehabilitation, cardiology, and oncology use HQL measurements to assess outcome, little is known about how audiologic disorders and interventions affect HQL, particularly when compared to other health-related disorders and treatments.

12 “Perception of well being” and “improved quality of life” seem like difficult concepts to measure. How do you do it?

There are several ways. HQL measures can be categorized as disease-specific or generic.

Disease-specific measures are useful for comparing different treatment options for the same health condition. An example of a disease-specific HQL measure in audiology is the HHIE. Because the HHIE includes questions relating to the emotional impact of hearing loss, it serves as an appropriate HQL instrument.

Generic HQL measures, on the other hand, are designed for use across health conditions. For example, a generic HQL measure was recently used to demonstrate the cost-effectiveness of cochlear implantation relative to other health interventions, such as coronary angioplasty and neonatal intensive care.

13 Why is it important to compare what we do in audiology against other disorders or disciplines?

Because the healthcare dollar is going to go to those interventions that make the biggest impact on quality of life. If we ever want audiology examinations and intervention to be covered services throughout the health insurance industry, we are going to have to demonstrate that what we do is comparable, in terms of improving quality of life, to a hip replacement or cataract removal.

To do this, we need to conduct well-designed and well-controlled clinical trial research. As part of that design we can incorporate some unique measures called utilities that allow us to make cross-disciplinary comparisons. Utility measures have traditionally been used by healthcare economists and policy makers to determine a community's perception of the impact of different diseases on their quality of life. For example, respondents are asked to rank-order a list of disorders (e.g., blindness, deafness, uncontrolled diabetes, quadriplegia) on a visual analog scale (VAS) ranging from 0.0 (death) to 1.0 (perfect health). How they rank these disorders is an indication of their health state preference, which is another name for utility. Because the same VAS can be used across different diseases, we can compare the effect of hearing impairment and our treatment on self-perceived quality of life against other disorders and treatments.

14 I've heard you talk about “standard gamble” and “time trade-off” measures. Could you explain what these are?

Standard gamble and time trade-off are other forms of utility assessments. In the standard gamble technique, patients are asked how much they would be willing to gamble for perfect health given an x% chance of death. The time trade-off technique is a variation of standard gamble where the patient is asked how many years of life he or she would be willing to trade for perfect health.

15 Aren't these very difficult questions to ask a patient?

As with the VAS, we substitute “perfect hearing” and “deafness” for “perfect health” and “death” so it's not quite so dramatic. Also, patients understand that these are hypothetical choices designed to quantify their perceived quality of life, not real treatment consequences. Standard gamble and time trade-off can be very effective outcome measures, however. The extent to which a patient is willing, hypothetically, to gamble or trade off years of life before and after treatment is a quantifiable and meaningful measure of how much our intervention has impacted the person's quality of life.

16 These are certainly fascinating concepts, but I'd like to get back to something a bit more mundane—money. It's one thing to demonstrate that our patients do better on the APHAB, HHIE, COSI, or on a visual analog scale, but we seem to be under increasing pressure to show that our treatment is “cost-effective.” How do we measure that?

Great question. Actually, I consider measuring the costs associated with outcome as another outcome domain—the economic domain. There are several measures available, depending on what you want to measure in the economic domain. For example, if you want to compare the money saved by providing audiology treatment against the money you would have to spend by not providing treatment you would perform a cost-benefit analysis (CBA).

This is how many communities gained support for universal newborn hearing screening and early intervention. It has been demonstrated that the costs associated with screening and intervening at birth are far less over the lifetime of the individual than the costs associated with not providing that care. Children who are not identified early require more intensive and costly special education resources. Also, they are more likely to earn less money as adults, resulting in lower tax revenues.

17 We can't be expected to perform a cost-benefit analysis for each patient, can we?

No, of course not. The cost-benefit analysis is a special measure that is primarily used to compare treatment vs. no treatment or several treatment alternatives against each other among populations.

There is another cost-analysis measure that probably is more applicable to hearing aid outcomes. It's known as a cost-effectiveness analysis (CEA). A CEA can compare the clinical benefits of one hearing aid technology against another for a single patient. For example, we can compare the change in the signal-to-noise ratio (SNR) required for 50% understanding on the HINT of an analog hearing aid with directional-microphone technology against a single-microphone, fully digital instrument.

Given the results I've seen reported in the literature, it's likely that the much less expensive analog instrument equipped with a directional microphone will yield a better SNR improvement than the fully digital instrument (without directional-microphone technology). The cost per 1 dB of SNR improvement will be much less for the analog than for the digital instrument, making it considerably more cost-effective.

18 Why is that important to know?

Well, it is precisely this type of data that insurers look to when determining reimbursement and coverage policies. Why would an insurer agree to a higher reimbursement rate for a particular hypertension-lowering drug if another one does the same thing at a lower cost? We have to start asking the same type of question concerning hearing aids.

19 But patient acceptance of a particular hearing aid involves a lot more than how much the SNR was improved on a single test, doesn't it?

Exactly. Patient acceptance involves all of the domains we've been talking about, which brings us back to the very important issue of selecting the most appropriate outcome measure, or better yet, using several measures that assess different outcome domains. As a matter of fact, if you look at the hearing aid research performed at some of our better laboratories, you will see that the investigators use a number of objective speech-perception and subjective self-report instruments to determine treatment efficacy.

Before we conclude, I would like to briefly mention one other measure that cuts across the QOL and economic domains known as a cost-utility analysis (CUA). A CUA incorporates utility measures such as a visual analog scale, standard gamble, or time trade-off to determine costs per quality adjusted life years gained (QALY). The QALY is a single index of monetary value that combines a qualitative and quantitative measure of treatment benefit calculated over the lifetime of the individual.

A recent article examining the cost-utility of cochlear implants in adults concluded that it costs $11,125 per QALY for the cochlear implant compared to $34,836 for a defibrillator implant and $59,292 for a knee replacement.3 I believe the QALY calculation will emerge as a major method for making healthcare decisions in the near future.

20 It seems to me that you need to measure more than one outcome to get a complete picture of how the patient has benefited from our treatment. But, how do you respond to practitioners who say they can barely afford the time to administer one measure, let alone several?

My response to them is that they can't afford not to—particularly when insurers, government, managed care organizations, our patients, and their families are demanding evidence of improvement.

Keep in mind that several of the measures I discussed are conducted by researchers and health economists. Practically speaking, it's relatively easy to incorporate outcome measures into your practice.

Here's a protocol that you can implement Monday morning:

  • The COSI goals can be established during the initial patient interview and measured at post-fitting follow-up (activity/participation domain).
  • Real-insertion gain is measured during the fitting process (impairment domain).
  • Finally, the patient's overall perception of the hearing aid and your level of care can be determined using the SADL, either during post-fitting follow-up or through the mail (satisfaction domain).

Three domains, no waiting. You can't afford not to do it!


1. Kochkin S: Customer satisfaction and subjective benefit with high-performance hearing instruments. Hear Rev 1996;3(12):16–26.
2. Kochkin S: Customer satisfaction with single and multiple microphone digital hearing aids. Hear Rev 2000;7(11):24–34.
3. Cheng A, Rubin H, Powe N, et al.: Cost-utility analysis of the cochlear implant in children. JAMA.
Copyright © 2001 Wolters Kluwer Health, Inc. All rights reserved.