Secondary Logo

Journal Logo

HIGHLIGHTS FROM THE ACIA 15TH SYMPOSIUM ON COCHLEAR IMPLANTS IN CHILDREN IN SAN FRANCISCO

Hybrid Music Perception Outcomes: Implications for Melody and Timbre Recognition in Cochlear Implant Recipients

Parkinson, Aaron J.; Rubinstein, Jay T.; Drennan, Ward R.; Dodson, Christa; Nie, Kaibao†,§

Author Information
doi: 10.1097/MAO.0000000000002126
  • Free

Abstract

INTRODUCTION

It is well established that electric stimulation via traditional cochlear implantation generally results in significantly improved speech perception, over acoustic hearing aids for appropriate candidates, in quiet and noisy listening conditions. On the other hand, music perception has long been recognized as problematic for those relying on electric stimulation via cochlear implants (1–8). Limitations related to music perception for cochlear implant users largely result from their significantly impaired auditory systems and the historical focus on improving speech perception (9). Coding strategies have proven to be efficient at extracting and delivering cues important for relatively high levels of speech understanding in quiet that are not necessarily optimal for the coding of auditory cues important for music.

The continued difficulties that cochlear implant users experience with music perception have been attributed to the relatively poor conveyance of two essential aspects of music, namely, pitch and timbre (10). These difficulties have been largely attributed to the poor representation of spectrotemporal fine structure by current cochlear implant coding strategies, which is relevant for the perception of pitch and melody as well as the discrimination of competing melodies and musical instruments (11).

Electric–acoustic or “hybrid” stimulation represents a relatively new innovation for individuals with significant high-frequency sensorineural hearing loss. Initially described by von Ilberg and colleagues (12) in Europe and by Gantz and Turner (13) in the United States, hybrid stimulation involves the provision of electrically generated hearing for mid to high frequencies (typically above 750–1000 Hz) in conjunction with low-frequency acoustic hearing in the same ear. The target population for this technology consists of individuals with so called “ski-slope” hearing loss. Candidates typically have mild-to-moderate low-frequency sensorineural hearing loss, steeply sloping to severe or profound levels for high frequencies. Individuals with such losses find themselves in a difficult position due to the significant impact loss of effective high-frequency hearing has on speech perception and, concomitantly, communication with others.

Hybrid stimulation has been shown to provide benefits over both amplification alone and electric stimulation alone for speech recognition in quiet and in noise, sound quality, music recognition and appreciation, as well as aspects of hearing related to quality of life (6,14–26). Roland et al. (25) described the speech perception and hearing sensitivity outcomes for 50 subjects who received the Cochlear Nucleus® Hybrid L24 Cochlear Implant System (Cochlear Ltd., Sydney, Australia) as part of a Food and Drug Administration (FDA) regulated clinical trial. As with other studies, the clinical trial demonstrated the significant benefits that hybrid hearing can provide in terms of speech perception in quiet and in noise. The clinical trial also included measures of music perception, the outcomes of which were recently reported by Kelsall et al. (27).

Kelsall et al. (27) compared outcomes on the University of Washington's Clinical Assessment of Music Perception (CAMP) test for Hybrid L24 subjects with those for traditional cochlear implants. Hybrid L24 subjects largely maintained their music-related capabilities pre- to postoperative due to retention of low-frequency acoustic hearing. Further, melody recognition remained superior to that seen in traditional cochlear implant users, by 40 percentage points, after receiving the Hybrid L24 implant whereas timbre recognition was closer, within 15 percentage points, to that noted in traditional cochlear implant users.

The objective of the present study was to investigate whether or not the availability of low-frequency acoustic cues played a role in the outcomes observed in the Hybrid L24 clinical trial (25). Hybrid hearing was simulated in a group of normally hearing individuals that permitted the independent and combined contributions of low-frequency acoustic and high-frequency electric hearing to be assessed for music perception. Acoustic simulation of electric–acoustic hearing in normally hearing individuals was used to more uniformly and accurately control the extent of acoustic and electric information made available, independent of the potentially confounding effects of sensorineural hearing loss, beyond the loss of high-frequency acoustic sensitivity (e.g., variance in residual hair-cell and neural survival across subjects). Of specific interest was determining the relative contributions low-frequency acoustic stimulation and high-frequency electric stimulation may have for melody and timbre perception.

MATERIALS AND METHODS

University of Washington Clinical Assessment of Music Perception (CAMP)

The University of Washington Clinical Assessment of Music Perception (CAMP) test, described by Kang et al. (2) consists of three subtests each designed to provide an assessment of fundamental auditory skills important for music perception. One subtest provides an assessment of pitch direction discrimination, the second an assessment of isochronous melody recognition, and the third subtest assesses the perception of timbre. The focus in this study was on outcomes for the melody and timbre subtests.

For melody recognition, listeners are asked to identify familiar melodies, created using complex tones, from a closed-set of 12 items (e.g., “Happy Birthday,” “Three Blind Mice,” “Twinkle, Twinkle, Little Star”) with rhythmic cues removed. Timbre recognition, also a closed-set test, involves the listener identifying one of 8 musical instruments (e.g., piano, acoustic guitar, clarinet). Further details regarding the test design, its development, validation and stimuli can be found in Drennan et al. (1), Kang et al. (2), and Nimmons et al. (28).

Subjects

Nine normally hearing individuals aged 22 through 29 years completed CAMP testing, listening to stimuli processed to simulate electric–acoustic hearing. These subjects were undergraduate or graduate students attending the University of Washington in Seattle. The only requirement to be included as a subject was the presence of normal hearing defined by audiometric thresholds better than 25 dB HL over the range 250 through 8000 Hz, bilaterally. Although musical background was not probed, musical experience was not a determining factor for inclusion or exclusion as a subject. Subjects were consented under Institutional Review Board approval prior to study testing commencing.

Hybrid Simulation Design

Acoustic simulation of electric–acoustic hearing allows variables of interest to be controlled independent of any potentially confounding effects of sensorineural hearing loss, for example, (29–31). To simulate electric stimulation an eight-band noise vocoder was used to process the stimuli from the melody and timbre subtests of the CAMP. The vocoder processed input over the frequency range of 813 through 7938 Hz, typical for Hybrid implant subjects in the clinical trial. An eight-band vocoder was designed to represent the default program settings for the ACE coding strategy used by Hybrid users. Specifically, while more than eight frequency bands encompass this range and all can potentially generate stimulation, no more than eight, on average, are stimulated within each stimulation cycle. A pre-emphasis filter was first applied to the input, which was then divided into eight bands of equal bandwidth on a logarithmic scale. Within each band, the envelope was extracted by full-wave rectification and low-pass filtered at 30 Hz. The envelopes within each band were modulated with band-limited noise and summed to generate simulated electric stimulation via the Hybrid implant.

To simulate residual low-frequency acoustic hearing melody and timbre stimuli were filtered using a steep low-pass filter at 1000 Hz. Attenuation was then applied commensurate with average audiometric thresholds at 125, 250, 500, 750, and 1000 Hz for 33 Hybrid clinical trial subjects (Fig. 1). These were clinical trial subjects who maintained low-frequency acoustic sensitivity that corresponded to hearing loss of a severe degree or better (≤ 90 dB HL) representing patients who most likely benefitted from postoperative acoustic stimulation. Input above 1000 Hz was further attenuated by at least 80 dB (effectively removing acoustic contribution above 1000 Hz). For conditions simulating electric–acoustic hearing, the acoustic and electric components were combined.

FIG. 1
FIG. 1:
Average audiogram used to derive low-pass filter characteristics for acoustic simulation over the range 125 through 1000 Hz. Error bars: ±1 standard deviation of the mean.

Test Procedure

Test stimuli processed to simulate low-frequency acoustic hearing as well as high-frequency electric stimulation were presented three times each, in random order for both subtests. Presentation level was at 65 dBA via a loudspeaker located in front of the listener at 0° azimuth. Three listening conditions were assessed for each subject:

  • 1. Electric alone, wherein subjects listened to processed stimuli designed to represent the information available (813–7938 Hz) to a Hybrid listener via the electric domain alone,
  • 2. Acoustic alone, wherein subjects listening to processed stimuli designed to represent the information available to a Hybrid listener, with residual low-frequency hearing, via the acoustic (up through 1000 Hz) domain only,
  • 3. Electric–acoustic stimulation, wherein the electric and acoustic domains are brought together, as would be the case for a Hybrid cochlear implant user with residual low-frequency acoustic hearing.

Statistics

Mean differences were subjected to one-way repeated measures analysis of variance (RM ANOVA) with follow-up pair-wise comparisons based on the Holm-Sidak method (SigmaPlot Version 14.0, Systat Software, Inc., San Jose, CA). If there was significant evidence that the assumptions of normality and/or equal variance were not met a RM ANOVA on Ranks was performed (Friedman RM ANOVA on Ranks) with follow-up paired comparisons based on the Tukey Test.

RESULTS

Melody Recognition

Mean and individual scores for melody recognition are shown for the nine study subjects in Figure 2. For reference, mean scores for acoustic alone and electric–acoustic conditions are plotted for 47 subjects (with matched scores) from the Hybrid L24 clinical trial (25), for 10 normally hearing (NH) individuals (2), and 145 cochlear implant users (1) to the right of the graph. The mean scores for the acoustic alone and electric–acoustic conditions were 67.9% (S.D. = 25.4%) and 65.9% (S.D. = 29.5%), respectively, for the Hybrid subjects. Normally hearing individuals (acoustic alone) scored 87.5% (S.D. = 8.3%) on average, whereas cochlear implant subjects (1) scored 26.2% (S.D. = 19.9%).

FIG. 2
FIG. 2:
Individual and mean melody recognition scores for simulation study subjects (left). Mean scores for Hybrid L24 clinical trial subjects, normally hearing individuals (NH), and cochlear implant users (CI) are shown to the right for reference. Error bars indicate +1 standard deviation of the mean.

For the subjects in the present study, to the left of Figure 2, mean melody recognition for the simulated electric-alone condition was 39.2% (S.D. = 18.1%), 73.5% (S.D. = 15.4%) for the acoustic-alone condition, and 67.9% (S.D. = 21.1%) for the electric–acoustic condition. RM ANOVA indicated a significant effect of listening mode (F = 19.24, p < 0.001). Follow-up paired comparisons (Holm-Sidak method) showed that the mean score for the electric-alone condition was significantly lower than scores for the acoustic and electric–acoustic modes (p < 0.001, in both cases). As for the Hybrid L24 clinical trial subjects (25), mean scores for the acoustic-alone and electric–acoustic modes were not significantly different (p > 0.05).

Drennan et al. (1) calculated confidence intervals based on test–retest scores for the CAMP Melody and Timbre tests. In the case of melody recognition, they suggested that a difference of >8.9 percentage points can be considered significant for an individual cochlear implant listener taking the test under multiple conditions. On this basis, electric-alone performance was poorer than conditions where acoustic information was available for all nine subjects, with the exception of the electric–acoustic condition for Subject 2 (i.e., electric alone and electric–acoustic conditions were within 8.9 percentage points). The acoustic-alone condition was equivalent to (i.e., difference was within 8.9 percentage points) or poorer (i.e., difference was greater than 8.9 percentage points, in favor of electric–acoustic) than the electric–acoustic condition for 5 subjects and better for 4 subjects (i.e., difference was >8.9 percentage points, in favor of acoustic alone).

Timbre Recognition

Mean and individual scores for timbre recognition are shown for the 9 study subjects in Figure 3. For reference, mean scores for acoustic alone and electric–acoustic conditions are plotted for 47 subjects (with matched scores) from the Hybrid L24 clinical trial (25), for 10 normally hearing (NH) individuals (2) and 145 cochlear implant users (1) to the right of the graph. The mean timbre scores for the acoustic alone and electric–acoustic conditions were 51.6% (S.D. = 17.9%) and 56.6% (S.D. = 22.7%), respectively, for the Hybrid subjects. Normally hearing individuals (acoustic alone) scored 94.2% (S.D. = 4.0%) on average, whereas cochlear implant subjects (1) scored 43.2% (S.D. = 22.0%).

FIG. 3
FIG. 3:
Individual and mean timbre recognition scores for simulation study subjects (left). Mean scores for Hybrid L24 clinical trial subjects, normally hearing individuals (NH), and cochlear implant users (CI) are shown to the right for reference. Error bars indicate +1 standard deviation of the mean.

Results from this study are plotted to the left of Figure 2. Mean timbre recognition for the electric-alone condition was 36.1% (S.D. = 17.7%), 38% (S.D. = 20.4%) for the acoustic-alone condition, and 40.7% (S.D. = 19.7%) for the electric–acoustic condition. RM ANOVA indicated no significant effect of listening mode (F = 0.345, p > 0.05). That is, mean scores across electric-alone, acoustic-alone and electric–acoustic modes were not significantly different. Similarly, mean Hybrid L24 scores (25) for the acoustic-alone and electric–acoustic modes were not significantly different either as shown in Figure 2.

The timbre test is subject to larger variance in cochlear implant users than that observed for melody recognition, consequently a difference greater than 28 percentage points was considered significant for an individual taking the test under multiple conditions by Drennan et al. (1). Consistent with the larger variability observed for the Timbre test, there was no particular pattern for one listening mode to be better or poorer than another across individuals.

DISCUSSION

Data from the Hybrid L24 clinical trial indicate that the presence of low-frequency acoustic hearing permits electric–acoustic users to maintain good melody perception abilities relative to cochlear implant users (27). Results from the present study support the notion that the music outcomes, most notably for melody recognition, observed in the Hybrid trial are a related to the availability of low-frequency acoustic cues not present in the electric domain for both Hybrid recipients (27) and traditional cochlear implant users (1,2). Melody recognition scores in this study were consistently better for listening conditions that included acoustic information (i.e., acoustic alone and electric–acoustic) and mean scores for both the acoustic and electric–acoustic scores were significantly better than the electric alone condition. This was not the case for timbre recognition for which scores were more variable across simulated listening modes with no significant differences found in mean scores across electric, acoustic, and electric–acoustic conditions. Low-frequency acoustic hearing appears to be particularly important for melody recognition but less so for timbre recognition. This would explain the observed superior performance of the Hybrid L24 subjects for melody recognition and comparable performance for timbre recognition, relative to traditional cochlear implant users.

Drennan et al. (1) reported CAMP outcomes for 145 cochlear implant subjects. For melody recognition, the mean score for traditional cochlear implant subjects was 26.2% (Fig. 1) compared with 65.9% for hybrid subjects. It is worth noting that the mean score for simulated electric–acoustic hearing in the present study was similar in magnitude, 67.9%, to that observed in the Hybrid L24 trial. For the timbre subtest, the mean score for traditional cochlear implant subjects was 43.2% (Figure 2), which compared well with the score noted for the simulated electric alone score of 36.1% in this study. The latter score reflected electric stimulation over an abbreviated frequency range of 813 through 7938 Hz, similar to that provided to Hybrid users. Perhaps, if the normally hearing subjects had access to the full frequency range (188–7938 Hz), as would a traditional cochlear implant user, the mean score may have been higher. Although not reported by Kelsall et al. (27), 10 of the 47 Hybrid trial subjects did not retain sufficient low-frequency acoustic hearing to make use of amplification and were programmed with full frequency assignments (typically, 188–7938 Hz). Interestingly, the mean timbre score of 47.1% (S.D. = 18.6%) for these 10 subjects was also comparable to the traditional implant subjects, referred to above. Presumably, the poorer traditional cochlear implant performance for melody recognition reflects the paucity of low-frequency spectral cues that are present in the acoustic domain but not in the electric signal.

Absolute comparisons of mean scores should be treated cautiously given the subjects in this study had normal hearing and, presumably, normal auditory systems. However, there is some assurance provided by these comparisons in that music perception appears to have been comparably compromised in the simulated listening conditions, as would be the case for Hybrid listeners. More importantly, the pattern of the results, overall, was similar to that observed in Hybrid users. Namely that the presence of low-frequency acoustic hearing allows for better melody recognition, relative to traditional cochlear implant users, but is less impactful for timbre perception. In addition, melody scores in electric–acoustic or acoustic alone conditions are better than timbre scores, for simulated and nonsimulated subjects, whereas the opposite is true for traditional cochlear implant users (1).

Implications for Cochlear Implant Users

Hybrid listeners have speech and music perception abilities that greatly exceed those of traditional cochlear implant users. However, their timbre recognition is still quite limited. This likely reflects poor broadband spectral and potentially temporal perception of both Hybrid and traditional cochlear implant users relative to those with normal hearing. To our knowledge, the only intervention reported to improve timbre perception in cochlear implant users is the Harmonic Single-Sideband Encoding (HSSE) strategy (10). Average timbre perception for eight subjects, using HSSE acutely (i.e., the subjects had relatively limited exposure, compared with their own clinical strategies), was 16 percentage points better than for the same subjects’ clinical processing strategy. This was statistically significant. HSSE was developed specifically to improve temporal and harmonic coding and as far as is known does not enhance broadband spectral cues. Since hybrid users appear to have excellent spectral discrimination in the low frequencies, where their thresholds are near normal (20), it is clear that better broadband spectral representation is one mechanism to improve timbre perception in both hybrid and traditional cochlear implants. However, improved temporal or harmonic coding may also be an independent route to better timbre perception. Either way, it appears that better timbre perception currently is out of reach for either traditional or hybrid cochlear implant users, regardless of residual hearing preservation, without further technology development.

The data for this particular study involved a small number of subjects. However, further experimentation, using simulated electric–acoustic stimulation, might be better spent exploring how music perception relates to the degree of low-frequency hearing preservation. The average audiogram used to generate the simulated acoustic hearing in this study represented a mild sloping to severe loss up to 750 Hz. Future study could examine the impact of hearing loss in terms of audibility (i.e., the degree of threshold shift loss) and the frequency domain. That is, how impaired can one be and still retain good music perception and up to what frequency does one need good audibility? Further work involving coding strategies, such as the HSSE strategy, is also needed to explore means for improving music perception.

CONCLUSION

Recipients of hybrid cochlear implants demonstrate music perception abilities superior to those observed in traditional cochlear implant recipients. Results from the present study support the notion that hybrid stimulation confers advantages related to the availability of low-frequency acoustic hearing, most particularly for melody recognition. However, timbre recognition remains more limited for both hybrid and traditional cochlear implant users. Opportunities remain for new coding strategies to improve timbre perception.

Acknowledgments

The authors would like to thank the study participants for giving their time and effort to complete testing for this study.

REFERENCES

1. Drennan WR, et al. Clinical evaluation of music perception, appraisal and experience in cochlear implant users. Int J Audiol 2015; 54:114–123.
2. Kang R, et al. Development and validation of the University of Washington Clinical Assessment of Music Perception test. Ear Hear 2009; 30:411–418.
3. Limb CJ, Rubinstein JT. Current research on music perception in cochlear implant users. Otolaryngol Clin North Am 2012; 45:129–140.
4. Gfeller K, Lansing CR. Melodic, rhythmic, and timbral perception of adult cochlear implant users. J Speech Hear Res 1991; 34:916–920.
5. Gfeller K, et al. Recognition of familiar melodies by adult cochlear implant recipients and normal-hearing adults. Cochlear Implants Int 2002; 3:29–53.
6. Gfeller KE, et al. Music perception with cochlear implants and residual hearing. Audiol Neurootol 2006; 11 (Suppl 1):12–15.
7. Looi V, et al. Music perception of cochlear implant users compared with that of hearing aid users. Ear Hear 2008; 29:421–434.
8. McDermott HJ. Music perception with cochlear implants: a review. Trends Amplif 2004; 8:49–82.
9. Drennan WR, Rubinstein JT. Music perception in cochlear implant users and its relationship with psychophysical capabilities. J Rehabil Res Dev 2008; 45:779–789.
10. Li X, et al. Improved perception of music with a harmonic based algorithm for cochlear implants. IEEE Trans Neural Syst Rehabil Eng 2013; 21:684–694.
11. Crew JD, et al. Contributions of electric and acoustic hearing to bimodal speech and music perception. PLoS One 2015; 10:e0120279.
12. von Ilberg C, et al. Electric-acoustic stimulation of the auditory system. New technology for severe hearing loss. ORL J Otorhinolaryngol Relat Spec 1999; 61:334–340.
13. Gantz BJ, Turner CW. Combining acoustic and electrical hearing. Laryngoscope 2003; 113:1726–1730.
14. Gantz BJ, Turner C. Combining acoustic and electrical speech processing: Iowa/Nucleus hybrid implant. Acta Otolaryngol 2004; 124:344–347.
15. Gantz BJ, Turner C, Gfeller K. Expanding cochlear implant technology: Combined electrical and acoustical speech processing. Cochlear Implants Int 2004; 5 (Suppl 1):8–14.
16. Gantz BJ, et al. Preservation of hearing in cochlear implant surgery: advantages of combined electrical and acoustical speech processing. Laryngoscope 2005; 115:796–802.
17. Gantz BJ, Turner C, Gfeller KE. Acoustic plus electric speech processing: preliminary results of a multicenter clinical trial of the Iowa/Nucleus Hybrid implant. Audiol Neurootol 2006; 11 (Suppl 1):63–68.
18. Gantz BJ, et al. Multicenter clinical trial of the Nucleus Hybrid S8 cochlear implant: Final outcomes. Laryngoscope 2016; 126:962–973.
19. Gifford RH, et al. Cochlear implantation with hearing preservation yields significant benefit for speech recognition in complex listening environments. Ear Hear 2013; 34:413–425.
20. Golub JS, et al. Spectral and temporal measures in hybrid cochlear implant users: on the mechanism of electroacoustic hearing benefits. Otol Neurotol 2012; 33:147–153.
21. Kiefer J, et al. Combined electric and acoustic stimulation of the auditory system: results of a clinical study. Audiol Neurootol 2005; 10:134–144.
22. Kong YY, et al. Music perception with temporal cues in acoustic and electric hearing. Ear Hear 2004; 25:173–185.
23. Kong YY, Stickney GS, Zeng FG. Speech and melody recognition in binaurally combined acoustic and electric hearing. J Acoust Soc Am 2005; 117 (3 Pt 1):1351–1361.
24. Lenarz T, et al. European multi-centre study of the Nucleus Hybrid L24 cochlear implant. Int J Audiol 2013; 52:838–848.
25. Roland JT Jr, et al. United States multicenter clinical trial of the cochlear nucleus hybrid implant system. Laryngoscope 2016; 126:175–181.
26. Turner CW, et al. Speech recognition in noise for cochlear implant listeners: benefits of residual acoustic hearing. J Acoust Soc Am 2004; 115:1729–1735.
27. Kelsall DC, Arnold RJG, Lionnet L. Patient-Reported Outcomes From the United States Clinical Trial for a Hybrid Cochlear Implant. Otol Neurotol 2017; 38:1251–1261.
28. Nimmons GL, et al. Clinical assessment of music perception in cochlear implant listeners. Otol Neurotol 2008; 29:149–155.
29. Dorman MF, et al. Acoustic simulations of combined electric and acoustic hearing (EAS). Ear Hear 2005; 26:371–380.
30. Fu QJ, Galvin JJ, 3rd X, Wang. Integration of acoustic and electric hearing is better in the same ear than across ears. Sci Rep 2017; 7:12500.
31. Fu QJ, Galvin JJ 3rd, Wang X. Effect of carrier bandwidth on integration of simulations of acoustic and electric hearing within or across ears. J Acoust Soc Am 2017; 142:EL561.
Keywords:

cochlear implant; electric–acoustic; hearing preservation; hybrid; low-frequency acoustic hearing; music perception

© 2019, Otology & Neurotology, Inc.