Success in adult cochlear implant (CI) users is primarily measured by the users’ ability to understand speech without visual cues, and to function in challenging listening environments—hearing in the presence of background noise and music perception are perhaps the most frustrating aspects of such challenges. The perception of music, even among the upper echelons of implant performers, may be very limited, and can be an extremely disappointing part of the implant experience. It has been said that current technology in cochlear implantation may not provide the spectral resolution and fine structures necessary for music perception, as high scores in speech understanding in quiet are not predictive of music enjoyment and perception (1–3). Despite this, many implant recipients do enjoy listening to music and have sufficient perceptual accuracy to play musical instruments. Over the last decade, a great deal of research has been dedicated to exploring the question of how to improve CI music perception. As many point to technological limitations of devices, manufacturers have been attempting to improve their speech processing strategies; however, we have yet to see corresponding improvements in music perception stem out of these changes in CI signal processing (4). As such, an alternative approach to improving music appreciation and perception in CI users may be through music training.
Evidence has shown that a significant variable in a listener's potential to enjoy music is the ability to maintain focus and to employ selective, alternating, and divided attention in the context of pitch processing (5–8). Therefore, it seemed logical to question whether these same factors would influence music perception and enjoyment in CI users. In a qualitative report, Bartel et al. (8) demonstrated that when it comes to music enjoyment, musical skills, critical listening, and one's expectations are important determinants of outcome. Furthermore, the ability to focus and maintain attention during a period of self-rehabilitation may prove to be critical contributors in both the enjoyment and perception of music with a CI. The HearTunes* software was developed with these concepts in mind. (Note: Previously referred to as MusicEAR.) A study utilizing an earlier, purely diagnostic [HearTunes* (Diagnostic)] version of this software was undertaken to evaluate its discriminant validity by examining music perception and enjoyment in three groups of subjects: cochlear implantees; normal hearing nonmusicians; and normal hearing musicians (9). The authors were able to use this diagnostic software to distinguish CI users from other groups; normal hearing musicians scored better than normal hearing non-musicians, and both groups scored significantly better than CI users in all aspects of perception and pattern recognition. This study also confirmed that CI users’ abilities to perceive and enjoy music could be quantified using this software.
A second study was conducted to examine the test–retest reliability and internal consistency of the HearTunes* (Diagnostic) software (10). Forty CI users participated in two sessions, 4 to 6 weeks apart. Results showed no significant change over time between the two administrations of the diagnostic software in any dimension. Thus, any observed changes in performance with time or after rehabilitation can be attributed to that treatment, and not to test–retest variability.
Altogether, these results confirmed that CI users’ abilities to perceive and enjoy music could be quantified by the diagnostic version of this software, and this encouraged the authors to move forward in developing a rehabilitative musical training software in the form of HearTunes* (Rehab).
Although past music training paradigms may have focused on improving perceptual skills such as pitch, timbre, and rhythm (11–14), HearTunes* (Rehab) is premised on a more holistic approach by including sections with increasing levels of complexity that involve challenging pattern and melody recognition tasks. It is entirely internally referenced in its design and content and is devised to place increasing demands on CI users’ attention, forcing them to improve upon their listening skills and focus. It begins with simple patterns and increases in complexity through nine levels up to complex patterns within melodies that requires users to divide their attention between melodies or distinguish variations in pitch and timbre simultaneously.
The goal of this research was to determine if a self-administered computer-based rehabilitation program, HearTunes* (Rehab), could improve music appreciation and speech understanding in adults who have a CI.
A group of 21 multichannel CI users (14 women and 7 men; 2 bilateral, 19 unilateral) were recruited from the CI program at Sunnybrook Health Sciences Centre (SHSC), Toronto, between October 2013 and March 2014. Each had been implanted with a contemporary standard Med-El (Med-El Corp., Innsbruck, Austria), Advanced Bionics (Corp., Valencia, CA), or a Nucleus (Cochlear Ltd., Sydney, Australia) device. Their ages ranged from 32 to 82 years (mean: 56.7 yr; SD: 13.7). The etiology of hearing loss varied and was unknown in many patients. Participants used at least one implant for an average duration of 24.5 months (range 3 mo–13 yr) before participating in this research. (As the participants’ duration of CI experience varied substantially, an ANCOVA was run to rule out duration of implantation as a confounding variable. None of the reported outcomes changed significantly from the values reported within this article.) See Table 1 for patient demographics. A repeated-measures within-participant design was used to compare the participant's baseline music perception, music appreciation, and speech perception scores to their post-training scores, and their 6-month follow-up scores.
Exclusion criteria included being unable to perform the visual and/or motor tasks required of the test (to see the computer screen and press appropriate keys) or having evidence of dementia. This study was conducted in compliance with procedures approved by the Toronto Academic Health Sciences Network Human Subjects Research Ethics Board; all subjects provided informed consent.
Subjects participated in two sessions (pre and post-training) conducted 4 weeks apart, and then returned for a third session conducted 6 months after the training ended. Subjects were instructed to refrain from using the rehabilitation software during this 6-month post-training period. Each session took approximately 120 minutes to complete.
In the first session, participants completed a demographic questionnaire as well as a questionnaire that assessed their formal musical training, their exposure to and experience with music. They then completed the diagnostic music test and speech perception tests in quiet and noise. Listeners who used contralateral hearing aids and/or with any level of residual acoustic hearing in their contralateral ears had their ears occluded with a foam earplug. For patients with bilateral implants, the better performing ear was used at each testing time point.
Speech stimuli were presented from a single loudspeaker at 0 degrees azimuth. Speech recognition in quiet was assessed using two IEEE sentence lists presented at 65 dBA. Speech recognition in noise was assessed using two IEEE sentence lists presented at 65 dBA at a +5 dB signal-to-noise ratio (SNR) using multitalker babble.
Diagnostic Music Testing
Instructions for the HearTunes* (Diagnostic) software were then provided to each participant before completing this diagnostic music test. HearTunes* (Diagnostic) consists of three fundamental components—the music enjoyment assessment tool (Module I), aural acuity tests (Module II), and perceptual ability tests (Module III). The music enjoyment assessment tool (Module I) was created from an interaction of two variables known to affect CI recipients’ response to music: musical style and textural complexity (15–17). It contains 12 excerpts that are typical exemplars of jazz, pop, and classical music. Excerpts are 30 seconds in length, and are presented to the listener who then responds on a semantic differential scale consisting of the “affective” factors from Gfeller et al. (1).
The first part of the aural acuity tests (Module II) is a pitch-discrimination task comprised of 32 pairs of pitches. When each pair of pitches is presented, participants are required to judge whether the second pitch is higher, lower, or the same as the first pitch presented. The second part of the aural acuity tests is a timbre discrimination task comprised of 20 pairs of timbres. In this section, three families of timbre are presented—string, woodwind, and brass, with three instruments in each category. These families are paired against each other, with a total of 7 “same” pairings and 13 “different” pairings. Participants are required to judge whether the two presented timbres are the same or different from each other. This section is internally referenced; participants need not identify the names of the musical instruments being played, but rather are required to correctly identify and pair the timbres to achieve an accurate score.
The perceptual ability tasks (Module III) are made up of four increasingly complex sections where users must identify target patterns of music:
- mono pitch patterns, where simple 3- and 4-note pitch patterns are presented;
- patterns in two-line melodies, which require selective attention to select a target pattern during a continuous melody;
- patterns in three-line music, which require alternating attention in more complex melodies to select a target pattern;
- patterns in two-line music, which require divided attention to select one of two targets.
Results were collected by the software program and saved directly to a laptop hard drive. This data included percent correct and incorrect scores for all the tasks in Modules II and III and the subjective rating scores of music enjoyment (Module I).
During administration of the HearTunes* (Diagnostic) test, participants were seated in a sound-attenuated booth, 1 m in front of a speaker located at 0 degrees azimuth. The HearTunes* (Diagnostic) test was presented in sound field at 65 dBA (calibrated at CI level with a Bruel & Kjaer Sound Level Meter 2250) using a Dell notebook computer routed through a GSI 61 Audiometer to the speaker. Subjects were allowed to adjust their device volume setting to their most comfortable level.
Subjects then were introduced to the HearTunes* (Rehab) software. HearTunes* (Rehab) is a computer-based training program that was designed to place increasing demands on CI users’ attention, thereby forcing them to improve upon their listening skills and focus. It is an expansion of the HearTunes* (Diagnostic) software that uses the same principles in a rehabilitative format, with practice areas and test areas. The easiest levels consist of simple three- or four-note patterns where users are asked to pick out a target pattern out of a series of five patterns of notes. They are introduced to these five patterns in a “practice” area where they can take their time listening to the target and each of the other four “distractor” patterns before moving on to a test section. During the ensuing test, each time they hear the target pattern play they are required to click their mouse anywhere on the screen. They are provided with feedback for correct responses as well as incorrect responses, and must obtain a minimum score of 80% correct for each task to move on to the next task in each level. These levels increase in complexity up to a 9th—most difficult—level, which presents users with complex musical patterns within melodies, requiring them to distinguish variations in pitch and timbre simultaneously. In these hardest tasks users are listening for two target samples simultaneously within melodies containing multiple lines of music.
Participants were provided with a URL link so that they could download the software to use on their home computers. They were required to use this software for a minimum of 3.5 hours a week for 4 weeks. All of the participants’ data was transferred into an online database so that we were able to track their usage statistics and ensure compliance with the required amount of training each week. All participants were asked to patch into their computers directly using their speech processor cables for each at-home training session, to control for variability in computer speaker quality as well as differences in binaural hearing abilities (i.e., hearing aids, second CIs, etc.). For patients with bilateral implants, the better performing ear was used for training and at each testing time point.
At all three in-lab testing sessions, the diagnostic music test, and speech perception tests in quiet and noise were administered. The same volume level and CI program were used for all testing sessions. There were no clinical visits or CI program changes between sessions.
All statistical analyses were performed using SPSS 22 (SPSS, Inc., Chicago, IL, U.S.A.).
Musical background was based on a questionnaire determining skill level by a number of factors, such as ability to read sheet music, ability to play an instrument (and if so, the number of years that they played that instrument), ability to take direction by a band leader/conductor, etc. This questionnaire was adapted from the Preparatory Set Profile questionnaire (18).
A median-split was performed on participants’ scores, with those scoring above 60% categorized as “high” in musical ability (M = 77, SD = 8.6), and those scoring 60% or less categorized as “low” in musical ability (M = 48, SD = 12.5).
Music Discrimination Outcomes
Music Enjoyment (Module I)
All outcomes are shown in Table 2 for those low in musical ability, and Table 3 for those high in musical ability. Repeated measures ANOVAs revealed significant improvements pre- to post-training for those low in musical ability (from 55.8 to 69.3% self-reported enjoyment levels) but not for those high in musical ability. However, when participants returned for their third session 6 months after the completion of their training (refraining from using the training software during that time), an increase in musical enjoyment emerged for the high musical ability group. No differences emerged for the low musical ability group after this 6-month refractory period.
Aural Acuity Tests (Module II)
A repeated-measures ANOVA revealed statistically significant improvements in pitch pre- to post-training for those low in musical ability (from 71 to 83% correct) but not for those high in musical ability. However, when these participants returned for their third session 6 months after the completion of their training (refraining from using the training software during that time), this significant improvement disappeared in the low ability group, with percent correct scores dropping down to 74%. No differences emerged for the high musical ability group after this 6-month refractory period.
No significant differences emerged for timbre perception for either group at any of the three timepoints.
Perceptual Ability (Module III)
Participant performance for this module was recorded as a percent score for each task, summed together for the lower levels (simple pattern perception) separately from the higher levels (complex pattern perception).
Simple Pattern Perception
A repeated-measures ANOVA revealed statistically significant improvements pre- to post-training for those low in musical ability (from 35 to 75% correct) but not for those high in musical ability. In the former, 6 months after the completion of their training (refraining from using the training software during that time), average scores remained improved at 61%. No differences emerged for those high in musical ability.
Complex Pattern Perception
A repeated-measures ANOVA revealed statistically significant improvements pre- to post-training for those low in musical ability (from 19 to 60% correct), as well as for those high in musical ability (from 53 to 69% correct). In the former, 6 months after the completion of their training, average scores dropped slightly to 48%. In the latter, scores dropped to 63%.
Speech Perception Outcomes
Statistically significant differences emerged in the low musical ability group pre- to post-training for speech in quiet (from 40 to 55%) and speech in noise (from 17 to 40%). Six months later when subjects had refrained from training, scores were 44% (quiet) and 31% (noise). No differences emerged for those high in musical ability.
Studies have shown that music training of adult CI recipients can result in improvements in areas such as melodic contour identification and familiar melodies (19,20), in timbre recognition (21,22), and in music appreciation (13). However, these studies were primarily based on training software that targeted isolated musical tasks—whereas the present study used an at-home training system that included all musical domains, with an internally referenced design, and entirely novel content.
There are also several other instruments for testing the music perception of CI users. For example, the Montreal Battery for Evaluation of Amusia (MBEA) test (23) is used to assess a variety of mechanisms that underlie music perception and does not rely on memories for familiar melodies. The MBEA test has been shown to have good sensitivity, reliability, and validity based on listening tests from 160 normal-hearing listeners.
Another standardized test shown to have good test–retest reliability and adapted by Gfeller et al. (24) is known as the “Primary Measures of Music Audiation” (PMMA). It consists of two components—tonal and rhythm—each of which is comprised of 40 pairs of synthesized items. The participant is required to respond with “same” or “different” on a printed answer sheet to indicate their perception of differences in short tonal or rhythmic patterns.
The University of Washington Clinical Assessment of Music Perception (CAMP) (25) test is a computer-based test that examines pitch direction, melody recognition, and timbre recognition. It evaluates the ability to perceive the intervallic direction of pitch pairs and to identify common melodies and the sounds of various musical instruments from closed sets, respectively. Kang et al. (25) demonstrated that the CAMP test is a valid and reliable music perception test for adult CI users in a study that included 42 CI users and 10 normal-hearing listeners via moderate to strong test–retest correlations. However, one of the limitations of their methodology is that many subjects were tested twice over 2 consecutive days due to scheduling restrictions. This may have skewed the data, as shorter time intervals between tests may result in carry-over effects due to memory, practice, or mood.
The Appreciation of Music in Cochlear Implantees test, developed by Spitzer et al. (26), assesses discrimination of music versus noise, identification of musical instruments, recognition of musical styles, and recognition of individual musical pieces.
Even though test reliability is stable in some of the other music perception instruments described above, there are also several caveats. For example, one of the main shortcomings of the adapted PMMA instrument is that this test was not designed for CI users and did not cover all dimensions of music. In addition, it requires trained musical personnel to code the responses.
The CAMP test, due to lack of assessment of responsiveness (ability to detect changes), cannot be considered a valid instrument to measure changes in music perception. In the MBEA test, the pitch tests are quite difficult for CI users, and although the test addresses rhythm and pitch, it does not measure timbre acuity.
The construct validity of HearTunes* (Diagnostic)(9), followed by its test–retest validation that demonstrated no significant changes over time in any dimension (10), served as the basis in the diagnostic and rehabilitative strategies of HearTunes* (Rehab). The observed changes in performance after rehabilitation should, theoretically, be attributed to the intervention rather than test–retest variability (10). In this study, enhanced music appreciation and perception, as well as improvement in speech understanding in subsets of CI patients are more likely to be related to rehabilitation than simply training effects due to familiarization. Although many of the improvements identified were related to low musicality users, participants who had a musical background and/or greater experience with music showed improvement in tasks requiring increased attention to multiple lines of music, reflecting the increased complexity of these tasks in both perception as well as in memory and concentration. For those who are more experienced with music, more complex challenges may be required to trigger the same level of active listening and concentration that effected change in those participants who had little musical background. Furthermore, 6 months after refraining from using the software, those more experienced with music reported greater levels of enjoyment of music than they had pretraining, this may be part and parcel with enhancement in complex music perception. This requires further examination.
Research with normal hearing listeners has suggested that music experience and training can improve certain areas of cognitive functioning, including speech perception in noise (27–30). It is reasonable to theorize that skills acquired via music training are transferrable to speech perception, although to date, only correlational data has shown any relationship between music perception and speech perception in CI recipients (21,25,31,32). In this study, improvement in speech perception (both in quiet and noise) after music rehabilitation was limited to those with low music abilities. Nevertheless, the results are unique and encouraging given the short duration of training, and that training occurred, on average, 24.5 months after device activation. Earlier training and a randomized prospective trial would be justified in future renewals of this training paradigm, particularly with tonal language users.
A short rehabilitation period followed by cessation of use saw a drop in performance in low musicality users, and so those less experienced with music may need to continually train themselves to see greater long-lasting improvements or there may be a minimal amount of training that is required to see long-term change; there may be a strategy with a minimum threshold of training that is required for long-term impact; this is an area that deserves future studies. Nonetheless, the sustained overall improvement in several parameters among low musicality users is both interesting and encouraging.
HearTunes*(Rehab) is a new rehabilitative option in music perception and appreciation training for CI users. Early results suggest it may also have a positive impact on speech understanding. A larger, multicentered study is necessary to demonstrate its effectiveness, particularly in sustained benefit. Music training had the largest impact in the group of participants who had the least background with music, and so more complex and challenging versions of the software may also be necessary to develop to elicit more significant improvements in those CI users with more experience and background in music. A prospective clinical trial is needed to understand the impact of a more rigorous musical rehabilitation program in CI users for music and speech perception.
1. Gfeller K, Christ A, Knutson JF, et al. Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant
recipients. J Am Acad Audiol
2. Gfeller K, Oleson J, Knutson JF, et al. Multivariate predictors of music perception
and appraisal by adult cochlear implant
users. J Am Acad Audiol
3. Migirov L, Kronenberg J, Henkin Y. Self-reported listening habits and enjoyment of music
among adult cochlear implant
recipients. Ann Otol Rhino Laryngol
4. Looi V, Wong Y, Loo JHY. The effects of training on music perception
and appreciation for cochlear implant recipients. Adv Otolaryngol
5. Sohlberg MM, Mateer CA. Effectiveness of an attention-training program. J Clin Exp Neuropsychol
6. Madsen CK, Geringer JM. Meaningful listening and focus of attention: A model. Bull Counc Res Music Educ
7. Gfeller K, Turner C, Mehr M, et al. Recognition of familiar melodies by adult cochlear implant
recipients and normal-hearing adults. Cochlear Implants Int
8. Bartel L, Bodmer D, Shipp D, et al. Qualitative case studies of five cochlear implant recipients’ experience with music
. Cochlear Implants Int
9. Alexander AJ, Bartel L, Friesen L, et al. From fragments to the whole: A comparison between cochlear implant users and normal-hearing listeners in music perception
and enjoyment. J Otolaryngol Head Neck Surg
10. Smith L, Chen J, Bartel L, Alexander A, Amoodi H. Validating the discriminatory properties and test-retest reliability of a diagnostic music
battery for CI users: MusicEAR. Presented at the 14th Symposium on Cochlear Implants in Children, Nashville, Tennesse, December 11–13, 2014.
11. Gfeller K, Knutson JF, Woodworth G, et al. Timbral recognition and appraisal by adult cochlear implant
users and normal-hearing adults. J Am Acad Audiol
12. Fujita S, Ito J. Ability of nucleus cochlear implantees to recognize music
. Ann Otol Rhinol Laryngol
13. Gfeller K, Witt S, Woodworth G, et al. Effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal. Ann Otol Rhinol Laryngol
14. McDermott HJ. Music perception
with cochlear implants: A review. Trends Amplif
15. Gfeller K, Christ A, Knutson JF, et al. The effects of familiarity and complexity on appraisal of complex songs by cochlear implant recipients and normal hearing adults. J Music Ther
16. Gfeller K, Olszewski C, Rychener M, et al. Recognition of “real world” musical excerpts by cochlear implant recipients and normal hearing adults. Ear Hear
17. Looi V, She J. Music perception
of cochlear implant users: A questionnaire, and its implications for a music
training program. Int J Audiol
18. Bartel L. The effect of preparatory set on musical response in college students. J Res Mus Edu
19. Galvin JJ, Fu Q, Nogaki G. Melody contour identification by cochlear implant listeners. Ear Hear
20. Gfeller K, Witt S, Stordahl J, Woodworth G. The effects of training on melody recognition and appraisal by adult cochlear implant
recipients. J Acad Reabil Audiol
21. Gfeller K, Witt S, Adamek M, et al. Effects of training on timbre recognition and appraisal by postlingually deafened cochlear implant recipients. J Am Acad Audiol
22. Driscoll VD, Oleson J, Jiang D, Gfeller K. Effects of training on recognition of musical instruments presented through cochlear implant simulations. J Am Acad Audiol
23. Peretz I, Champod AS, Hyde K. Varieties of musical disorders: The montreal battery of evaluation of amusia. Ann N Y Acad Sci
24. Gfeller K, Lansing CR. Melodic, rhythmic, and timbral perception of adult cochlear implant
users. J Speech Hear Res
25. Kang R, Nimmons GL, Drennan W, et al. Development and validation of the university of washington clinical assessment of music perception
test. Ear Hear
26. Spitzer J, Mancuso D, Cheng MY. Development of a clinical test of musical perception: Appreciation of music
in cochlear implantees (AMICI). J Am Acad Audiol
27. Kraus N, Skoe E. Part CIII introduction. Ann N Y Acad Sci
28. Shahin AJ. Neurophysiological influence of musical training on speech perception
. Front Psychol
29. Musacchia G, Sams M, Skoe E, Kraus N. Musicians have enhanced subcortical auditory and audiovisual processing of speech and music
30. Musacchia G, Strait D, Kraus N. Relationships between behavior, brainstem and cortical encoding of seen and heard speech in musicians and non-musicians. Hear Res
31. Gfeller K, Turner C, Oleson J, et al. Accuracy of cochlear implant recipients on pitch perception, melody recognition, and speech reception in noise. Ear Hear
32. Limb CJ, Rubinstein JT. Current research on music perception
in cochlear implant users. Otolaryngol Clin North Am
Keywords:Copyright © 2017 by Otology & Neurotology, Inc. Image copyright © 2010 Wolters Kluwer Health/Anatomical Chart Company
Adult cochlear implant; Music; Music enjoyment; Music perception; Rehabilitation; Speech perception