Whereas the current generation of cochlear implants allows a rather high level of language comprehension for most cochlear implant users (CIU), listening to music is still challenging for many CIU and leads to reduced musical appraisal (1–3) [4–8] (9–13)
The latter paradigms are particularly suitable as they do not require the participants’ behavioral response but rather give an objective measurement of CIU’ distinctive skills. An event-related potential component that is particularly sensitive to discrimination is the mismatch negativity (MMN). In an oddball paradigm two types of stimuli are presented with different probabilities. In case of discrimination, the brain's response to the rare stimulus in a series of frequent standard stimuli is characterized by a greater negativity, the MMN (14,15)
Recently, a multifeature paradigm was developed for MMN measurement which combines several features to be tested within a single session without needing additional time (16,17) (9) (18) (11) (10)
In the present study, we also examined these four basic parameters, namely pitch, timbre, intensity, and rhythm and used two different degrees of deviation. As opposed to previous studies we directly compared three groups of CIU who were all implanted as adults but had completely different requirements with regard to CI usage and learning perspectives: 1) bilaterally, prelingually deafened adult CIU (preCIU), 2) bilaterally hearing impaired postCIU, and 3) adults with a unilateral deafness and normacusis on the contralateral side. The unilateral deafness was acquired postlingually, i.e., in adulthood. Thus, groups 2 and 3 differed only with regard to the contralateral hearing condition. However, this difference has important implications. On one hand, group 2 is compelled to hear speech and music via the cochlear implant, whereas group 3 uses the CI as additional information and can rely on the nh ear in nonchallenging listening conditions. On the other hand, training with the cochlear implant is mainly confined to explicit training situations (via additional facilities like induction or cable) in group 3, whereas group 2 permanently encounters training in daily routine. These differences may be reflected in their discrimination abilities. In general, we expected higher MMN effects for the greater deviations and smaller effects for CIU than for nh controls.
METHODS
Participants
Thirty-six CIU (
= 52.9 yr;
= 53 yr; range: 23–80 yr, 19 women) participated in the two experiments (cf. Table 1 A). The implant experience ranged from 4 to 15 months; mean 11 months (with the exception of subject 1 who had over 3 years of hearing experience with his tested CI device). Seventeen participants had a postlingually, 12 a prelingually acquired bilateral severe-to-profound hearing loss. Seven participants showed nh on one side (mean threshold at 0.5, 1, 2, and 4 kHz better than 30 dB hearing level) and had a postlingually acquired deafness on the contralateral side (single-sided deafness [SSD]).
TABLE 1: A , Subject data of the CI-user groups (SSD, single-sided deafness); B , parameter-setting for both experiments (ISI, interstimulus interval)
PreCIU were defined as having suffered from a bilateral profound hearing loss already during early childhood resulting in articulatory and phonetic deficits. CIU in the postlingual group have had nh abilities during childhood and adolescence and acquired their bilateral hearing problems only as adults. Many participants used a conventional hearing aid on one side. However, they were not able to understand speech without any hearing equipment.
Seven CI patients were implanted sequentially bilateral but testing was conducted using only one CI matching the above-mentioned experience time. A group with bilaterally nh participants matched for age and sex served as controls (
= 53 yr;
= 54.5 yr; range, 25–81 yr).
Comparison of age by means of a one-factorial ANOVA showed that the mean age differed significantly between the postCIU including their matched controls (63.5 yr) and the preCIU and their respective controls (41.5 yr).
All participants in both groups were nonmusicians according to the following criteria: no musical training before age seven, no musical training longer than 10 years, no musical practicing more often than three times a week within the 3 years before the study (19,20)
Stimuli and Study Design
The study design was adapted from the multifeature MMN paradigm by Vuust et al. (18)
Experiment I
Experiment I slightly modified the Vuust et al. paradigm using the following deviants (see Table 1B): 1) pitch deviant, which was created by downtuning the standard note by 24 ct (= 0.24 semitones [ST]), 2) timbre deviant, which was created by applying the “old-time radio” effect by means of Adobe Audition (Adobe Systems Incorporated, San Jose, California, U.S.A.), 3) intensity deviant, with a reduced intensity by 6 dB compared with the standard note, and 4) rhythm deviant, which was created by anticipating the standard note by 30 ms. Each tone was presented in stereo, with a sample frequency of 44.1 kHz, a tone duration of 200 ms, and then an interstimulus interval (ISI) of 5 ms. For the rhythm deviant the note before the third note was shortened to 170 ms and the ISI after the third note was extended to 35 ms.
The standard pattern consisting of piano sound tones was presented alternating with pseudorandomized deviant four-tone patterns in measures, where the third note was modified (see Fig. 1 A). At the beginning, 30 standards were presented to establish a corresponding representation. Every fifth measure the key was changed. The order of the six applied major keys (F, Bb, Eb, Ab, Db, Gb) was also pseudorandomized. All notes were within the range from Eb3 (156 Hz) to Db5 (554 Hz) and thus well represented in a standard cochlear implant stimulation mode under presumably constant pitch discrimination thresholds (11)
FIG. 1: Example of two measures in Experiment I (A ) containing two standards and a deviant coupled with a standard; and triads in octave position as used in Experiment II (B ); modified note in deviants marked with “x.”
Experiment II
Experiment II applied the same setting except that the stimuli were slightly modified. The four-tone pattern was presented with the triads in octave position. This means that the first note (bass note) was transposed by an octave upward, thus appearing as the top note in the third position (see Fig. 1 B). This adjustment was selected as behavioral pretesting with CIU revealed that deviance detection was easier if the deviant appeared on the highest note. Additionally, the tone duration was stretched to 210 ms resulting in a reduced tempo of 140 bpm.
The deviants were also modified: deviant (1), pitch was created by transposing the third note by 100 cent (1 ST) upward, deviant (2), timbre was created by replacing the standard “piano” with a “harp” sound, the deviant (3), intensity was created by reducing the intensity by 12 dB compared with the standard, and deviant (4), rhythm was created by delaying the third note by 105 ms. This was achieved by extending the preceding ISI by 105 ms and shortening the third note to 105 ms (see Table 1B). Again 180 deviant stimuli of each category were presented resulting in an experimental duration of 20.8 minutes.
Procedure
All participants completed both experiments in one session starting always with Experiment I. They were seated comfortably in front of two loudspeakers (nEar 05 eXperience, ESI-Audiotechnik GmbH, Leonberg, Deutschland) and a PC screen. Participants were told to concentrate on a silent documentary movie running on screen during the experiments or to read a text, which they were free to choose. Stimuli were presented by means of Presentation Software (Neurobehavioral Systems, Inc., Berkeley, CA, U.S.A.) via two loudspeakers. If the bilateral severe-to-profound CIU were still possessing residual hearing on their contralateral ear, this ear was damped with earmuffs (Bilsom 303L, Howard Leight, Honeywell International, Inc., Morristown, NJ, U.S.A.). For the SSD group, stimuli were presented via audio cable to ensure that they were processed only by the cochlear implant.
Electroencephalographic Recording and Data Preprocessing
Electroencephalography was recorded from nine Ag/AgCl electrodes fixed into an elastic cap (EASYCAP, Herrsching, Germany) according to the International 10–20 System (Fz, Cz, Pz, F3, C3, P3, F4, C4, and P4). Further electrodes were placed on the nose and both mastoids and served selectively as reference electrodes. To control overlaying muscle potentials resulting from eye ball movements or eye-blinks an electrooculogram was recorded: for the horizontal electrooculogram two electrodes besides the right and the left eye, for representation of vertical electrical currents two electrodes above and below the right eye were positioned. The biosignals were recorded with a 32-channel-amplifier (ANT neuro, Enschede, The Netherlands) with electrode Cz as online reference and a sampling rate of 512 Hz. All electrode impedances were kept below 5 kΩ. Further processing of the data was done with software EEProbe by ANT Neuro (Enschede, The Netherlands). For processing, the data were downsampled offline to 256 Hz and then filtered with a band pass filter of 1 to 30 Hz. For the determination of latency and amplitude of MMN potentials data were rereferenced to the mastoid electrode opposite the implant site. To detect possible polarity inversions at the mastoids, the signal of the nose electrode served as reference in additional analyses. Trials including artifacts were excluded from averaging (approximately 20%).
Data Analysis
For both experiments potentials of the four deviants and the standard were averaged in the time window from −100 to 400 ms relative to the onset of the third note of the four-tone pattern. The section of 100 ms before the critical tone served as baseline. Grand averages for the CI group as well as for the control group were computed separately for each condition and difference waves (deviant minus standard) were calculated. MMN amplitudes and latency measures were based on these difference waves. For quantification the most negative peak at Fz (with mastoid reference) in a time window from 100 to 260 ms from onset of the deviant note was determined in individual data. The respective latency served as MMN peak latency value and the amplitude value was calculated in a 20 ms window around the respective maxima. Latency values for the rhythm condition were adjusted to account for the onset shift of the deviance.
First, a global analysis including the within-subject variables experiment (Experiment I; Experiment II), deviant type (pitch, timbre, intensity, and rhythm), laterality (left, middle, right), and frontality (anterior, central, and posterior) and the between-subject variable group (CIU, controls) was conducted for amplitude values. In case of a reliable main effect of the variable “experiment,” data for the two experiments were evaluated separately. Additional analyses were performed including the variable “deafening” to evaluate possible differences across the three CI subgroups (pre- and postlingually deafened, and postlingually deafened SSD).
To determine significant MMN potentials for each deviant and group, two-sided t tests were computed which compared the amplitudes of the standard to the amplitudes of each deviant at Fz with mastoid reference and at the mastoid electrodes with nose reference.
RESULTS
Amplitudes
The global analysis of mean amplitudes revealed highly reliable main effects for all four within-subject factors as well as for the between-subject factor group (all p <0.0001). Therefore, the subsequent analyses were performed separately for each experiment.
Experiment I
CIU differed significantly from the control group in all deviant conditions except for “timbre” (see Fig. 2 A and Table 2A) reflecting smaller amplitudes than nh participants. The t test results for the MMN effect for the different deviants (see Table 3A) showed significant MMNs for timbre, rhythm, and intensity in the CIU group. Additional analyses separately for the three subgroups of CIU did not reveal any reliable effect of the variable deafness (pitch: F[2,33] = 1.43, p < 0.26; timbre: F <1; intensity: F[2,33] = 1.51, p <0.24; rhythm: F <1).
FIG. 2: A , Experiment I: grand averages of event-related potentials for all four deviants (pitch, timbre, intensity, and rhythm) for CI users and normal hearing controls (NH). Upper three rows of panels show responses to standard (solid line ) and to the deviants (dotted line ) recorded at the frontal electrode Fz referenced to the mastoids and at both mastoid electrodes (left/ML and right/MR) referenced to the nose electrode. Difference waveforms (deviant minus standard) at Fz referenced to the mastoids are given below the horizontal dashed line , respectively. The corresponding data for Experiment II are displayed in (B ).
TABLE 2: Effect of between-subject-factor “group” (CI versus controls) in all four deviant conditions in Experiments I (A) and II (B) for amplitude measurements
TABLE 3: A and B , Amplitudes of MMN effects (deviant minus standard) with standard deviations (SD) and mean latencies at FZ, left and right mastoids (ML/MR)
Experiment II
Analysis of the between-subject factor “group” revealed significant main effects for the deviants pitch, timbre, and intensity (see Table 2B) reflecting smaller MMN effects in CIU than controls in these three conditions (see Fig. 2 B). By contrast, rhythm did not significantly differ across groups. The t tests revealed that the controls showed highly significant negativities at Fz and reversed effects at the mastoids for all deviant conditions. Although the overall amplitudes were smaller than for controls, the CI group also presented a highly significant frontal negativity in all conditions. However, the reversal at the mastoids was lateralized to the left mastoid for the pitch and the intensity deviant (see Table 3B). The analyses of the different groups of CIU did not show significant effects for the conditions timbre, intensity, and rhythm (all F <1) but a reliable effect for pitch (F(2,33) = 3.88, p <0.05). Additional analyses revealed that the negativity was significant for all subgroups of CIU with the postlingual group showing the largest and the preCIU showing the smallest MMN effect (see Fig. 3 ).
FIG. 3: Experiment II: grand averages of event-related potentials for all four deviants (pitch, timbre, intensity, and rhythm) for CI user subgroups (prelingually and postlingually deafened CI users as well as CI users with a postlingually acquired single side deafness [SSD]). Panels show responses to standard (solid line ) and to deviants (dotted line ) recorded at Fz and the respective difference waveforms to the right.
Latencies
A global analysis across experiments showed a main effect of the variable experiment (F[1,71] = 4.53, p <0.04), a reliable interaction of experiment and condition (F[3,213] = 4.97, p <0.003), but no main effect of condition (F[1,71] = 1.23, p <0.30). Subsequent analyses were conducted separately for each experiment.
Experiment I did not reveal any significant latency differences across groups or deviant types (all F < 1). Analyses of Experiment II showed a tendency toward shorter latencies for controls compared with CIU (F[1,70] = 3.2, p <0.08) and a reliable main effect of deviance type (F[3,210] = 5.98, p <0.001) but no interaction between the two. Further analyses demonstrated that the rhythm deviant elicited a slightly shorter difference potential than the other deviant conditions.
DISCUSSION
In two experiments we used a multifeature paradigm with musical stimuli varying with regard to pitch, timbre, intensity, and rhythm. The amount of deviation was higher in Experiment II than in Experiment I. As expected, larger MMN effects were elicited in Experiment II than in Experiment I. This was the case for the nh group as well as for CIU. Interestingly, there were significant MMN effects for intensity, timbre, and rhythm even in Experiment I although the physical deviations were very small. However, effects were generally smaller in the CI group compared with the control group. This was especially true for pitch but also for intensity and rhythm.
Of interest is the comparison with the recently published studies of 12 teenaged CIU with relatively low auditory input during early childhood by Petersen (10) (11)
With regard to pitch processing, the previous CI studies showed reliable effects for a postlingual group (Timm et al.) but not for a prelingual adolescent group (Petersen et al.). Our pitch deviant in Experiment II of +1 semitone elicited a good MMN potential, whereas deviation of −24 ct in Experiment I did not, independent of the patient history (pre- versus postlingual). We conclude that the real performance optimum of pitch discrimination of the CI stimulation might be still somewhat below 1 semitone. This ability may not be developed sufficiently in adolescence.
Both of our timbre deviants as well as both intensity deviants (−6 and −12 dB) elicited strong and stable MMN effects with bilateral mastoid inversions and thus further corroborate previous MMN data. These two tonal features seem to be easily processed by CIU. Note that automatic deviance detection as displayed by MMN responses has to be discriminated from musical appraisal. PostCIU often complain about deficits in recognizing typical timbre characteristics for example of instruments. This may be true despite rather fine-grained deviance detection mechanisms (see also [20]
Surprisingly, our rhythm deviants—even the one from Experiment I (−30 ms)—were both linked with strong MMN responses. This contrasts the data for adult CIU published by Timm et al. where no MMN was found for an anticipated tone by 60 ms. However, rhythmic processing is usually thought to be largely unimpaired in CIU as the implant provides a high temporal resolution and has limitations rather with regard to the spectral resolution. Behavioral studies seem to support this view (e.g., [21]
The MMN latencies in Experiment I of our nh controls were slightly higher than in the previous related studies (22–24)
Three subgroups of CIU participated in our study: bilaterally hearing impaired adults with an onset of severe or profound deafness either pre- or postlingually and a single-sided deafness group. Surprisingly, there were hardly any differences across CI groups despite their large differences with regard to their hearing history and the relevance of their implant in everyday life. Apparently, CIU with severe hearing deficits during their language acquisition period seem to benefit as much as CIU with only short-term, postlingually acquired severe hearing deficits with regard to discrimination of basic tonal properties in a musical context.
Detrimental to the analysis of CI subgroups might have been the allocation of subjects to different groups. The degree of hearing impairment in early childhood—30 or more years ago—is oftentimes not accurately documented by audiometric testing. Therefore, inclusion into the preCIU group was based not only on a detailed assessment of the subjects individual medical history but also on the presence of phonematic speech peculiarities being likely correlated with a profound hearing impairment during the sensitive period of speech development.
CONCLUSION
The present study reveals a further confirmation of the feasibility of the multifeature paradigm for musical stimuli developed by Vuust et al. (18) (24)
REFERENCES
1. Gfeller K, Witt S, Adamek M, et al Effects of training on timbre recognition and appraisal by postlingually deafened cochlear implant recipients.
J Am Acad Audiol 2002; 13:132–145.
2. Looi V, Gfeller K, Driscoll V. Music appreciation and training for cochlear implant recipients: A review.
Semin Hear 2012; 33:307–334.
3. Migirov L, Kronenberg J, Henkin Y. Self-reported listening habits and enjoyment of music among adult cochlear implant recipients.
Ann Otol Rhinol Laryngol 2009; 118:350–355.
4. Gfeller K, Witt S, Woodworth G, Mehr MA, Knutson J. Effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal.
Ann Otol Rhinol Laryngol 2002; 111:349–356.
5. Jung KH, Won JH, Drennan WR, et al Psychoacoustic performance and music and speech perception in prelingually deafened children with cochlear implants.
Audiol Neurootol 2012; 17:189–197.
6. Gfeller K, Turner C, Oleson J, et al Accuracy of cochlear implant recipients on pitch perception, melody recognition, and speech reception in noise.
Ear Hear 2007; 28:412–423.
7. Mitani C, Nakata T, Trehub SE, et al Music recognition, music listening, and word recognition by deaf children with cochlear implants.
Ear Hear 2007; 28 (2 Suppl):29S–33S.
8. Koelsch S, Wittfoth M, Wolf A, Müller J, Hahne A. Music perception in cochlear implant users: An event-related potential study.
Clin Neurophysiol 2004; 115:966–972.
9. Sandmann P, Kegel A, Eichele T, et al Neurophysiological evidence of impaired musical sound perception in cochlear-implant users.
Clin Neurophysiol 2010; 121:2070–2082.
10. Petersen B, Weed E, Sandmann P, et al Brain responses to musical feature changes in adolescent cochlear implant users.
Front Hum Neurosci 2015; 9:7.
11. Timm L, Vuust P, Brattico E, et al Residual neural processing of musical sound features in adult cochlear implant users.
Front Hum Neurosci 2014; 8:181.
12. Torppa R, Huotilainen M, Leminen M, Lipsanen J, Tervaniemi M. Interplay between singing and cortical processing of music: A longitudinal study in children with cochlear implants.
Front Psychol 2014; 5:1389.
13. Rahne T, Plontke SK, Wagner L. Mismatch negativity (MMN) objectively reflects timbre discrimination thresholds in normal-hearing listeners and cochlear implant users.
Brain Res 2014; 1586:143–151.
14. Näätänen R. The mismatch negativity: A powerful tool for cognitive neuroscience.
Ear Hear 1995; 16:6–18.
15. Duncan CC, Barry RJ, Connolli JF, et al Event-related potentials in clinical research: Guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400.
Clin Neurophysiol 2009; 120:1883–1908.
16. Näätänen R, Pakarinen S, Rinne T, Takegata R. The mismatch negativity (MMN): Towards the optimal paradigm.
Clin Neurophysiol 2004; 115:140–144.
17. Pakarinen S, Takegata R, Rinne T, Huotilainen M, Näätänen R. Measurement of extensive auditory discrimination profiles using the mismatch negativity (MMN) of the auditory event-related potential (ERP).
Clin Neurophysiol 2007; 118:177–185.
18. Vuust P, Brattico E, Glerean E, et al New fast mismatch negativity paradigm for determining the neural prerequisites for musical ability.
Cortex 2011; 47:1091–1098.
19. Pretorius LL, Hanekom JJ. Free field frequency discrimination abilities of cochlear implant users.
Hear Res 2008; 244:77–84.
20. Sandmann P, Kegel A, Eichele T, et al Neurophysiological evidence of impaired musical sound perception in cochlear-implant users.
Clin Neurophysiol 2010; 121:2070–2082.
21. Won JH, Drennan WR, Kang RS, Rubinstein JT. Psychoacoustic abilities associated with music perception in cochlear implant users.
Ear Hear 2010; 31:796–805.
22. Parbery-Clark A, Skoe E, Kraus N. Musical experience limits the degradative effects of background noise on the neural processing of sound.
J Neurosci 2009; 29:14100–14107.
23. Parbery-Clark A, Skoe E, Lam C, Kraus N. Musician enhancement for speech-in-noise.
Ear Hear 2009; 30:653–661.
24. Petersen B, Weed E, Sandmann P, et al Brain responses to musical feature changes in adolescent cochlear implant users.
Front Hum Neurosci 2015; 9:7.
