Cochlear implants (CI) can provide a remarkable method of rehabilitating auditory perception and provide substantial benefit to many hearing impaired patients. Nevertheless, there is still significant variability in the speech and hearing outcomes in these patients (1,2). The signal provided by a CI is distorted in comparison to acoustic hearing and postlingual CI users must undergo some degree of adaptation to the new signal. One of the distortions they encounter is a mismatch between the acoustic frequency of the input signal and the characteristic frequency of the neurons being stimulated by the electrode array (3). Many users adapt to this mismatch over time, but the speed and ultimate extent of adaptation may vary from individual to individual.
The tonotopic organization of the cochlea is reproduced by CIs by mapping input acoustic frequencies to electrode locations within the cochlea. A frequency allocation table (FAT) is used to assign frequencies to electrodes; high frequency inputs are allocated to basal electrodes and low-frequency inputs are allocated to apical electrodes within the array. In current clinical practice, most CI users are provided with a standard or default FAT.
The actual location and characteristic frequency of neurons being stimulated by each electrode may be variable across CI users. The cochlear duct length varies among individuals, and manufacturers often offer different electrode array designs that produce varying intracochlear positions (4–6). Even if arrays may be surgically inserted with standardized methods, there is at least some degree of variation in position among patients. Given this variability in position, a default FAT may or may not be optimal for all patients (7). The human brain is able to adapt to the signal provided by a CI, but there may be a subset of CI users for whom that adaptation to the frequency mismatch may be incomplete (8–12). In these cases, an adjustment in the FAT may provide benefit for speech perception, as well as potentially other hearing outcomes (13).
One obstacle to addressing this potential frequency mismatch with changes in fitting parameters is that currently no clinical tools exist for clinicians to systematically explore different settings in a time-efficient manner. No current methods exist for rapid evaluation of many different frequency allocation settings using clinical programming software. One potential solution to this problem is to provide a portable tool that CI users can use to explore many different parameters in real time. In this report, we describe the initial steps in developing, testing, and validating a smartphone application that CI users can use to select custom FATs.
A software tool for custom FAT selection was designed for a mobile operating system (iOS, Apple Inc., Cupertino, CA). Development of this software extended and used some of the implementation of a previously developed portable application that produced an acoustic simulation of a CI speech processor (14,15). The aim for this application was to allow patients to simulate and select different FATs in real time without altering their clinical speech processor. To achieve this, the application was designed to function as a preprocessor of input acoustic signals, as a front-end to the patient's clinical speech processor. Instead of acoustic input being directly received by the clinical processor, it is taken in through the mobile device's microphone, processed within the software application, and then output directly to the clinical speech processor by a direct audio input (DAI) connection.
The input signal is processed by frequency analysis filter banks. The filter banks use sixth-order bandpass Butterworth filters with a sampling rate of 44,100 Hz. A default filter bank encompassing 22 channels across the range of 188 to 7938 Hz was created, representing one CI manufacturer's default (Cochlear Americas, Centennial, CO). The output of filter banks is full-wave rectified and low-pass filtered (using a sixth-order Butterworth filter) to extract channel envelopes. The low-pass cut-off frequency of the envelope is set at 30 Hz, and an option to change this default setting is available through the interface of the app. A group of filter banks was created to represent a range of possible FATs. To create this wide range of possible FATs, frequency ranges to be allocated were varied, using a minimum frequency cutoff across 13 steps between 63 and 1813 Hz and varying the maximum frequency edge across nine steps ranging from 3372 to 18,938 Hz. This produced a total of 117 different filter banks, each representing a different FAT. To produce an acoustic output that would simulate the chosen FAT through the clinical speech processor, the output envelope of each channel modulates a single pure tone centered at the midpoint of the corresponding channel in the default speech processor's FAT. We hypothesized that producing a single pure tone would allow the software to reliably activate any chosen channel in the clinical processor. For example, a modulated pure tone at 7438 Hz is expected to primarily activate electrode 1 (the most basal electrode) which is assigned to the range 6938 to 7938 Hz in the standard settings of the Cochlear clinical processors. The center frequency for each of the 22 channels in a default processor FAT was assigned as pure-tone outputs of the application. The software application would then shift outputs accordingly to simulate changes in FATs.
The software interface also allows the frequency of the output pure tones to be changed in case a given user's clinical FAT is nonstandard. This typically happens when electrodes are removed from the map. The output gain can also be adjusted within the graphical user interface (GUI) to be between 0 and 20 dB.
Graphical User Interface
The application's GUI presents a grid of FAT selection options that can be selected via the touch screen interface. A screenshot of the interface is shown in Figure 1. Each button represents a unique FAT, and the horizontal and vertical axes of the grid represent the boundary (highest and lowest) frequencies of that particular FAT. Changes along the x-axis alter the upper frequency edge, while movement along the y-axis changes the lower frequency edge. Movement within the grid can produce both expansion or contraction of the total frequency span being allocated, as well as shifting the midpoint of the FAT to higher or lower ranges. The frequency edges of each FAT range are also represented by two sliders in the lower portion of the interface, which correspond to the grid selection and can also be used to change selections. The application has switches for turning processing on or off, as well as to enable output as pure tones intended to stream directly to CI clinical speech processors. Selections made by a subject produce real-time effects to acoustic input received by the smartphone's microphone, so users are able to make rapid comparisons.
To investigate the validity of the software application's output when used as a preprocessor to a clinical speech processor, electrodograms were created for several acoustic stimuli and application configurations. An electrodogram is a representation of the pulse sequence that is sent to the implanted array from the speech processor in response to a given stimulus. This pulse sequence is a record of which electrodes are to be stimulated at a given point in time, and at which current amplitude. In currently available speech processors, the pulse sequence is encoded as a radiofrequency signal and transmitted from the speech processor to the internal CI receiver/stimulator where it is decoded. Using hardware to capture and decode the radiofrequency signals transmitted by a clinical speech processor, an electrodogram can be recorded for any processed acoustic input.
In this study, electrodograms of acoustic input were recorded for multiple configurations: 1) the clinical speech processor without the smartphone front-end connected; 2) the smartphone front-end connected to the clinical speech processor via DAI, but with the app in “off” mode (i.e., no additional signal processing); 3) the same condition as #2 except with the app in “on” mode using the standard FAT; and 4) the same condition as #3 except using an alternative FAT. The first testing configuration allowed for a representation of the expected output of a typical CI. The second testing configuration allowed comparisons to isolate any effects of the smartphone hardware (e.g., microphone and front end audio) alone as a preprocessor in front of the clinical processor without any signal processing or FAT alterations. The third testing configuration would reveal what changes the application's signal processing would introduce after the input signal was converted to sequences of pure tones processed through a user's clinical device. Finally, the fourth testing configuration would show the changes in stimulation patterns when an alternative FAT is selected in the app. Recordings were made using a Nucleus Freedom processor. When recording through the smartphone, microphone sensitivity was set to zero to minimize input through the microphone of the speech processor. Acoustic input to the smartphone was adjusted in level to yield electrodogram outputs with overall current level equivalent to electrodogram outputs produced when acoustic stimuli were delivered to the CI speech processor without the smartphone connected. In the latter case, stimuli were presented at 65dB-C SPL to the CI speech processor microphone using default volume and sensitivity settings. Acoustic input used for electrodogram measurements included a sinusoid swept in frequency from 100 through 8000 Hz, words, and sentences.
Subjects and Testing
A total of six adult, postlingually deafened, English-speaking CI subjects were enrolled in this pilot study (Table 1). Approval from the New York University School of Medicine Institutional Review Board was obtained. All subjects were using the manufacturer default FAT range in their daily clinical speech processors. Speech perception was measured using consonant–nucleus–consonant (CNC) word lists for each subject in four listening conditions: 1) using their clinical speech processor, 2) using the app “off,” 3) with the app “on” and using a FAT corresponding to the manufacturer's default, and 4) with the app “on” and using a subject's self-selected FAT.
Testing was conducted in a sound attenuated booth and acoustic stimuli presented over loudspeaker. All subjects were tested in a CI-only condition, with an N5 speech processor, and plugging and muffing of the contralateral ear for any subjects with residual contralateral hearing. Subjects were given the opportunity to interactively search the grid of FATs while listening to running speech to find a selection, or a group of FATs that maximized speech intelligibility. Subjects had no predetermined time limit to explore the FATs, but routinely spent approximately 20 minutes exploring the grid to make a selection. Once satisfied with their self-selected FAT, the testing materials were administered acutely, no additional time for adaptation to the FATs was provided. A survey using 10-point rating scales about sound clarity, sound quality, and listening effort for each listening condition, totaling 12 questions, was administered to each subject (see Supplemental Digital Content: Subjective Rating Form, http://links.lww.com/MAO/A516).
One subject, S1, had been using the manufacturer's default FAT for approximately 2 years. However, 2 weeks before participating in this experiment, the user had her map changed to deactivate two adjacent electrodes while using the same default range (188–7938 Hz). To accommodate this scenario, a new map for this subject was created in the processor used for testing that included the standard FAT and all 22 channels, but set the threshold and comfort levels for the two deactivated electrodes to zero. This essentially created the default allocations while not stimulating the two electrodes that had recently been deactivated, so that her FAT shifts and selections within the application would be applied to the remaining 20 electrodes. No modifications to the default output pure tones from the software interface were necessary.
Electrodograms recorded for a frequency swept sinusoid input are shown in Figure 2 for presentations to a clinical processor alone (2A), to the application while connected to the smartphone and processing is turned “off” (2B) and when turned “on” (2C). Figure 3A–D depicts electrodograms for one recorded speech input (“Ready…June.”) for the same three configurations, as well as with an alternative FAT. In both examples, the electrodograms for the processor alone and for the application “off” are similar. With the smartphone application “on,” the frequency sweep shows a very recognizable pattern of pulse sequences; however, there is greater overlap in stimulation of adjacent electrodes. For a clinically relevant speech input, such as a CNC word presentation in Figure 3, the patterns across the three conditions seem to be very similar. In Figure 3D, the electrodogram with an alternate FAT is shown. This alternative FAT allocates a wider frequency range, using a higher upper frequency edge (188–18,938 Hz) than the standard FAT. With this alternative FAT including a larger high-frequency range, we would expect the stimulation pattern to be shifted toward the apical end of the array (higher electrode numbers). A comparison of Figure 3C and D shows this shift and a “compression” of the pulse sequence pattern toward the apical end of the array, showing adjustments to the FAT in the software app produced expected shifts to the stimulation pattern. Electrodograms in response to sentences showed similar results to those observed for words.
Every subject successfully used the smartphone app to self-select a custom FAT during their testing session and every individual self-selected a FAT that was different from the manufacturer default. The mean CNC word score using clinical processors was 43.3% (SD 17.5) and 44.5% (SD 17.8) when using the smartphone app in the “off” mode. With the app “on,” the mean CNC score using the default FAT selection was 28.5% (SD 16.8) and 29.5% (SD 16.4) when using a self-selected FAT. A comparison of these scores is shown for each individual in Figure 4.
All subjects completed the questionnaire rating their experience in the four conditions—a clinical speech processor, the app while turned “off,” and the app while “on” with a standard FAT and with a self-selected FAT. In rating the clarity of sound, where a higher score denoted better clarity, mean responses were 7.1 (SD 1.5) using a speech processor and 6.8 (SD 1.8) when using the app “off.” With the app connected and “on,” the mean user rating for the standard FAT was 4.8 (SD 2.4) and 5.7 (SD 1.1) when using their self-selected maps. For sound quality, where again higher scores denoted better quality, for the speech processor the mean rating was 6.5 (SD 1.6) and 6.3 (SD 2.1) with the app “off.” When the app was “on” with a standard FAT the mean rating was 4.0 (SD 2.3) compared with 4.2 (SD 2.1) with the self-selected FAT. Finally, when reporting the degree of listening effort, for which a higher score denoted a less desirable rating (more effort required), with the clinical processors the mean score was 5.3 (SD 1.9) compared with 5.8 (SD 1.7) with the app “off.” With the app “on” and the standard FAT, the mean effort rating was 7.6 (SD 2.1) compared with 6.8 (SD 1.9) using the self-selected FAT. Figure 5 shows the comparisons for the conditions with the application while “on.”
In this study, our goal was to develop a new mobile tool for CI recipients to explore changes to their FATs. In previous work, a desktop-based system had been developed to allow for self-selection of FATs (16). That system, however, relied upon manufacturer-specific hardware and was logistically limited to a research environment. The tool in the present study was developed as a mobile smartphone application, allowing for use on widely available hardware and affording portability for potential use in environments outside of an audiology facility or laboratory setting. The application is not intended to alter stimulation parameters long term, but rather as a means for CI users to explore different FATs in their natural listening environments that may aid their audiologist in determining beneficial changes with standard clinical fitting software.
The application serves as a preprocessor for acoustic input and simulates changes of speech processor fitting parameters but does not alter the clinical speech processor in any way. With this method of using portable software as a front-end processor, some distortion in the intended stimulation pattern delivered to CI users is expected. Assessing that these changes do not markedly distort the original signal, and that the changes are consistent is an important part of assessing the feasibility of this smartphone software. Electrodograms allowed for the comparison of pulse sequences for various stimuli, and showed that the method of preprocessing the signal does produce some changes to the final pulse sequences, but for speech input produces a reliable representation. Even when the application preprocesses the signal and outputs an altered acoustic signal composed of 22 pure tones, this purposefully designed new signal is processed through the clinical device to produce an electrode stimulation pattern that maintains a close representation of what occurs without the smartphone. It can be expected that the addition of a front-end processor may degrade performance to some degree, and this is seen in the results of word recognition and subject ratings of their experience in comparison with a clinical processor alone. The purpose of the tool however is not to improve performance with its output directly, but rather to allow for meaningful user exploration of parameters that may be relevant to actual clinical fitting. Even though the pulse sequences are not identical to a clinical processor, the application does provide a baseline from which user-directed selections can produce useful relative comparisons between FAT selections.
Although a simple observation, the fact that all tested subjects could successfully navigate the smartphone application as a front-end to their CI and successfully self-select a preferred FAT was meaningful for the feasibility of this novel tool for clinical use. The similarity between word recognition using a clinical processor and the application while “off” was expected, but did confirm a lack of significant clinical changes due to the introduction of the smartphone hardware and DAI connection. Similarly, the patient questionnaire showed very similar results for sound clarity, quality, and listening effort in these two conditions.
Among other signal distortions, CI users adapt to a frequency mismatch between the frequency of input auditory signals and the characteristic frequency of the neurons being stimulated. At activation, CI users may experience a basalward shift of acoustic input (perceiving tones at a higher frequency than the incoming acoustic signal) but over time can adapt to these mismatches to at least some degree (17,18). Changes to the frequency-to-electrode allocations may affect the process of adaptation or performance, and one possibility of addressing this is to allow listeners to self-select parameters in real time (7,19–21). This new smartphone application allows for an innovative method of comparing FATs in real time. In this initial study, there was relative similarity in word recognition scores between the standard FAT and self-selected FATs. Every subject did choose a different FAT from the standard FAT however, implying a potential uncompensated frequency mismatch. Some potential factors for these results include the small number of subjects, as well as the acute experimental design that does not allow for potential adaptation to a self-selected FAT. It is possible that for some users, the benefits of frequency allocation changes may be reflected in the clarity or quality of sound that is not necessarily measurable through speech recognition testing. In this group, no significant differences in ratings between the self-selected and standard FATs were observed.
The smartphone application developed in this study takes a novel approach to simulating changes to CI fitting parameters for patients and has shown feasibility for self-selection of FATs in CI users. Although not addressed in this initial study, the capability for users to explore FAT changes in real-world settings is a potentially significant advantage of this new software platform. The smartphone application as a preprocessor may also provide a method for investigating other fitting parameters in addition to frequency allocations. This software application may be a valuable tool for improving future methods of CI fitting for individual patients.
The authors thank Christopher J. Smalt, PhD, for assistance and whose original work on a portable acoustic simulation of CI was the basis for this project. The authors also thank the assistance of Jonathan Neukam, AuD in subject testing and data acquisition.
1. Holden LK, Finley CC, Firszt JB, et al. Factors affecting open-set word recognition in adults with cochlear implants. Ear Hear
2. Blamey P, Artieres F, Baskent D, et al. Factors affecting auditory performance of postlinguistically deaf adults using cochlear implants: an update with 2251 patients. Audiol Neurotol
3. Landsberger DM, Svrakic M, Roland JT, Svirsky M. The relationship between insertion angles, default frequency allocations, and spiral ganglion place pitch in cochlear implants. Ear Hear
4. Hardy M. The length of the organ of Corti in man. Am J Anat
5. Miller JD. Sex differences in the length of the organ of Corti in humans. J Acoust Soc Am
6. van der Marel KS, Briaire JJ, Wolterbeek R, Snel-Bongers J, Verbist BM, Frijns JH. Diversity in cochlear morphology and its influence on cochlear implant
electrode position. Ear Hear
7. Fitzgerald MB, Sagi E, Morbiwala TA, Tan C-T, Svirsky MA. Feasibility of real-time selection of frequency tables in an acoustic simulation of a cochlear implant
. Ear Hear
8. Fu Q-J, Shannon RV, Galvin JJ. Perceptual learning following changes in the frequency-to-electrode assignment with the Nucleus-22 cochlear implant
. J Acoust Soc Am
9. Harnsberger JD, Svirsky MA, Kaiser AR, Pisoni DB, Wright R, Meyer TA. Perceptual “vowel spaces” of cochlear implant
users: implications for the study of auditory adaptation to spectral shift. J Acoust Soc Am
2001; 109 (5 pt 1):2135–2145.
10. Svirsky MA, Silveira A, Suarez H, Neuburger H, Lai TT, Simmons PM. Auditory learning and adaptation after cochlear implantation: a preliminary study of discrimination and labeling of vowel sounds by cochlear implant
users. Acta Otolaryngol (Stockh)
11. Sagi E, Fu Q-J, Galvin JJ, Svirsky MA. A model of incomplete adaptation to a severely shifted frequency-to-electrode mapping by cochlear implant
users. J Assoc Res Otolaryngol JARO
12. Svirsky MA, Fitzgerald MB, Sagi E, Glassman EK. Bilateral cochlear implants with large asymmetries in electrode insertion depth: Implications for the study of auditory plasticity. Acta Otolaryngol (Stockh)
13. Gani M, Valentini G, Sigrist A, Kós M-I, Boëx C. Implications of deep electrode insertion on cochlear implant
fitting. J Assoc Res Otolaryngol JARO
14. Smalt CJ, Talavage TM, Pisoni DB, Svirsky MA. Neural adaptation and perceptual learning using a portable real-time cochlear implant
simulator in natural environments. Conf Proc IEEE Eng Med Biol Soc
15. Smalt CJ, Gonzalez-Castillo J, Talavage TM, Pisoni DB, Svirsky MA. Neural correlates of adaptation in freely-moving normal hearing subjects under cochlear implant
acoustic simulations. NeuroImage
16. Jethanamest D, Tan C-T, Fitzgerald MB, Svirsky MA. A new software tool
to optimize frequency table selection for cochlear implants. Otol Neurotol
17. McDermott H, Sucher C, Simpson A. Electro-acoustic stimulation. Acoustic and electric pitch comparisons. Audiol Neurootol
2009; 14 (suppl 1):2–7.
18. Reiss LA, Turner CW, Karsten SA, Gantz BJ. Plasticity in human pitch perception induced by tonotopically mismatched electro-acoustic stimulation. Neuroscience
19. Svirsky MA, Talavage TM, Sinha S, Neuburger H, Azadpour M. Gradual adaptation to auditory frequency mismatch. Hear Res
20. Li T, Galvin JJ, Fu Q-J. Interactions between unsupervised learning and the degree of spectral mismatch on short-term perceptual adaptation to spectrally shifted speech. Ear Hear
21. Svirsky MA, Silveira A, Neuburger H, Teoh S-W, Suárez H. Long-term auditory adaptation to a modified peripheral frequency map. Acta Otolaryngol (Stockh)
Cochlear implant; Frequency allocation table; Self-selection; Smartphone; Software tool
Supplemental Digital Content
Copyright © 2017 by Otology & Neurotology, Inc. Image copyright © 2010 Wolters Kluwer Health/Anatomical Chart Company