Share this article on:

Outcomes of Children with Hearing Loss: Data Collection and Methods

Tomblin, J. Bruce1; Walker, Elizabeth A.1; McCreery, Ryan W.2; Arenas, Richard M.1; Harrison, Melody3; Moeller, Mary Pat2

doi: 10.1097/AUD.0000000000000212
Research Articles

Objectives: The primary objective of this article was to describe recruitment, data collection, and methods for a longitudinal, multicenter study involving children with bilateral mild to severe hearing loss. The goals of this research program were to characterize the developmental outcomes of children with mild to severe bilateral hearing loss during infancy and the preschool years. Furthermore, the researchers examined how these outcomes were associated with the child’s hearing loss and how home background and clinical interventions mediated and moderated these outcomes.

Design: The participants in this study were children who are hard of hearing (CHH) and children with normal hearing (CNH) who provided comparison data. CHH were eligible for participation if (1) their chronological age was between 6 months and 7 years of age at the time of recruitment, (2) they had a better-ear pure-tone average of 25 to 75 dB HL, (3) they had not received a cochlear implant, (4) they were from homes where English was the primary language, and (5) they did not demonstrate significant cognitive or motor delays. Across the time span of recruitment, 430 parents of potential children with hearing loss made contact with the research group. This resulted in 317 CHH who qualified for enrollment. In addition, 117 CNH qualified for enrollment. An accelerated longitudinal design was used, in which multiple age cohorts were followed long enough to provide overlap. Specifically, children were recruited and enrolled continuously across an age span of 6.5 years and were followed for at least 3 years. This design allowed for tests of time (period) versus cohort age effects that could arise by changes in services and technology over time, yet still allowed for examination of important developmental relationships.

Results: The distribution of degree of hearing loss for the CHH showed that the majority of CHH had moderate or moderate-to-severe hearing losses, indicating that the sample undersampled children with mild HL. For mothers of both CHH and CNH, the distribution of maternal education level showed that few mothers lacked at least a high school education and a slight majority had completed a bachelor’s degree, suggesting that this sample of research volunteers was more advantaged than the United States population. The test battery consisted of a variety of measures concerning participants’ hearing and behavioral development. These data were gathered in sessions during which the child was examined by an audiologist and a speech-language examiner. In addition, questionnaires concerning the child’s behavior and development were completed by the parents.

Conclusion: The Outcomes of Children with Hearing Loss study was intended to examine the relationship between variation in hearing ability across children with normal and mild to severe hearing loss and variation in their outcomes across several domains of development. In addition, the research team sought to document important mediators and moderators that act between the hearing loss and the outcomes. Because the study design provided for the examination of outcomes throughout infancy and early childhood, it was necessary to employ a number of different measures of the same construct to accommodate changes in developmental performance across age. This resulted in a large matrix of measures across variable types and developmental levels, as described in this manuscript.

The primary objective of this article was to describe subject recruitment, data collection, and methods for a multicenter, longitudinal study involving young children with mild to severe hearing loss (Outcomes of Children with Hearing Loss; OCHL).Supplemental Digital Content is available in the text.

1Department of Communication Sciences and Disorders, University of Iowa, Iowa City, Iowa, USA; 2Center for Audiology, Boys Town National Research Hospital, Omaha, Nebraska, USA; 3Department of Speech and Hearing Sciences, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, USA; and 4Center for Childhood Deafness, Boys Town National Research Hospital, Omaha, Nebraska, USA.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and text of this article on the journal’s Web site (www.ear-hearing.com).

The authors have no conflicts of interest to declare. The authors had full editorial control of this work and manuscript.

Received December 2, 2014; accepted June 24, 2015.

Address for correspondence: J. Bruce Tomblin, Wendell Johnson Speech and Hearing Center, 250 Hawkins Drive, Iowa City, IA 52242, USA. E-mail: j-tomblin@uiowa.edu

Back to Top | Article Outline

INTRODUCTION

In the first article (Moeller & Tomblin 2015, this issue, pp. 4S–13S), we described the motivation and goals of the Outcomes of Children with Hearing Loss (OCHL) program: to characterize the developmental outcomes of children with mild to severe bilateral hearing loss (HL) during infancy and the preschool years. (For a Table of Acronyms, see Supplemental Digital Content, Appendix A, http://links.lww.com/EANDH/A202.) Furthermore, we wanted to examine how these outcomes were associated with the child’s HL and how home background and clinical interventions mediated and moderated these outcomes. To address these goals, we employed concepts and methods of two closely related fields of research. One of these consisted of theories and methods in child development concerned with risk and protective factors with respect to developmental outcomes. In this case, HL can be viewed as a risk factor that may lead to compromised well-being. The risk posed by HL can be the result of mediators that causally link the reduced audibility of speech to poorer speech and language development. The realization of this risk may be affected by moderators that interact with the relationship between the risk and the outcome. Moderators may be either clinical interventions such as hearing aid (HA) use, receipt of speech and language services, or ambient factors in the child’s life such as parental talk.

The second field of research we drew upon involves the effectiveness of health services on patient well being. Programs of Health Services Outcomes Research (Tomblin & Hebbeler 2007) address concerns with variations in clinical service provision on patient outcomes (Youngs & Wingerson 1995). This research is focused on the effectiveness of clinical interventions within observational research methods and can be incorporated into a risk-resilience model for child development by treating clinical interventions as a moderator of a health risk factor. Thus, in the OCHL study, we have three major classes of variables. HL—its severity and age of onset—serves as the principal health risk factor. Factors such as clinical interventions and background characteristics (e.g., home, neighborhood, and child attributes) are treated as moderators. Outcomes reflect a broad class of variables that concern the various potential domains of child well-being that may be affected by the HL.

Back to Top | Article Outline

MATERIALS AND METHODS

Participants

The participants in this research were the children and their families. The children were either children who are hard of hearing (CHH) or children with normal hearing (CNH) who provided comparison data.

Back to Top | Article Outline

Children Who Are Hard of Hearing

Children with bilateral HL were enrolled in the research study if (1) their chronological age was between 6 months and 7 years of age at the time of recruitment, (2) they had a better-ear pure-tone average (BEPTA) of 25 to 75 dB HL, (3) they had not received a cochlear implant, (4) they were from homes where English was the primary language, and (5) they did not demonstrate significant cognitive or motor delays.

At entry into the study participants had mild to severe bilateral HL that may or may not have been managed with HAs. In those instances where the child’s HL progressed beyond a BEPTA of 75 dB HL after the initial testing session, the child continued to participate in the study. Children who received a cochlear implant after enrollment in the study continued to participate in data collection, but data collected postimplantation are not included in the current articles.

Back to Top | Article Outline

Ascertainment of Children Who Are Hard of Hearing

Participants were recruited and seen in the home states of three research teams (Iowa, Nebraska, and North Carolina) as well as in regions adjacent to these states. The objective was to locate and approach all parents of children with mild to severe HL within these catchment areas. For recruitment purposes, we utilized records maintained by the Early Hearing Detection and Intervention programs of children who had been referred on the basis of their newborn hearing screening. Through the programs, parents of children who had been referred from the newborn hearing screening, had a confirmed HL, and were within the age range of the study were contacted by the program coordinators. In addition, 6800 recruitment brochures were sent to parents of children by audiologists, early intervention specialists, and educators who served children with HL. These parents were encouraged to return a card indicating interest in the study; this resulted in 188 cards returned. Finally, the research centers in North Carolina and Nebraska were the primary clinical service providers for children with HL in their regions. These centers contacted all parents of CHH who were served in their centers and provided them with information about the study.

All families were paid $15 per hour and those who had to travel more than 50 miles were reimbursed for their travel to the research site. This method of ascertainment aimed to recruit all children in the research catchment areas with HL; however, because recruitment was largely driven through referrals from those serving children with HL, the sample of research participants was largely limited to those being seen for clinical and/or educational management of a HL. Thus, children who were not identified or served were unlikely to be sampled.

Back to Top | Article Outline

Children with Normal Hearing

Our objective in recruiting CNH was to enroll a cohort of children who were similar in home and family background to the CHH. These children provided data to augment standardized, norm-referenced measures. Furthermore, the CNH control group provided critical comparison data for assessments that were not norm referenced. All CNH were required to have four frequency pure-tone averages ≤ 20 dB HL in both ears. CNH were also required to meet the inclusionary criteria employed for the CHH in that they came from homes where English was the primary language and did not present with significant cognitive or motor delays.

Back to Top | Article Outline

Ascertainment of Children with Normal Hearing

Our objective was to recruit approximately 1 CNH for every 3 CHH (3:1 ratio). This ratio was based on the view that we wanted similar numbers of children across the range of hearing levels: normal, mild, moderate, and moderately severe. Thus, the CNH group represented one subgroup of children with a similar hearing status. We recruited these participants by two methods. First, we asked the parents of CHH to nominate friends or neighbors with children in the same age range. We expected that this method would be sufficient to provide all the CNH; however, often parents were either reluctant to nominate or those nominated were not willing to participate. As an alternative, we developed a database containing socioeconomic status features for zip codes in the regions from which the CHH were sampled. As CHH were enrolled, their socioeconomic status was determined and recruitment efforts were directed toward children living in zip codes with comparable socioeconomic statuses. To do this, the Iowa and Nebraska sites used existing registries of children and families who had either participated in prior research or indicated a willingness to participate. In addition, notices were posted in preschools and pediatricians’ offices in the identified zip code areas. Table 1 shows the distribution of the participants in this study by their state of residence at the time of enrollment.

TABLE 1

TABLE 1

Back to Top | Article Outline

Participant Characteristics

Children Who Are Hard of Hearing

Across the time span of recruitment, 430 parents of potential children with HL made contact with the research group. Of those, 81 children were excluded because the child did not meet the hearing criteria. An additional group of 32 children were found to have developmental disorders during the initial examination that would complicate interpretation of the HL effects. This resulted in 317 CHH (173 male; 144 female) who qualified at enrollment. At the time of their enrollment, these children averaged 40.37 months of age (SD = 21.91). The racial composition of the sample was 80% white, 7.2% black, 2.3% Hispanic, 4.6% multi-racial, 2% Asian-Pacific, 3.2% other with 0.7% unknown. Table 2 provides the distribution of maternal education level, and Table 3 provides the distribution of household income. These data show that few mothers had less than a high school education and a slight majority had completed a bachelor’s degree. The fathers’ education levels were similar although fewer fathers reported postgraduate enrollment or degrees (15.8%). These data indicate that this sample of research volunteers was better educated than the US population. The 2012 census shows that 30.6% of women greater than the age of 25 years had completed a bachelor’s degree (Educational Attainment of the Population 18 Years and over, by Age, Sex, Race, and Hispanic Origin: 2012. US Census Bureau. Retrieved July 16, 2013) in comparison to 51.4% of the mothers in our sample. This bias toward higher levels of education is common in research that involves volunteers (Holden et al. 1993).

TABLE 2

TABLE 2

TABLE 3

TABLE 3

During the 5-year course of the study, 31 CHH declined to continue participation. This resulted in an attrition rate of 9.78% for the CHH. A comparison of the children who dropped out of the study with those who remained showed the groups to be similar with respect to mother’s education χ2 (3, 317) = 5.75, p = 0.12 and income χ2 (5, 290) = 5.21, p = 0.39. Thirteen additional children received cochlear implants during the course of the study; however, these children did contribute data before cochlear implantation. The distribution of severity of BEPTA is shown in Figure 1. These data are averaged BEPTAs across the waves of observation. In 19 instances, children’s BEPTAs fell outside the criterion range (25 to 75 dB HL) at study entry. These cases were discussed by the OCHL team and were included either because of prior clinical assessments that showed thresholds in the qualifying range or because of audiological or medical circumstances (e.g., HL in low or high frequencies only, mixed HL with fluctuation due to otitis media with effusion, or enlarged vestibular aqueduct syndrome). One child could not be tested at enrollment and did not continue. The distribution in Figure 1 shows that the majority of children had moderate or moderate-to-severe HL. If we had fully sampled all children in the population with HL in this range, we would have expected to have a distribution where the most frequent occurrence would be in the mild HL range (Russ et al. 2003). The fact that there were fewer children in the mild HL range is likely an indicator that many of these children had not been identified, that the parents of these children were less likely to volunteer to participate, and/or that service providers were a frequent referral source. Regardless of the explanation, the sample is likely to have under-sampled children with mild HL.

Fig. 1

Fig. 1

Back to Top | Article Outline

Children with Normal Hearing

Using the recruitment methods described earlier, 117 CNH (54 male, 63 female) qualified for enrollment. These children averaged 44 months of age (SD = 20.33) at baseline. As shown in Table 2, the distribution of educational levels of these mothers was similar to the mothers of CHH (Wilcoxon Z(1) = 1.09; p = 0.28). Likewise, the household incomes of the children in the two groups was also similar (Table 3; Wilcoxon Z(1) = 1.61; p = 0.11). Although these groups are similar, we should note that the sample of CNH was deficient in the middle education and income level (postsecondary education and $40,001–$60,000) and had a surplus of families in the higher educational and income levels. During the 5-year course of the study, 6 CNH declined to continue participation. This resulted in an attrition rate of 5.13% for the CNH. A comparison of the children who dropped out of the study with those who remained showed the groups to be similar with respect to mother’s education χ2 (3, 117) = 2.92, p = 0.40 and income χ2 (5, 109) = 9.40, p = 0.09. An additional 69 CNH participated in testing on the nonstandardized assessments only, and were only seen for single visits. These children are not included in the attrition rate because they were not part of the longitudinal cohort of CNH.

Back to Top | Article Outline

Research Design

The aims of this research required that we document developmental outcomes during the preschool years and associate these with measures obtained concurrently and at earlier points in development concerning hearing, clinical interventions, and home factors. Thus, a longitudinal research design was needed. The age range to be studied spanned 6.5 years; however, the funding available for this research was limited to 5 years. These factors led to our adoption of an accelerated longitudinal design (Stanger & Verhulst 1995). The traditional prospective longitudinal design involves one cohort of the same or similar age that is followed over a period of time. The accelerated longitudinal design remains prospective, but employs multiple age cohorts that are followed long enough to provide overlap. In this study, rather than having a small number of age cohorts, children were recruited and enrolled continuously across an age span of 6.5 years and were followed for at least 3 years. In this respect, each child represented a cohort of 1. An important feature of this design is that it allows for tests of time (period) versus cohort age effects that could arise by changes in services and technology over time and yet still reveal important developmental relationships.

Within the general framework of the longitudinal design were landmarks for data collection. The first landmark (baseline) represented an observation obtained soon after the child was enrolled in the study regardless of the child’s age. A portion of this assessment was aimed at determining eligibility for the study and to establish contact with the family. Subsequently, follow-up waves of observation were linked to the children’s chronological age. Assessment points were set at 6-month intervals from 6 to 24 months (6, 12, 18, and 24) and then at yearly intervals from 24 months on. Children had a 4-month window to be seen at the 6-, 12-, and 18-month intervals, and a 6-month window at the 2- to 9-year intervals. The rate of participation by CHH and CNH across the study period is shown in Table 4. Each row in this table contains the number of children in each group who were enrolled at a particular age interval, forming a particular age cohort, and then the rate of participation in subsequent waves of assessment through the remainder of the study period. This table shows the pattern of overlapping observations across the age cohorts and also shows that the greatest amount of data obtained in this study spanned the developmental interval of 18 months through 7 years of age.

TABLE 4

TABLE 4

Back to Top | Article Outline

General Data Collection Methods

The data concerning participants’ hearing and behavioral development were gathered in sessions during which the child was examined by an audiologist and a speech-language examiner. In addition, questionnaires concerning the child’s behavior and development were completed by the parents. These visits were typically held on 1 or 2 days and lasted between 2 and 4 hr depending on the age of the child. Assessments generally involved either the family coming to a laboratory at one of the three research sites or coming to a facility that was made available to the research team near the home of the child. At the Iowa site, research vans were used to test children in their home community. All examiners were trained on a common protocol and one research coordinator reviewed videotaped samples of the examinations to ensure that procedures were similar across the sites. For those measures that were standardized, the research coordinator trained examiners in accordance with the test manual. Often, supplementary scoring guidelines were developed to deal with scoring issues that were not addressed in the manuals. Those measures that were non-standardized required that a procedure and scoring guide be developed and examiners trained in accordance with this procedure.

The data were entered on an item-by-item basis into a computer database using Microsoft Access via a web-based program (SharePoint). All data were range and type checked and also double entered to minimize data entry errors. These item level data were then used to compute summed raw scores. For computation of norm-referenced scores, a computer-based table look-up was used to minimize errors in table access.

In addition to the data concerning the child’s status, surveys were also administered to the parents and clinicians serving the child via telephone or via the internet. Parent interviews were conducted by a trained interviewer who contacted the parents each year by telephone at a time midway between the face-to-face visits. Parents were interviewed using a structured questionnaire designed for this study and adopted from the National Early Intervention Longitudinal Study (Bailey et al. 2004) concerning the family (parent education, parents, other adults and children in the home) and the child’s health status. In addition, this questionnaire obtained extensive information about the nature and amount of audiology, communication, and preschool services being provided to the child, as well as the parents’ satisfaction with these services.

Back to Top | Article Outline

Measures

This study was intended to examine the relationship between variation in hearing status among children with and without mild to severe HL with variation in multiple domains of developmental outcomes. In addition, we sought to document important mediators and moderators that act between the HL and the outcomes. As shown in Figure 2, mediators serve to link a precursor such as HL to an outcome and thus operate within causal chain. Our conceptualization was that aided audibility was a key mediator between HL and outcomes, such as speech and language ability. In contrast, moderators are variables that interact with causal effects within the causal chain. Thus, age at HA fitting and HA use moderate the effect of the HL on audible hearing, so that the effects of HL on downstream outcomes will also be altered.

Fig. 2

Fig. 2

We can also see in Figure 2 that outcomes can, in turn, become mediators of the HL on other downstream outcomes, for instance language ability can be an outcome, but also is a mediator of the HL on academic performance. Using the model shown in Figure 2 as a heuristic, we selected measures of what we viewed as important mediators, moderators, and outcomes of HL in early childhood; most of these are reflected in Figure 2. Because the study design provided for the examination of outcomes throughout infancy and early childhood, it was necessary to employ a number of different measures of the same construct to accommodate changes in developmental performance across age. This resulted in a large matrix of measures across variable types and developmental levels. This matrix is provided in Appendix B (see Supplemental Digital Content, Appendix B; http://links.lww.com/EANDH/A205) where the domains of assessment, the specific measures used and the ages at which children received these measures are shown. In the following section, the measures will be described according to the construct being measured and within this, the different measures based on developmental level.

Back to Top | Article Outline

Hearing and Hearing Aids

Unaided Hearing

An audiologist with pediatric experience completed all audiological assessments. A test assistant participated in assessments as needed. The audiologist attempted to obtain air- and bone-conduction thresholds at as many octave and inter-octave frequencies as possible, using visual reinforcement audiometry, conditioned play audiometry, or conventional audiometry, depending on the age of the child. Ear-specific thresholds were obtained with insert earphones, circumaural headphones, or the child’s own earmolds paired with insert earphones. Audiologists obtained soundfield thresholds if the child would not tolerate the testing with earphones or headphones. If a full audiogram could not be completed, the audiologist obtained a copy of the child’s most recent audiogram. In the majority of cases, this audiogram was obtained within 3 months of the OCHL test visit. The BEPTA was calculated for subsequent analyses using the average of 500, 1000, 2000, and 4000 Hz or 500, 1000, and 2000 Hz.

Back to Top | Article Outline

Hearing Aid Verification and Audibility Measures

The audiologist determined that HAs were functioning with manufacturer specifications using conformity measures of HA function. Measures of HA function included total harmonic distortion, frequency range, and output sound pressure level at 90 dB SPL obtained in a 2-cc coupler following ANSI S3.22 (2003).

Because pure-tone average does not accurately represent the differential importance of acoustic spectral information for speech perception, the speech intelligibility index (SII; ANSI S3.5 1997) was calculated for unaided hearing using the child’s audiometric thresholds. The SII is a numerical estimate of audibility across the frequency range of speech. It is calculated by estimating the audibility of an average speech signal compared with the listener’s hearing thresholds or level of background noise, whichever is greater. The calculation is completed for a discrete number of frequency bands, which are each assigned an importance weight based on the contribution of that frequency band to the average speech recognition score for a group of adult listeners with normal hearing. The audibility of each band is multiplied by the importance weight for that band. The weighted audibility of all bands are summed to create a number between 0 and 1 that describes the weighted audibility of the long-term average speech spectrum, where a value of 0 indicates that none of the spectrum is audible and 1 represents complete audibility. Unaided SII provides an estimate of audibility for the acoustic cues for speech and language that provides specific weight to individual frequency bands without amplification. In addition, most of the children in this study had been fitted with HAs; therefore, we computed indices of aided SII based on the child’s HA verification data.

Simulated real-ear measures were used to calculate aided and unaided SII in cases where the response of the HA could not be measured in the child’s ear. The audiologist initially conducted probe microphone measures to quantify the real-ear-to-coupler difference (RECD; Bagatto et al. 2005). An age-related average RECD estimated the acoustic characteristics of the child’s occluded ear, when the RECD could not be measured due to limited cooperation or subject noise. HA verification was then completed in the 2-cc coupler. Audioscan Verifit software (Cole 2005) calculated aided SII at users’ settings and unaided SII for the participants, using the standard male carrot passage (Cox & McDaniel 1989) presented at soft (50 or 55 dB SPL) and average (60 or 65 dB SPL) levels following ANSI S3.5 (1997). A swept pure tone at 90 dB SPL measured maximum output. The obtained fitting data were then compared with the prescriptive targets of the Desired Sensation Level (version 5.0) algorithm (Scollie et al. 2005). A subset of children in the study were fit with nonlinear frequency compression, which is a signal processing strategy that lowers high-frequency speech energy to lower frequencies to improve audibility. For children with nonlinear frequency compression, the SII was modified to account for the location of each one-third octave band used in the SII calculation after lowering. The sensation level of speech bands that were filtered to correspond with frequency bands used in the SII calculation were measured (3000, 4000, 5000, 6300 Hz) in the Audioscan Verifit. A spreadsheet was used to calculate the SII based on the sensation level of the frequency band. Importantly, this method assumes that information that is lowered carries the same importance for speech recognition as in cases without frequency lowering and does not account for loss of spectral distinctiveness that may occur with nonlinear frequency compression. The modification of the SII was intended to quantify the amount of speech information that is audible and was not intended to predict speech recognition.

Our research protocol called for the collection of data on the adequacy of the HA fit. We expected to find children who were poorly fit. This raised the issue as to whether, knowing that the HA did not meet fitting standards, should we intervene? If we were to intervene, we faced the prospect of contaminating an important variable in our study such that we would not be able to address the question of whether HA fitting was important. However, the rational for studying this issue was that there was little evidence to support the importance of HA fitting. Thus, we were in a situation where our research objectives seemed to be in potential conflict with ethical principles having to do with the welfare of the children in our study. This situation constituted a case of “clinical equipoise.” In such cases where there is equipoise with regard to the treatment, clinical inaction is considered acceptable. Thus, we concluded that it was not unethical to leave the HAs fit as they were. We also believed that any intervention on our part, such as altering the fit was inappropriate, as we were not seeing the child in a clinical capacity and the clinician who had fit the aid may have had good reasons for choosing a particular fit. Our research team remained uncomfortable with taking no action, particularly where the HA was very poorly fit. Thus, we decided that for children who had HAs fit with an SII of less than .25, we would send a letter to the parents informing them of this and suggesting that they consult with the person responsible for the HA fit.

Back to Top | Article Outline

Speech Perception

The intended purpose of speech perception measures is to assess how an individual is using functional hearing to recognize and understand speech. These tasks typically take the form of single word or sentence repetition in quiet or noise, although such measures must be adapted for younger children to accommodate their level of language ability, speech production skills, and attention span. Therefore, we utilized age appropriate measures, depending on the age of the child. Speech perception abilities for 12- and 18-month olds were assessed via two parent-report measures, the LittlEars questionnaire (Coninx et al. 2009) and the Parents’ Evaluation of Aural/Oral Performance of Children (PEACH; Ching & Hill 2007). Two-year olds were administered the PEACH and the Early Speech Perception Test (ESP-low verbal; Moog & Geers 1990), in which children point to objects in sets of 4. Three-year olds were administered the ESP-standard version (Moog & Geers 1990), which requires children to point to pictures. Both the ESP-low verbal and standard versions were administered live voice, with an acoustic hoop. Four- and 5-year-old children were administered recorded single-word repetition measures in quiet. The measures consisted of the Multisyllabic Lexical Neighborhood Test and the Lexical Neighborhood Test (Kirk et al. 1995) and the Phonetically Balanced Kindergarten Words List (Haskins 1949); all lists were presented at 65 dB SPL. Six-, 7-, 8-, and 9-year olds only received the Phonetically Balanced Kindergarten Words List. Finally, 7-, 8-, and 9-year olds were tested on their ability to perceive single words in background noise using the Computer-Assisted Speech Perception Assessment (CASPA; Boothroyd 1999). The CASPA was presented with noise fixed at 55 dB SPL and speech at 50 dB SPL (−5 dB SNR), 65 dB SPL (+10 dB SNR), and 75 dB SPL (+20 dB SNR).

Back to Top | Article Outline

Hearing Aid Use

At every visit, an examiner conducted an interview with the parent that addressed daily HA use practices. Parents estimated the average amount of time the child used HAs per day during the week and on the weekends. In addition, audiologists collected objective measures for average use time per day by connecting the HAs to a HiPro box and using the appropriate manufacturer’s HA software. If the values were different between ears, the larger value was included in data analyses.

Back to Top | Article Outline

Speech Sound Production

The ability to accurately produce the sound properties of words is clearly dependent on having sufficient access to the sound property of words and more generally to the phonology of the child’s ambient language. Through this process, auditory and phonological representations of words are built; we might expect that impaired hearing would have negative consequences on this learning process. As we will note later, impairments in these phonological representations could have an impact on the development of grammatical morphology, reading, and spelling. Speech sound production is an early skill that is dependent on the development of robust phonological representations. Thus, we incorporated several measures of speech sound production at 6, 12, and 18 months of age, as well as 2, 3, 5, and 7 years of age. All children who participated for 3 years received at least one measure of speech sound development. Speech sound development from 6 through 18 months is comprised of prelinguistic vocal behavior generally described as babbling. A measure of babbling development was designed to obtain systematic reports from parents of the child’s vocalizations. This measure provided parents with audio examples of infant vocalizations demonstrating different levels of vocal development (precanonical, canonical, or word). The parents would then indicate the degree to which their infant produced vocalizations like the example. Responses were used to develop a score reflective of the level of the infant’s vocal development. By age 2 years, vocal development is expected to follow basic phonological features of the ambient language and to be used to express words of the language. Thus, the measure of speech at this point shifted to assessment of the degree to which the child’s production of words conformed to adult standards of production. At age 2 years, the Conditioned Assessment of Speech Production (Ertmer & Stoel-Gammon 2008) was used to elicit simple vowel, consonant–vowel, and consonant–vowel–consonant syllables. At this same interval, speech production was also assessed using the Open & Closed Set Test (Ertmer et al. 2004), which elicits a set of 10 words using models and pictures. At 3, 5, and 7 years of age the Goldman Fristoe Test of Articulation-2 (Goldman & Fristoe 1999) was given. This is a commonly used standardized test of speech sound production accuracy that provides norms based on hearing children. In addition, the Beginner’s Intelligibility Test (Osberger 1994) was administered at age 5 years. In the Beginner's Intelligibility Test, short sentences are elicited from the child and audio recorded. Three naive listeners then independently transcribe the sentences and the sentences are scored for the percentage of words correctly understood.

Back to Top | Article Outline

Language

Across the study, language skills were measured in a variety of domains, using several methods. The selection of the specific measures obtained at particular ages was based on our interest in measuring language skills that were developmentally emerging and important at that stage of development. In many instances, standardized, norm-referenced tests were used in order that the performance of CHH and CNH could be compared with national norms for hearing children.

Back to Top | Article Outline

Six Months to 2 Years

Language development during infancy moves from very limited language skills to the emergence of early simple sentences and basic vocabulary. Structured tests of language are challenging for infants and toddlers and therefore often language is measured via parent report or in semistructured activities. The words and gestures form of the MacArthur Bates Communicative Development Inventory (Fenson et al. 2007) was completed by parents at 12 and 18 months of age. At 24 months, the words and sentences form of the MacArthur Bates Communicative Development Inventory was administered. In addition, the expressive and receptive language subtests of the Vineland Adaptive Behavior Scales (Vineland-II; Sparrow et al. 2005), which is a parent-report scale, were administered at 12 and 24 months. The Vineland-II was also administered at 3 and 4 years of age. Complementing these parent-report measures were three measures that involved elicited language in semistructured activities. At 18 months, an adaptation of the behavior sample of the Communication and Symbolic Behavior Scales-Developmental Profile (Wetherby & Prizant 2001) was given along with the caregiver questionnaire. Also, the receptive and expressive scales of the Mullen Scales of Early Learning (Mullen 1995) were given at ages 12 and 24 months. These scales involve elicitation of developmental and communicative behaviors by the parent and/or examiner which are then scored by the examiner. At the 18-month visit, caregivers and children spent approximately 5 min engaged in the Art Gallery task, which has been used with children with normal hearing (Adamson et al. 2004) and children with HL (Quittner et al. 2004). Ambrose et al. (2015, this issue, pp. 48S–59S) provide a complete description of the Art Gallery task.

Back to Top | Article Outline

Three to 5 Years

By age 3 years, it becomes more feasible to administer standardized language tests rather than rely on parent report. The preschool years represent a time of considerable growth in the major domains of language, especially grammar and vocabulary, and conversational interactions of limited extent are expected. By the end of the preschool years, the child’s language is often developed to the point of supporting extended conversations and the use of language for problem solving and academic learning. At age 3 years, 15-min language samples were obtained in a naturalistic setting involving play-centered activities. Caregivers and children also participated in the 5-min Art Gallery task. These samples were transcribed and coded for analysis using the Systematic Analysis of Language Transcripts software (SALT; Miller & Chapman 1998). At ages 3 and 4 years, language was measured by the core subtests of the Comprehensive Assessment of Spoken Language (CASL; Carrow-Woolfolk 1999). These subtests measure receptive semantics, sentence use (understanding and production), and pragmatics. At age 3 and 4 years, grammatical morphological skills were measured using a morphological elicitation task that tested the child’s ability to produce nine different morphological forms (regular and irregular plurals and tense, possessive, third person agreement, copulas, auxiliaries, and progressives). This additional task focusing on grammatical morphology was motivated by the hypothesis that this aspect of language would be particularly sensitive to good audibility of speech information, as some of these grammatical morphemes such as third-person singular -s, plural -s, and possessive -s have reduced phonetic substance (Montgomery & Leonard 2006). The introduction to this OCHL Supplement (Moeller & Tomblin 2015, this issue, pp. 4S–13S) provides additional description regarding this hypothesis. At age 4 years, a measure of oral vocabulary knowledge reflected in a word definition task was obtained as a part of the Wechsler Preschool & Primary Scale of Intelligence-III (WPPSI-III; Wechsler 2002). At age 5 years, the Preschool Language Assessment Instrument-II (Blank et al. 2003) was administered to examine language skills that are particularly important to early school performance. We also administered a measure that targeted morphological knowledge, the word structure subtest of the Clinical Evaluation of Language Fundamentals-4 (Semel et al. 2004), and a measure of receptive vocabulary, the Peabody Picture Vocabulary Test-4 (Dunn & Dunn 2007). These tests were given at ages 5 and 7 years.

Back to Top | Article Outline

Six to 9 Years

By 6 years of age, many children will have a substantial command of vocabulary and grammar found in ordinary adult communication and will be able to follow extended oral directions, participate in extended conversation, and tell simple narratives. Language growth during these primary grade years is found in further development of complex sentences, growth in more advanced vocabulary, and development of narrative and expository skills. At age 6 years, spontaneous language was sampled in the context of a 15-min play-based conversation that was transcribed and coded for SALT analysis. Language was measured via standardized testing using the core subtests of the CASL 5–6 that measure semantic skills through the child’s production of antonyms, syntax via completion of sentences and comprehension of short paragraphs, and pragmatics by providing the child with different communication situations and asking what the child would do. At age 8 years, the core subtests of the CASL 7–10 were given; this consisted of the same subtests as the CASL 5–6, but with the addition of a measure of nonliteral language. At age 7 and 9 years, the Peabody Picture Vocabulary Test-4 and a measure of expressive vocabulary from the Wechsler Abbreviated Scale of Intelligence-II (WASI-II; Wechsler 2011) were administered. Also at these two ages, samples of text-level discourse were elicited through spontaneous and retell narratives and expository discourse. These samples were transcribed and entered into SALT for analysis. The narratives were entered in SALT and coded using an adaptation of the SALT-based narrative scoring scheme (Heilmann et al. 2010), which rates the child’s narrative production on story grammar elements, cohesion, connecting events, rationale for characters’ behaviors, and referencing.

Back to Top | Article Outline

Prereading and Academic Outcomes

Achievement in the critical areas of reading, spelling, and mathematics are often viewed as one of the most important outcomes of childhood. Success in reading in particular is used as a benchmark of both the educational success of children and of the educational system. Reading skills are typically depressed in children with HL, even though on the surface this seems to be a visual task (Allen 1986). Despite its apparent dependence on vision, reading achievement in CNH has been primarily linked to oral language experience and ability (Catts et al. 2002). Success in reading is well known to begin before school entry via good oral language development and skills in phonological processing as well as exposure to books and thus knowledge of letters and print conventions. Early phonological processing skills are reflected in the ability to remember novel phonological sequences, as well as attend to and manipulate sublexical structures (syllables, syllable onsets, and rhymes and phonemes). These skills are linked to the development of early word reading. HL could result in weak phonological representations that would impact these early prereading skills. Such early reading skill acquisition involves learning relationships between sub-lexical structures and patterns of print. Accomplishment of this aspect of reading is typically examined using nonword-reading tasks. Reading, however, is ultimately concerned with the construction of meaning from the text and this can be examined by reading real words or by reading passages. Our protocol incorporated measures of all of these aspects of early reading foundations and subsequent development of basic reading skills. Spelling and reading are usually developed in tandem. Spelling is quite sensitive to the child’s learning of relationships between phonology and graphemes (letter and letter combinations), thus again, if HL effects the development of robust phonological knowledge, spelling may be affected. Finally, we examined basic arithmetic skills due to their importance in school performance, but we might expect that these skills would not depend on the language and listening abilities needed for reading and spelling.

Back to Top | Article Outline

Prereading

All of the early spoken language measures described earlier can be viewed as prereading skills. In addition to these language measures, we examined early phonological processing and knowledge of print beginning at age 4 years using the print knowledge and phonological awareness subtests of the Test of Preschool Early Literacy (TOPEL; Lonigan et al. 2007). The print knowledge subtest asks the child to identify letters, name specific letters, identify letters associated with specific sounds, and say the sounds associated with specific letters. The phonological awareness subtest consists of elision tasks that ask the child to say what is left in a word after deleting some of the segments (e.g., cat without the /k/ is at) and a blending task where the child forms a word based on hearing separate segments (e.g., /k/, /æ/, /t/ is “cat”). At age 5, print knowledge was assessed using the TOPEL, and phonological processing was measured using the Comprehensive Test of Phonological Processing (Wagner et al. 1999). The Comprehensive Test of Phonological Processing tests phonological awareness with a wider range of tasks than the TOPEL (sound matching, phoneme isolation, and syllable segmentation as well as elision and blending) and also tests phonological memory (nonword repetition and digit span) and naming speed (digits, colors, and objects). This test was also given at ages 7 and 9 years.

Back to Top | Article Outline

Reading

Reading skills were examined at ages 6 and 8 years using three subtests of the Woodcock Reading Mastery Tests–R/NU (Woodcock 1998). Specifically, the child’s ability to employ phonics knowledge to decode nonwords was measured by the word attack subtest. The word identification subtest was given to measure real-word reading, reflecting the child’s ability to use phonics or any other means to read real words. The passage comprehension subtest evaluates the ability of an individual to understand a short-written passage. Each item consists of one or more sentences with one missing key word. The child read the sentences and identified the missing word. Passages are designed so that it is difficult to identify the target word without reading the entire selection.

Back to Top | Article Outline

Spelling and Mathematics

Spelling and mathematics skills were examined at ages 7 and 9 years using the Wechsler Individual Achievement Test-II (WIAT-II; Wechsler 2005). The spelling subtest of the WIAT-II assesses children’s ability to spell correctly by writing either letters or letter combinations when the word was read by an examiner within a sentence context. In the case of the letter or letter combinations, the sound and its place in the word were identified (“write the letter that makes the /m/ at the beginning of “man”). In addition to the quantitative score, this subtest also provides a qualitative analysis of the child’s spelling. The WIAT-II mathematical reasoning subtest assesses the ability to count, identify shapes, and solve word problems that are given via speech along with either written or picture material.

Back to Top | Article Outline

General Abilities/Intelligence and Social Cognition

General Abilities/Intelligence

The construct of intelligence has been at the center of individual differences psychology for well over a century. During that time many different models have been proposed. Intelligence is often viewed as an inborn (innate) long-term learning ability (Burt 1958). In contrast, other models consider intelligence as a “developed ability” (Anastasi 1980, 1984) that represents the individual’s attainment of skills that are prerequisite for and predictive of future learning. This latter view of intelligence is employed in this study and therefore we view intelligence as a form of developmental outcome and also a developmental state that influences subsequent learning. Within such a perspective, intelligence is open to the influence of altered experience resulting from HL and thus we can expect that certain domains of ability, such as verbal/language abilities that depend on hearing will be more affected by a HL than others. In this regard, we employed measures of intelligence based on the Wechsler scales of intelligence that provide verbal and nonverbal measures that we expect will be differentially influenced by HL and subsequent outcomes. At age 4 years, we administered four subtests of the WPPSI-III (Wechsler 2002). Two subtests measured performance (nonverbal) abilities: block design and matrix reasoning and two measured verbal abilities: similarities and vocabulary. At age 6 years, the Wechsler Abbreviated Scale of Intelligence-II (Wechsler 2011) was administered, which also entailed the same four subtests that were given at age 4 years. Complementing these traditional tests of intelligence, we also collected information about the child’s general development using the Vineland-II (Sparrow et al. 2005). These scales were originally designed to complement tests of intelligence by emphasizing social and functional skills that are important developmental outcomes. The Vineland-II, which was given at ages 12 and 24 months and 3 and 4 years, is completed by the caregiver and measures developmental abilities in communication, daily living, socialization, and motor skills.

Back to Top | Article Outline

Social Cognition

Communication is an inherently social behavior; the development of communication both builds on and promotes the development of social cognitive skills. An important subset of these skills concerns the ability to infer mental states (feelings, beliefs, and knowledge) of others. This subset of skills is known as “theory of mind” (ToM). ToM has been linked to the development of pragmatic skills, but also is thought to arise from the experience of conversational engagement in early childhood (Peterson & Siegal 2000). In this project, ToM was measured using four standard-false-belief trials. In two of the trials, the children were presented with a change of location story (Wimmer & Perner 1983). In this type of story, one character places an object somewhere for safe keeping and leaves the room. Unbeknownst to the first character, a second character moves the object to a different location and then exits. When the original character returns to get the object, children are asked to predict where this character will first look for the object. The task requires that the child understand that the character in the story did not see that the object was moved and therefore has a false belief about its true location. To answer correctly, children must understand that the character’s belief about the situation is different than their own, and that the character will act on his/her own understanding. The second set of trials used an unexpected contents format (Hogrefe et al. 1986). Children were shown a crayon box and were asked to state what they thought was in the box. Next, they were allowed to look inside and see that something unexpected (a spoon) was inside. Finally, they were asked to predict what a child who had not looked inside the box would say was in it. The correct response (crayons) requires the child to recognize that someone who has not looked inside will not know what it contains, and will predict the contents based on what they do know about the box (that it is a crayon box).

Back to Top | Article Outline

Behavior, Family, and Service Provision

In addition to the annual family interview, a set of measures were obtained regarding the child’s behavioral and interpersonal development using scales from the Achenbach System of Empirically Based Assessment (Achenbach & Ruffle 2000), specifically, the Child Behavior Checklist, and the Teacher Report Form. Providers of each child’s audiologic services and early child education services were surveyed using a web-based survey.

Back to Top | Article Outline

CONCLUSIONS

The goal of the OCHL study was to examine the association of hearing status of children with and without mild to severe HL within multiple domains of developmental outcomes. We also sought to document important mediators and moderators that act between the HL and the outcomes. To meet these goals, we recruited 317 CHH and 117 CNH. The current article describes demographic characteristics of the research participants and their families, the research design of the study, and data collection methods. Subsequent articles in this OCHL Supplement will describe sources of variability and outcomes for the cohort in an effort to provide evidence-based research for clinicians and researchers who work with CHH.

Back to Top | Article Outline

ACKNOWLEDGMENTS

This research was supported by a grant from the National Institutes of Health/National Institute on Deafness and Other Communication Disorders: R01DC009560.

Back to Top | Article Outline

REFERENCES

Achenbach T. M., Ruffle T. M.. The child behavior checklist and related forms for assessing behavioral/emotional problems and competencies. Pediatr Rev. (2000);21:265–271
Adamson L. B., Bakeman R., Deckner D. F.. The development of symbol-infused joint engagement. Child Dev. (2004);75:1171–1187
Allen T. E.Schildroth A. N., Karchmer M. A.. Patterns of academic achievement among hearing impaired students: 1974 and 1983 In Deaf Children in America. (1986) San Diego, CA College-Hill Press:161–206
Ambrose S. E., Walker E. A., Unflat-Berry L. M., et al. Quantity and quality of caregivers’ linguistic input to 18-month and 3-year-old children who are hard of hearing. Ear Hear. (2015);36:48S–59S
Anastasi A.. Abilities and the measurement of achievement In New Directions for Testing & Measurement. (1980);Vol 5 San Francisco, CA Jossey-Bass:1–10
Anastasi A.. 7. Aptitude and Achievement Tests: The Curious Case of the Indestructible Strawperson. Social and Technical Issues in Testing: Implications for Test Construction and Usage. Paper 9. (1984) Available at: http://digitalcommons.unl.edu/burostestingissues/9.
ANSI. ANSI. Methods for Calculation of the Speech Intelligibility Index. Technical Report S3.5-1997. (1997) New York, NY American National Standards Institute
ANSI. ANSI. S3.22–2003. Specifications of Hearing Aid Characteristics. (2003) New York, NY American National Standards Institute
Bagatto M., Moodie S., Scollie S., et al. Clinical protocols for hearing instrument fitting in the desired sensation level method. Trends Amplif. (2005);9:199–226
Bailey D. B. Jr, Hebbeler K., Scarborough A., et al. First experiences with early intervention: A national perspective. Pediatrics. (2004);113:887–896
Blank M., Rose S. A., Berlin L. J. Preschool Language Assessment Instrument. (2003)2nd ed. Austin, TX Pro-Ed
Boothroyd A. Computer-Assisted Speech Perception Assessment (CASPA v2. 2). (1999) San Diego, CA.
Burt C.. The inheritance of mental ability. Am Psychol. (1958);13:1–15
Carrow-Woolfolk E. Comprehensive Assessment of Spoken Language. (1999) Circle Pines, MN American Guidance Service
Catts H. W., Fey M. E., Tomblin J. B., et al. A longitudinal investigation of reading outcomes in children with language impairments. J Speech Lang Hear Res. (2002);45:1142–1157
Ching T. Y., Hill M.. The Parents’ Evaluation of Aural/Oral Performance of Children (PEACH) scale: Normative data. J Am Acad Audiol. (2007);18:220–235
Cole B. Audionote2: Verifit Test Signals. (2005) Dorchester, Ontario, Canada Audioscan
Coninx F., Weichbold V., Tsiakpini L., et al. Validation of the LittlEARS® Auditory Questionnaire in children with normal hearing. Int J Pediatr Otorhinolaryngol. (2009);73:1761–1768
Cox R. M., McDaniel D. M.. Development of the speech intelligibility rating (SIR) test for hearing aid comparisons. J Speech Hear Res. (1989);32:347–352
Dunn L. M., Dunn D. M. Peabody Picture Vocabulary Test. (2007)4th ed. Circle Pines, MN American Guidance Service
Ertmer D. J., Stoel-Gammon C.. The Conditioned Assessment of Speech Production (CASP): A tool for evaluating auditory-guided speech development in young children with hearing loss. Volta Rev. (2008);108:59–80
Ertmer D. J., Miller C. M., Quesenberry J. K. The Open and Closed Set Task. (2004) West Lafayette, IN Purdue University
Fenson L., Marchman V. A., Thal D. J., et al. The MacArthur-Bates Communicative Development Inventories. (2007) Baltimore, MD Paul H. Brookes Publishing Co
Goldman R., Fristoe M. Goldman-Fristoe Test of Articulation-2. (1999) Circle Pines, MN American Guidance Service Inc
Haskins J. Kindergarten Phonetically Balanced Word Lists (PBK). (1949) St. Louis, MO Auditec
Heilmann J., Miller J. F., Nockerts A., et al. Properties of the narrative scoring scheme using narrative retells in young school-age children. Am J Speech Lang Pathol. (2010);19:154–166
Hogrefe G.-J., Wimmer H., Perner J.. Ignorance versus false belief: A developmental lag in attribution of epistemic states. Child Dev. (1986);57:567–582
Holden G., Rosenberg G., Barker K., et al. The recruitment of research participants: A review. Soc Work Health Care. (1993);19:1–44
Kirk K. I., Pisoni D. B., Osberger M. J.. Lexical effects on spoken word recognition by pediatric cochlear implant users. Ear Hear. (1995);16:470–481
Lonigan C. J., Wagner R. K., Torgesen J. K. Test of Preschool Early Literacy. (2007) Austin, TX Pro-Ed
Miller J. F., Chapman R. SALT: Systematic Analysis of Language Transcripts. (1998). [Computer software]. Madison, WI: University of Madison-Wisconsin, Language Analysis Laboratory.
Moeller M. P., Tomblin J. B.. An introduction to the Outcomes of Children with Hearing Loss study. Ear Hear. (2015);36:4S–13S
Montgomery J. W., Leonard L. B.. Effects of acoustic manipulation on the real-time inflectional processing of children with specific language impairment. J Speech Lang Hear Res. (2006);49:1238–1256
Moog J. S., Geers A. E. Early Speech Perception Test. (1990) St Louis, MO Central Institute for the Deaf
Mullen E. Mullen Scales of Early Learning. (1995) Circle Pines, MN American Guidence Service
Osberger M. J.. Speech intelligibility of children with cochlear implants. Volta Rev. (1994);96:169–180
Peterson C. C., Siegal M.. Insights into theory of mind from deafness and autism. Mind Lang. (2000);15:123–145
Quittner A. L., Leibach P., Marciel K.. The impact of cochlear implants on young deaf children: New methods to assess cognitive and behavioral development. Arch Otolaryngol Head Neck Surg. (2004);130:547–554
Russ S. A., Poulakis Z., Barker M., et al. Epidemiology of congenital hearing loss in Victoria, Australia. Int J Audiol. (2003);42:385–390
Scollie S., Seewald R., Cornelisse L., et al. The desired sensation level multistage input/output algorithm. Trends Amplif. (2005);9:159–197
Semel E., Wiig E., Secord W. Clinical Evaluation of Language Fundamentals-4. (2004) San Antonio, TX The Psychological Corp
Sparrow S., Cicchetti D., Balla D. Vineland Adaptive Behavior Scales. (2005)2nd ed. San Antonio, TX Pearson
Stanger C., Verhulst F. C.Verhulst F. C., Koot H. M.. Accelerated longitudinal designs. In The Epidemiology of Child and Adolescent Psychopathology. (1995) New York, NY Oxford University Press:385–405
Tomblin B., Hebbeler K.. Current state of knowledge: Outcomes research in children with mild to severe hearing impairment–approaches and methodological considerations. Ear Hear. (2007);28:715–728
Wagner R. K., Torgesen J. K., Rashotte C. A. Comprehensive Test of Phonological Processing. (1999) Austin, TX: Pro-Ed.
Wechsler D. The Wechsler Preschool and Primary Scale of Intelligence. (2002)3rd ed. San Antonio, TX The Psychological Corporation
Wechsler D. Wechsler Individual Achievement Test. (2005)2nd ed. San Antonio, TX PsychCorp
Wechsler D., Hsiao-pin C. Wechsler Abbreviated Scale of Intelligence-II. (2011) San Antonio, TX Pearson
Wetherby A., Prizant B. M. Communication and Symbolic Behavior Scales Developmental Profile. (2001) Baltimore, MD Brooks Publishing Co
Wimmer H., Perner J.. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition. (1983);13:103–128
Woodcock R. Woodcock Reading Mastery Tests-Revised/Normative Update. (1998) Circle Pines, MN American Guidance Service
Youngs M., Wingerson L. The 1996 Medical Outcomes and Guidelines Sourcebook. (1995) New York, NY Faulkner & Gray, Inc
Keywords:

Children; Hard of hearing; Hearing loss

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.