Topics in Language Disorders:
Direct and Indirect Effects of Stimulating Phoneme Awareness vs. Other Linguistic Skills in Preschoolers With Co-occurring Speech and Language Impairments
Tyler, Ann A.; Gillon, Gail; Macrae, Toby; Johnson, Roberta L.
Department of Speech Pathology and Audiology, Western Michigan University, Kalamazoo (Dr Tyler); College of Education, University of Canterbury, Christchurch, New Zealand (Dr Gillon); School of Communication Science & Disorders, Florida State University, Tallahassee (Dr Macrae); and Washoe County School District, Reno, Nevada (Ms Johnson).
Corresponding Author: Ann A. Tyler, PhD, Department of Speech Pathology and Audiology, Western Michigan University, Kalamazoo, MI 49008-5355 (email@example.com).
Aim: The purpose of this study was to examine the effects of an integrated phoneme awareness/speech intervention in comparison to an alternating speech/morphosyntax intervention for specific areas targeted by the different interventions, as well as the extent of indirect gains in nontargeted areas.
Method: A total of 30 children with co-occurring speech sound disorder and language impairment, average age 4;5, participated in the study, 18 from the United States and 12 from New Zealand. Children from matched pairs were randomly assigned to the 2 proven efficacious treatments, which were delivered in 6-week blocks separated by a 6-week break. Phoneme awareness, speech sound production, and oral language outcome measures were collected pretreatment and after each intervention block.
Results and Conclusions: Both intervention groups made statistically significant gains in all measures, with the exception of a morpheme measure only approaching significance. There were clear trends in favor of the specificity of the interventions suggesting increased sample size might have led to some significant intervention differences. Results further implicate the need for early intervention that integrates oral language and phoneme awareness/early literacy skills for children with multiple deficits.
CHILDREN with speech sound disorder (SSD) and those with language impairment (LI) comprise heterogeneous populations in which those with co-occurring phonological and morphosyntactic impairments may account for one of the largest subgroups (Conti-Ramsden & Botting, 1999; Rapin & Allen, 1983, 1988; Shriberg & Austin, 1998). It is estimated that 50% to 70% of children with speech and language disorders experience general academic difficulty throughout school, with difficulty learning to read being of particular concern for much of the population (Aram, Ekelman, & Nation, 1984; Aram & Hall, 1989; Felsenfeld, Broen, & McGue, 1994; Shriberg & Kwiatkowski, 1988; Tallal, Curtiss, & Ross, 1989). The long-term outcome for this population with respect to continued impairment, communication status, and educational and occupational results is guarded. The majority of children, identified as language impaired (with or without accompanying SSD) at the age of 5 years in a longitudinal study of a community sample of 142, maintained that classification at 18 to 20 years of age (Johnson et al., 1999). Furthermore, research on literacy achievement and speech and language disorders suggests that it is precisely this subgroup, with SSD and concomitant LI, that is at greatest risk of reading disability, in comparison to the group with SSD only (Catts, 1993; Catts, Fey, Zhang, & Tomblin, 2001; Lewis, Freebairn, & Taylor, 2000; Snowling, Bishop, & Stothard, 2000).
It is well established that phonological awareness skills are related to literacy outcomes. Within the broader domain of phonological awareness, the awareness of phonemes as individual units is highly predictive of later reading and writing success. General phonological processing skills, including phonological awareness, are implicated in LI, and as a group, these children perform more poorly in this domain than their typical peers (Catts, 1993).
Research has shown that children with SSD as a group also demonstrate poor phonological awareness skills in comparison to their typically developing peers (Bird, Bishop, & Freeman, 1995; Leitao & Fletcher, 2004; Raitano, Pennington, Tunick, Boada, & Shriberg, 2004; Rvachew, Ohberg, Grawburg, & Heyding, 2003). Yet, it is known that some children with SSD and poor phonological awareness skills do not experience literacy difficulties (Larrivee & Catts, 1999; Leitao & Fletcher, 2004). A convergence of recent findings suggests that, although phoneme awareness skills are variable in the SSD population, it is not phonological awareness alone, but rather language skills, particularly syntax, and rapid serial naming, along with nonverbal IQ that best predict literacy outcomes (Pennington & Bishop, 2009; Peterson, Pennington, Shriberg, & Boada, 2009).
In the same vein, there is evidence from epidemiological and referral samples of comorbidity among paired combinations of the three developmental disorders, reading disability, SSD, and LI. Risk for these comorbidities can vary as a result of overlap with the third disorder. Such results provide support for a multiple-deficit model in which the combined and overlapping configuration of these multiple underlying deficits determines the observed disorder(s) (Pennington & Bishop, 2009).
From a treatment perspective, the implication of a multiple-deficit model is that it is becoming increasingly clear that interventions should target multiple skill areas (Tyler, 2010). Not only is there a need for facilitating speech normalization by school entry (Nathan, Stackhouse, Goulandris, & Snowling, 2004) but also there is a need for teaching foundational language skills (e.g., receptive and expressive vocabulary and grammar), as well as phoneme awareness skills that are linked with word-level decoding. Although children with both phonological and morphosyntactic disorders may comprise the largest segment of preschoolers with identified speech and language disorders (Broomfield & Dodd, 2004; Conti-Ramsden & Botting, 1999; Shriberg, 2010), the evaluation of behavioral interventions has more often focused on the efficacy of intervention within a specific linguistic domain, such as phonology, using a population selected for impairment in only that domain. On the contrary, the efficacy of a variety of individual speech sound and language interventions has been established (Gierut, 1998; Law, Garrett, & Nye, 2004; Tyler, 2008) as have interventions that focus on phonological awareness (Gillon, 2000; Hesketh, Adams, Nightingale, & Hall, 2000).
Efficacy research has focused less on interventions explicitly designed to capitalize on interactions between aspects of the linguistic system, such as between phonology and morphosyntax, or speech intelligibility and phonological awareness, as well as those that are more broadly focused. Notable exceptions are interventions focused on phonological awareness with integration of speech production targets (Gillon, 2005; Gillon & Moriarty, 2005), dynamic language (Hoffman & Norris, 2005, 2010), and morphosyntax (Haskill, Tyler, & Tolbert, 2001; Tyler, Lewis, Haskill, & Tolbert, 2002). Interventions, such as these, which are more broadly focused or consider linguistic interactions, have the potential for increased efficiency resulting from their integrated approach. Interventions that produce documented gains across domains provide a method of treating multifaceted deficits.
With respect to long-term language and literacy outcomes, the efficacy of phonological awareness intervention for producing gains in not only early reading achievement and phoneme awareness ability, but also speech intelligibility, in early elementary-aged children has been established in controlled group studies (Dodd & Gillon, 2001; Gillon, 2000; Hesketh et al., 2000). The improvement in speech intelligibility, however, may not always parallel the improvement made in phonological awareness (Harbers, Paden, & Halle, 1999). Additionally, there is one report of no indirect speech improvement from a short-term phonological awareness intervention (Denne, Langdown, Pring, & Roy, 2005). Hesketh et al. (2000) observed the reciprocal effect, however; that is, indirect improvements in children's phonological awareness skills resulted from an intervention directly targeting speech production skills, and the improvements were commensurate with those observed from a metaphonological approach.
Gillon (2005) extended investigation of a phonemic awareness (PA) intervention with integrated direct speech focus, which was originally developed for school-age students, to measure its effects with a younger, preschool-age group with SSD. When the children who had received the intervention as preschoolers were compared at 6-years of age with a matched group of children with SSD who had not received the intervention, the experimental group performed significantly better in phonological awareness skills. The intervention group had also improved in speech intelligibility so that there were no differences between the groups in speech accuracy. Importantly, the children who had received the PA intervention were performing significantly better in early reading skills than those who had not received this intervention. Gillon suggested that these data provide strong support for integrating activities to develop PA and letter knowledge in phonological interventions for preschool-age children with SSD.
Morphosyntactic intervention is an example of an intervention designed with consideration of morphophonemic interactions to focus on language, in particular finite morphemes (Haskill et al., 2001). As part of a larger project, Tyler et al. (2002) investigated the efficacy of this morphosyntax intervention and its cross-domain effects on phonology. Children who received the intervention not only made significantly greater morphosyntactic change than that observed for a no-treatment control group but also made significantly greater phonology change with a large effect size (d = 1.35). In addition, the improvement in phonology from the morphosyntax intervention was similar to that achieved by a comparison hybrid speech sound intervention. Tyler et al. (2002) hypothesized that their morphosyntactic intervention, which focused specifically on finite morphemes (Haskill et al., 2001), resulted in phonological gains because of its indirect focus on segments in final cluster forms. In the larger project, which was aimed at treating multiple deficit areas in children with co-occurring SSD and LI, this morphosyntactic intervention was delivered in a randomized controlled trial in different goal attack configurations along with the speech sound intervention (Tyler, Lewis, Haskill, & Tolbert, 2003). Blocked sequences of the interventions were compared to alternating and simultaneous strategies over 24 weeks. A cycle of alternating the speech sound and morphosyntax intervention every other week was found to result in the greatest gains for both speech and language domains, as measured by generalization of speech targets to untrained contexts and language targets in conversational samples.
With respect to the use of broader interventions to ameliorate multiple deficit areas, such as those just described, it is noteworthy that not all possible skill areas were incorporated. For example, the PA intervention (Gillon, 2005) did not systematically incorporate a focus on oral language, and the morphosyntax intervention (Haskill et al., 2001) did not focus on awareness of phonemes and knowledge of letters. In a later study aimed at facilitating literacy for children with oral language impairments, an intervention focused on phonology and reading was compared with an intervention focused on vocabulary, language comprehension, and narrative skills (Bowyer-Crane et al., 2008). Children (N = 152) were identified at school entry (mean age: 4;9) on the basis of below average scores on a verbal composite measure. The results indicated that both interventions were effective in promoting improvements in their respective skill areas. At the conclusion of the 20-week daily intervention, however, more than 50% of the children in the study were still in need of literacy support, as indicated by a reading standard score less than 85 (SD [standard deviation] = −1). The extent of co-occurring SSD in these participants is unknown, although the authors' participant description suggests that some did exhibit speech errors.
Although preliminary research suggests that phoneme awareness can be successfully stimulated in three- to four-year-old children (Gillon, 2005), with associated success in early reading and spelling experiences, more research is required to establish effectiveness in the large subgroup of this population with co-occurring SSD and LI. Evidence suggests that because children with co-occurring SSD and LI are at risk for reading difficulty, and further, because phonological awareness intervention has been shown to be successful, intervention focused on phonological awareness should be provided during preschool to prevent early reading failure (Gillon, 2002; van Kleeck, Gillam, & McFadden, 1998). It is imperative that research continue to develop and assess behavioral interventions that have potential to produce gains in a variety of skills across linguistic domains. Research is also needed to compare the success of integrated interventions in stimulating multiple skills, some indirectly, thus testing intervention efficiency. The goals of this project were to (1) determine whether phoneme awareness could be stimulated in a previously unstudied group, that is, preschool children who have co-occurring SSD and LI; and (2) to examine gains in specific areas targeted by two different interventions, as well as the extent of indirect gains in nontargeted areas.
Gillon's (2005) phoneme awareness intervention with speech sound (PA/SS) production integrated, as well as the constituent elements of Tyler et al.'s (2003) morphosyntax/speech sound (MS/SS) alternating intervention were used in the present study. Both previously had been tested against no-treatment controls and had been shown to be efficacious. In addition, the MS/SS intervention had been proven more efficient than other combination strategies for the population of interest in the present study. Thus, by allowing each intervention to serve as an alternate form of intervention control, the following research questions were addressed:
1. As compared to an alternate control (MS/SS) intervention, does PA/SS intervention facilitate change in measures of phoneme identity and letter name knowledge?
2. As compared to the alternate control (PA/SS) intervention, does MS/SS intervention facilitate change in measures of finite morphemes and mean length of utterance (MLU), replicating previous findings?
3. Is there a difference in gains observed in percent consonants correct and cluster accuracy following PA/SS and MS/SS interventions?
A total of 30 children participated in the study, 18 from the United States and 12 from New Zealand, with an age range of 3;10 (years; months) to 5;2 (M = 4;5). There were 19 males and 11 females. The children displayed co-occurring speech and language impairments and met the following criteria for inclusion: (a) presence of SSD confirmed by a score at least one SD below the mean on the Goldman-Fristoe Test of Articulation (GFTA-2; Goldman & Fristoe, 2000); (b) documentation of expressive language score at least one SD below the mean on the Structured Photographic Expressive Language Test-Preschool 2 (SPELT-P2; Dawson et al., 2005) and/or 1.5 SDs below the mean MLU for the child's age based on Miller and Chapman's (2000) normative data; (c) age-appropriate receptive vocabulary as confirmed by a score within 1.5 SDs from the mean on the Peabody Picture Vocabulary Test-Third edition (PPVT-III; Dunn & Dunn, 1997); (d) normal functioning on oral motor assessment (Robbins & Klee, 1987); and (e) neurological, behavioral, hearing, and motor skills reported within normal limits. Mean assessment scores for all participants and for US and NZ cohorts are shown in Table 1. In both countries, the study was conducted under institutional review board–approved protocols, and children identified as potential participants had been referred to university clinics or were recruited through early education facilities.
Design and intervention timeline
A two-group randomized experimental design was used to examine the specificity of each intervention, both of which previously had been shown to be efficacious for improving targeted skills. One experimental group received the PA/SS intervention (Gillon, 2005) and the other received the MS/SS intervention (Tyler et al., 2003). At each site, children were matched in pairs for age and severity of speech disorder, and with consideration of receptive vocabulary and gender when possible. Close matching was achieved for age (M difference = 3.6 mo) and percent consonants correct (PCC; M difference = 7%), with only three of 15 pairs showing larger PCC differences because of extreme low scores of one member. One child from each matched pair was randomly assigned to the PA/SS intervention, and the other member of the pair was assigned to receive the MS/SS intervention. At the US site, a total of nine children were assigned to each of the intervention conditions, and at the NZ site, six were assigned to each condition. Rolling entry of cohorts into the project was necessary because of the subject identification/selection process. Cohorts of five or eight children entered the project at different starting points throughout years 2005 to 2006. Two US cohorts consisted of five children because they were identified and intervention was scheduled to begin. In the first cohort of five, the child for whom a match had not yet been found was assigned randomly; a match for this child was then identified from the next cohort, and this child was assigned to the other intervention.
Participants received two 6-week blocks of their assigned treatment, separated by a 6- to 7-week break from treatment. For example, one cohort was recruited, assessed, and received Block 1 of treatment from November to December. This group received Block 2 from February to March. Assessment sessions occurred at pretreatment, mid-treatment (after Block 1), and posttreatment (after Block 2). Intervention occurred in small groups of two to three children scheduled twice weekly for 60 min. Thus, each intervention comprised a total of 24 hours administered over 12 weeks separated into two blocks. The rationale for the two blocks was to provide time for assimilation/accommodation of new skills. Senior or Master's level speech-language pathology students who were trained on the interventions by the researchers administered both interventions. These clinicians were supervised by certified doctoral students or professional speech-language pathologists (SLPs).
Pre- and posttreatment measures were administered to both intervention groups. They measured a range of skills, including phoneme awareness, alphabetic knowledge, speech, and oral language skills. Some measures were designed to detect changes targeted directly for one group and not targeted (i.e., targeted only “indirectly”) for the comparison group, and vice versa. Selected measures were not available for the NZ sample, as described later. Measures to assess phonological awareness included seven tasks: rhyme detection (Bradley & Bryant, 1983); phoneme awareness probes for initial phoneme identity both with and without printed words, phoneme blending and segmentation; and letter name and letter sound knowledge (Gillon, 2005). The rhyme detection and phoneme identity without words tasks each had 10 items; letter name and phoneme identity with words had 12 items; blending and segmentation had five items each; and letter identification had 26 items (all letters).
Measures of spoken language outcomes (i.e., morphosyntactic change) that were available for the US participants only (60% of participants) included the finite morpheme composite (FMC; Bedore & Leonard, 1998; Rice, Wexler, & Cleave, 1995) and MLU from spontaneous language samples. These variables were coded using systematic analysis of language transcripts (SALT; Miller & Chapman, 2000). The FMC is calculated as combined percent accuracy in obligatory contexts of the following markers of tense and agreement: contractible and uncontractible copula and auxiliary forms of to be, past tense -ed, and third person singular regular. The FMC was selected because finite morphemes were the primary language targets for the MS/SS intervention, and it has been shown that the finite system is especially vulnerable, in comparison to other grammatical markers, for preschoolers with LI (Bedore & Leonard, 1998; Rice, Wexler, & Cleave, 1995; Rice, Wexler, & Hershberger, 1998; Willet & Tyler, 2001). Language samples were collected at each assessment point from an interactive narrative retelling of a script developed for the wordless picture book, Carl Goes Shopping (Carl series by Alexandra Day, 1995); the script provided opportunities for production of the finite morphemes of interest.
Percent consonants correct (Shriberg & Kwiatkowski, 1982) was the primary measure of speech change. It was derived from computerized analysis (PROPH; Computerized Profiling, Long, Fey, & Channell, 2008) of participants' responses from the GFTA-2 and 25 additional words (Dodd, 1995). The GFTA-2 and additional words were administered only at pretreatment, as part of the initial selection assessment, and posttreatment after Block 2. Cluster accuracy was determined for the US cohort only from analysis of a cluster probe consisting of 15 single words selected to provide five opportunities each for production of initial /sp-, st-, sl-/ clusters. This probe was administered at all three assessment points.
Pretreatment assessment sessions were held 2 weeks prior to the onset of treatment, mid-treatment assessment sessions were held within 1 week of completing Block 1 (mid-treatment), and posttreatment assessment sessions were held 2 weeks after the second treatment block. Samples were audiotaped using a Sony ICD-P320 digital audio recorder with one or more external lapel microphones. Samples were elicited in small, quiet rooms in the sites' respective university clinics by speech-language pathology graduate clinicians, doctoral students, and the project SLP, none of whom was involved in treatment of the participant being assessed. These individuals also assisted in transcribing and scoring. As much as feasible, they were blind to treatment condition and sample number for the participants, but timing of the analysis tasks prevented complete blinding from information about sample number (pre, mid, or post).
Language samples were first transcribed by graduate assistants who had been trained in coding for subsequent SALT analysis. Child and examiner utterances were entered along with codes on child utterances to enable later determination of correct usage in obligatory contexts for grammatical morphemes, in particular the finite morphemes. SALT was used to determine MLU in morphemes and to identify each instance of correct, incorrect, and omitted grammatical morphemes. The trained SLP project assistant, serving as expert coder, then relistened to every language sample and verified the accuracy of the transcription and coding, modifying codes when errors or omissions occurred. From this corrected sample, percentage correct usage for each finite morpheme was calculated after verifying each instance of correct, incorrect, and omitted morphemes. The FMC was derived by dividing the total number of finite morpheme correct usages by the total number of obligatory contexts. As a final reliability check on accuracy of the FMC, the first author rechecked finite morpheme codes and recalculated FMC for each language sample. The percentage of agreement from the final reliability check for 10% of the participant samples was 96% for the identification of obligatory contexts and 98% for coding accuracy.
Broad phonetic transcriptions using the IPA were made online by graduate research assistants during pre- and posttreatment administrations of the GFTA-2 and 25 words. These transcriptions were then checked and modified from audiotape replay by one of three trained transcribers; a graduate assistant, doctoral candidate (third author), or the first author. Phonetic transcription reliability was assured through training to a criterion level of at least 90% agreement for the three expert transcribers. Training involved having raters independently transcribe GFTA-2 responses from two samples. Point-to-point reliability was calculated on the basis of each judge's transcription of each consonant. Segmental transcriptions that were identical (excluding diacritics) were coded as agreements. The three transcribers subsequently relistened to the training samples together to achieve consensus for any phonemes on which there was disagreement. They then independently transcribed the same two additional participant samples as a further reliability check. Their point-to-point agreement for these samples was calculated at 91%.
As a check on the reliability of scoring for the seven phonological awareness probe tasks, 13% of the probes from the three different assessment points across all participants were randomly selected for scoring by an independent judge. This judge rescored the recorded probe responses, and agreement was calculated to be 98%.
Intervention goals and procedures
The two interventions used in this study were the PA/SS (Gillon, 2005), which involved phoneme awareness and letter/sound knowledge, integrated with speech sound production and the MS/SS, which involved a morphosyntax intervention and a speech sound intervention provided in alternate weeks (Tyler et al., 2003). The PA/SS did not target morphosyntax directly and the MS/SS did not target phoneme awareness and letter/sound knowledge directly. Both interventions were manualized with written instructions, scripts, material lists and patterns, stimulus pictures, and books.
Phoneme awareness with integrated speech sound production
The PA/SS intervention was designed to use speech production practice to facilitate early phoneme awareness and letter knowledge simultaneously while reducing common speech error patterns (Gillon & Moriarty, 2005). For each group receiving the PA/SS intervention, a major error pattern was selected so as to provide a focus for selecting the stimulus items used in phoneme awareness and letter knowledge tasks. Speech goals were selected for both intervention conditions from error pattern results provided by the PROPH phonological process analysis. Error patterns that were prevalent in a child's speech and early developing received highest priority. Participants in the PA/SS intervention targeted early stopping of fricatives, final consonant deletion, /s/-cluster reduction, velar fronting, and infrequently gliding of /l/.
Phoneme awareness and letter knowledge activities were embedded in clinician directed play activities and games that encouraged the children's active participation. Tasks to facilitate phoneme awareness focused on the following skills: phoneme detection (e.g., “Let's find the word that starts with the /f/ sound” or “Does ball start with an /f/ sound?”); phoneme categorization (e.g., “Find all the toys that start with /k/”); initial phoneme matching (e.g., “Corn starts with a /k/ sound. Let's find the one that starts the same as corn: carrot, potato”); and phoneme isolation (e.g., “What sound does mouse start with?”). Letter name and letter sound knowledge were first introduced by using recognition activities. Letters were gradually introduced into the sessions beginning with a small group of these that were associated with the child's speech targets. Subsequently, additional groups of two or three letters were introduced using posters with lower case letters written in very large print. Picture cards with the name written under the picture were used in consonant bingo games to highlight the relationship between phonemes and graphemes in word initial position. Stimulus words used in activities and print were selected to contain target sounds in simple syllable structures.
Morphosyntax/speech sound alternating
The MS/SS intervention involved implementing the morphosyntax intervention, manualized in Months of Morphemes (Haskill et al., 2001) one week and a hybrid speech sound intervention the following week to improve expressive language and reduce speech error patterns, respectively. These two interventions were alternated every other week over each 6-week block. The specific finite morphemes targeted for all groups of children randomly assigned to receive this intervention were copula and auxiliary forms of to be, past tense -ed, and third person singular regular. The error patterns targeted in the alternating weeks that focused on speech sound production were final consonant deletion, velar fronting, stopping of fricatives and affricates, and /s/-cluster reduction.
Morphosyntax intervention procedures involved auditory awareness activities, focused stimulation activities, and elicited production activities (Camarata, Nelson, & Camarata, 1994; Cleave & Fey, 1997; Fey, Cleave, Long, & Hughes, 1993; Nelson, 1989). Detailed written scripts were available for all activities for each session to ensure reliable implementation of the intervention (Haskill et al., 2001). Auditory awareness activities were designed to heighten children's awareness of the morphosyntactic targets in the context of children's books and songs that were read and sung in each session. Focused stimulation activities were designed to provide children with multiple models of target structures in a natural communicative context. An average of 75 to 80 models of each morpheme were provided in each script; they involved recasts and expansions of children's utterances. Elicited production activities were implemented, with the goal of eliciting 35 to 45 productions of each target morpheme through providing opportunities to use forms in response to contextually relevant questions or prompts. These activities primarily involved forced choice and cloze tasks designed to obligate the production of a morpheme by either providing the child with the choice of two responses, both of which contained the target morpheme (“The man jumps or runs?”), or by giving the child opportunity to complete a sentence (“This man jumps and then look, he __”).
The speech sound intervention component of the MS/SS intervention, similar in structure to the morphosyntax component, involved auditory awareness activities and production practice in drill play and naturalistic activities. The awareness activities involved description and identification of the target sound in isolation, as contrasted to dissimilar sounds, and reading children's books that contained extensive alliteration of the target sound. Drill play activities were designed to allow establishment of new sound productions in single words and naturalistic activities facilitated productions in spontaneous communicative situations.
Multiple procedures were used to ensure that the two interventions were implemented as manualized. First, primary investigators trained graduate student clinicians and supervising SLPs on the intervention procedures through review of manuals and videotaped examples. Intervention manuals contained detailed scripts and patterns for standard sets of stimulus items. All intervention sessions were video recorded and clinicians also kept data sheets with records of the activities they administered during each session. Videotapes from a total of 10% of the intervention sessions from each site were selected randomly, split equally among the two intervention conditions, and analyzed by an independent judge for the presence of major components of each intervention protocol. Across both sites, 92% (range: 75%–100%) of the required activities and procedures were incorporated as specified in the intervention manuals. The interventions developed by the respective researchers were implemented with slightly higher percentages at their own sites (95%–100%).
The first two research questions asked if each of the interventions in this study produced change over time in the specific skill areas they were designed to target (as compared to the alternate control intervention). That is, did the PA/SS intervention produce greater differences from pre- to posttreatment in the dependent measures of phoneme identification and letter name knowledge than the MS/SS intervention. Similarly, did the MS/SS intervention facilitate greater change in the dependent measures, FMC and MLU, than the PA/SS intervention. The third question concerned change in speech accuracy dependent measures, PCC, and /s/-cluster accuracy, as compared for the two interventions, both of which targeted speech sound production. Initial pretreatment levels of all dependent measures were statistically analyzed for group differences. Independent t tests indicated there were no significant group differences with respect to initial levels of all dependent variables, including those obtained from the entire group of participants: PCC, letter naming, phoneme identification (t(28) = −0.35 to 0.65, p = .52 to .78); and additional measures obtained from the US cohort only: MLU, FMC, cluster accuracy (t(16) = −0.43 to 1.64, p = .12 to .76). This served as confirmation of the success of random assignment to treatment groups in achieving pretreatment similarity.
Factorial mixed-design (2 × 3) analysis of variance (ANOVA) procedures were applied for the 30 participants to answer the first research question regarding differences in phoneme identification and letter naming at the three assessment points, pretreatment, mid-treatment, and posttreatment, for the two intervention groups. Results are displayed in Table 2. The ANOVA results for phoneme identification indicated that there was a significant main effect for time, F(1, 28) = 14.61, p < .001, but no significant group difference or group-by-time interaction. This interaction was characterized by a small effect (η2 = .024). All follow-up pairwise comparisons of phoneme identification scores were significant, indicating that mid-treatment scores were significantly greater than pretreatment scores (p < .05) and posttreatment scores were significantly greater than mid-treatment scores (p < .01). The ANOVA results for letter naming also revealed a significant main effect for time, F(1, 28) = 11.48, p < .001. The results for group and the group-by-time interaction were nonsignificant, although both approached significance (p = .07 and .061, respectively) and showed a medium effect (η2 = .112 and .095, respectively). Results for the group-by-time interaction are displayed in Figure 1. Follow-up pairwise comparisons for letter naming scores across assessment points revealed significantly higher mid-treatment scores as compared to pretreatment (p < .001) and significantly higher posttreatment scores as compared to pretreatment (p < .05).
The ANOVA procedures also were used to determine whether there were differences at pre-, mid-, and posttreatment across the two intervention groups for the US cohort on the language measures, FMC and MLU, as addressed by the second research question. The ANOVA results for MLU indicated that there was a significant main effect for time, F(1, 15) = 4.91, p < .05, but no significant main effect for group or for the group-by-time interaction, although this interaction showed a medium effect (η2 = .099). Follow-up pairwise comparisons for MLU at the different assessment points revealed that posttreatment scores compared to pretreatment scores approached significance (p = .07), but pretreatment and posttreatment scores were not significantly different from mid-treatment MLU scores. For FMC, the ANOVA result for time approached significance, F(1, 15) = 3.15, p = .057; there was no significant main effect for group or group-by-time interaction, although this interaction showed a medium effect (η2 = .102).
For the third research question, which addressed change over time in speech accuracy measures for both intervention groups, group-by-time ANOVA procedures were applied to PCC and cluster accuracy scores. It should be noted that PCC is not available from the mid-treatment assessment because of the concern of repeated readministration of the GFTA-2. Instead, cluster accuracy and other individualized probes were administered at the mid-treatment assessment. The ANOVA results for PCC showed a significant effect for time from the pretreatment to posttreatment assessments, F(1, 28) = 54.40, p < .001). Neither the main effect for group nor the interaction was significant. The interaction was characterized by a small effect (η2 = .006). The ANOVA results for cluster accuracy for the US cohort also revealed a significant main effect for time, F(2, 15) = 11.73, p < .001, and no significant main effect for group or group-by-time interaction, although the main effect for group showed a large effect (η2 = .143) and the interaction showed a medium effect (η2 = .119). Follow-up pairwise comparisons of cluster accuracy scores at different assessment points indicated that posttreatment scores were significantly higher than both pretreatment (p < .01) and mid-treatment (p < .05) scores.
This study was designed to determine whether phoneme awareness could be stimulated in preschool children with co-occurring SSD and LI, as well as to examine the efficiency of two contrasting evidence-based interventions. Matched participants were assigned randomly to receive one of the two interventions for two 6-week periods, separated by a 6-week break. One of the interventions, PA/SS, targeted phoneme awareness directly and integrated speech-sound but not morphosyntactic skills. The other intervention, MS/SS, targeted speech sound production directly for 1 week and morphosyntactic skills directly the next week for the two 6-week periods, but not phoneme awareness and letter/sound knowledge. Measures were designed to detect growth in specific skills targeted directly by one of the two programs, but not directly targeted in the other.
Results showed that both intervention groups, those receiving PA/SS and those receiving MS/SS intervention, made statistically significant gains in all measures, with the exception of the finite morpheme measure, which approached significance. Posttreatment scores were significantly higher than pretreatment scores, and in most cases, pretreatment and posttreatment scores were not different relative to mid-treatment scores. The lack of significant intervention group differences in several of the dependent measures is mitigated by medium effect sizes for nonsignificant group-by-time interactions, suggesting that increased power may have revealed group differences. Generally, results trended toward hypothesized findings of differences favoring specific skills targeted by each intervention, but failure to find statistically significant differences may have been because of large SDs and small sample sizes. For example, letter naming results showed a strong trend of better performance over time for the PA/SS intervention group; likewise, both language measures showed trends in the hypothesized direction of greater gains for the MS/SS intervention. Both groups also made similar gains in speech accuracy measures; however, these were targeted within both interventions.
The finding that skills of letter naming and phoneme identification improved as a result of the PA/SS intervention indicates that these skills can be enhanced through intervention in children with severe co-occurring SSD and LI. The PA/SS intervention used in this study focused largely on establishing letter knowledge, sound-letter correspondence, and word-initial phoneme awareness as a foundation for blending and segmentation. These results extend those of Hesketh, Dima, and Nelson (2007) in showing that word-initial phoneme identification can be facilitated in preschool children with SSD who also have co-occurring LI. The findings also confirm Hesketh et al.'s (2007) in suggesting that more advanced phoneme manipulation skills may not display substantial improvement when children are preschool-aged. In the present study, phoneme segmentation could not be analyzed because of an apparent floor effect and limited change.
The additional finding that the group-by-time interaction for letter naming approached significance for the sample of 30 lends weak support for the hypothesis that the PA/SS intervention was the more efficient of the two interventions for increasing letter knowledge skills. Phoneme identification skills, in contrast, showed less of an advantage for the PA/SS group in comparison to the MS/SS group. A smaller effect for the PA/SS intervention might have been hypothesized in comparison to that achieved for SSD-only participants in previous studies because of the added LI experienced by the participants in the present study. Although children receiving the MS/SS intervention did not have goals, targets, or activities focused on identification of initial phonemes in words, it is quite possible that this skill was enhanced indirectly through production of initial speech sound targets that was a part of this program. The byproduct of phonological awareness improvement was also observed by Hesketh et al. (2000), who found no differences in performance on a phonological awareness measure between a group receiving a phonological awareness intervention and one that had received an articulation-based intervention. Hesketh et al. (2000) suggested that such findings provide evidence of a clear phoneme awareness benefit from speech production treatment, although the depth achieved across phonological awareness skills from speech production treatment is unknown. It is also possible that both phoneme identification and letter naming skills were facilitated through some of the participants' curricula in early childhood education. The majority of participants attended some type of early childhood program at least part-time, although the type of program and curricula varied.
The finding that children who received the MS/SS intervention showed some improvement in initial phoneme awareness is tempered by the opposite finding. That is, the children receiving the PA/SS intervention also showed improvement in language measures, although small in comparison to improvements for the children receiving the MS/SS intervention where grammar was a direct focus. The MLU growth may have been facilitated by both interventions simply through the “talk” that occurred in the group interactions during both phonological awareness and language activities. The language measures used as dependent variables in this study were FMC and MLU because the language component of the MS/SS intervention focused specifically on tense and agreement morphemes. The fact that differences in FMC from pre- to posttreatment assessments approached significance (p = .057) suggests that had the sample been larger, statistical significance might have been achieved. The same holds true for the group-by-time interaction for FMC, which showed a medium effect size, thus favoring the effect of the MS/SS intervention over time for gains in finite morphemes. Nonetheless, change in FMC highlights the protracted course of development for finite markers shown by children with LI (Rice et al., 1998). In addition, there is evidence that children with co-occurring SSD and LI show lower finite morpheme production than those with specific language impairment without co-occurring SSD (Haskill & Tyler, 2007). For example, even after 24 months of intervention during which language was a focus for approximately half of the time, and change was statistically significant, FMC scores for a group of 40 preschool children with co-occurring SSD and LI averaged only 61% (Tyler et al., 2003).
The finding that both groups made similar and highly significant gains in speech production measures adds to evidence that the PA/SS intervention used in the present study has the capacity to improve speech accuracy. The PA/SS intervention can produce equally as good change in percent consonants correct as the intervention focused on morphosyntax and speech (MS/SS) and even appeared to promote superior change in /s/-clusters. Increasing evidence suggests that among all deficits observed in children with co-occurring SSD and LI, speech may be the one most amenable to short-term change if there is not significant motor involvement.
Both interventions implemented in this study had previously been shown to be effective in comparison to no-treatment controls, but the lack of a no-treatment control group in the current study means that maturation as a factor cannot be completely ruled out. Although random assignment of participants to the two different interventions confirmed equivalency on experimental measures, there were no significant group effects. A preliminary conclusion might suggest that the different intervention strategies are equally efficient in facilitating growth across multiple skill domains in the children with severe SSD and LI who participated in this study. There were, however, some clear trends in the data suggesting greater gains in the measures for skills specifically targeted in the PA/SS intervention and the MS/SS intervention. For example, the PA/SS intervention group appeared to make greater improvement in letter naming, as demonstrated by the results approaching significance with a medium effect size. The MS/SS group displayed a trend of growth in FMC that contrasted with a more stable (flat) pattern for the PA/SS group. Although individual variability reflected in large SDs may have played a role in the lack of intervention-specific gains, it was more likely the relatively small sample sizes (US only) that interfered with difference tests. Had the sample been larger, especially for language measures, we might have found significant interactions for some of the dependent variables.
The two interventions in this study differentially focused on morphosyntax and phonological awareness skills and appeared to achieve intervention-specific effects, although gains were also observed in nontarget domains. Both interventions focused on speech sound production, but did so in a different format. Because significant change in PCC trended similarly for the two interventions, we have even further evidence that speech change can be achieved within a multifaceted focus (i.e., the MS/SS intervention; Tyler et al., 2003) and integrative teaching formats (i.e., the PA/SS intervention; Gillon, 2005).
The trends in favor of the specificity of each intervention suggest that the possibility of treatment-specific effects was obscured by small samples. These results are preliminary and highlight the need to extend the findings with larger clinical samples. The gains obtained across skill areas from both interventions did not, however, bring the children's skills within age-level expectations, punctuating the need to evaluate individual children's progress in all domains—speech sound production, early morphosyntax, and phonological awareness and other emergent literacy skills. Not only is it important to continue to test multifaceted interventions, but it also is important to examine a care path for children with co-occurring SSD and LI. Further research may help to unveil when in the developmental course it may be necessary to shift the focus of treatment as gains are made differentially across domains.
These results also suggest that if a speech-language intervention achieves the byproduct of change in the skill of phoneme identification, it will then be necessary to monitor more advanced skills such as phoneme segmentation and manipulation for continued development. This highlights the added value of a follow-up assessment. The skill of initial phoneme identification has been shown to prepare children with speech impairment for later phonological awareness and literacy development (Gillon, 2005); however, children in the current study also had markedly delayed grammatical skills (M MLU = 2.95). The magnified impact on phonological awareness of poor linguistic competence would be cause for determining whether it is necessary to provide a short-term focus on more complex phonological awareness skills such as segmentation. It is noteworthy that, as a group, the US participants' morphosyntax still remained markedly delayed at an average MLU of 3.61 and FMC of 35% at the end of the intervention period. A factor contributing to this protracted rate of change may be the relatively small amount of intervention provided. Participants received two, 6-week blocks (12 weeks total) during which language goals were targeted in alternating weeks (6 weeks total). Substantial change in oral language has often been achieved within a longer or more intensive schedule of service delivery (Bowyer-Crane et al., 2008; Law, Garrett, & Nye, 2004). These results further confirm the need for intensive focus during the preschool years on oral language skills, and the likely need for continued intensive intervention that integrates oral language and phonological awareness/early literacy if these children's risk of continued academic difficulties is to be reduced prior to or at the onset of school entry (Bowyer-Crane et al., 2008).
Aram D., Ekelman B., Nation J. (1984). Preschoo-lers with language disorders: Ten years later. Journal of Speech and Hearing Research, 27, 232–244.
Aram D. M., Hall N. E. (1989). Longitudinal follow-up of children with preschool communication disorders: Treatment implications. School Psychology Review, 18, 487–501.
Bedore L. M., Leonard L. B. (1998). Specific language impairment and grammatical morphology: A discriminant function analysis. Journal of Speech, Language, and Hearing Research, 41, 1185–1192.
Bird J., Bishop D. V. M., Freeman N. H. (1995). Phonological awareness and literacy development in children with expressive phonological impairments. Journal of Speech and Hearing Research, 38, 446–462.
Bowyer-Crane C., Snowling M. J., Duff F. J., Fieldsend E., Carroll J. M., Miles J., et al. (2008). Improving early language and literacy skills: Differential effects of an oral language versus a phonology with reading intervention. Journal of Child Psychology and Psychiatry, 49, 422–432.
Bradley L., Bryant P. (1983). Categorizing sounds and learning to read: A causal connection. Nature, 301, 419–421.
Broomfield J., Dodd B. (2004). Children with speech and language disability: Caseload characteristics. International Journal of Language and Communication Disorders, 39, 303–324.
Camarata S., Nelson K., Camarata M. (1994). Comparison of conversational-recasting and imitative procedures for training grammatical structures in children with specific language impairment. Journal of Speech and Hearing Research, 37, 1414–1423.
Catts H. (1993). The relationship between speech-language impairments and reading disabilities. Journal of Speech and Hearing Research, 36,948–958.
Catts H. W., Fey M. E., Zhang X., Tomblin J. B. (2001). Estimating the risk of future reading difficulties in kindergarten children: A research-based model and its clinical implications. Language, Speech, and Hearing Services in Schools, 32, 38–50.
Cleave P. L., Fey M. (1997). Two approaches to the facilitation of grammar in children with language impairments: Rationale and description. American Journal of Speech-Language Pathology, 6, 22–32.
Conti-Ramsden G., Botting N. (1999). Classification of children with specific language impairment: Longitudinal considerations. Journal of Speech, Language, and Hearing Research, 42, 1195–1204.
Dawson J., Stout C., Eyer J., Tattersall P., Foukalsrud J., Croley K. (2005). Structured Photographic Expressive Language Test—Preschool 2. Dekalb, IL: Janelle.
Day A. (1995). Carl goes shopping. New York: Farrar, Straus & Giroux.
Denne M., Langdown N., Pring T., Roy P. (2005). Treating children with expressive phonological disorders: Does phonological awareness therapy work in the clinic? International Journal of Language and Communication Disorders, 40, 493–504.
Dodd B. (1995). Differential diagnosis and treatment of children with speech disorder. San Diego, CA: Singular Publishing.
Dodd B., Gillon G. (2001). Exploring the relationship between phonological awareness, speech impairment and literacy. Advances in Speech-Language Pathology, 3, 139–147.
Dunn L. M., Dunn L. M. (1997). Peabody Picture Vocabulary Test III. Circle Pines, MN: American Guidance Service.
Felsenfeld S., Broen P. A., McGue M. (1994). A 28-year follow-up of adults with a history of moderate phonological disorder: Educational and occupational results. Journal of Speech and Hearing Research, 37, 1341–1353.
Fey M. E., Cleave P. L., Long S., Hughes (1993). Two approaches to the facilitation of grammar in children with language impairment: An experimental evaluation. Journal of Speech and Hearing Research, 36, 141–157.
Gierut J. A. (1998). Treatment efficacy: Functional phonological disorders in children. Journal of Speech and Hearing Disorders, 41, S85–S100.
Gillon G. (2000). The efficacy of phonological awareness intervention for children with spoken language impairment. Language, Speech, and Hearing Services in Schools, 31, 126–141.
Gillon G. (2002). Follow-up study investigating benefits of phonological awareness intervention for children with spoken language impairment. International Journal of Language and Communication Disorders, 37, 381–400.
Gillon (2005). Facilitating phoneme awareness development in 3- and 4-year-old children with speech impairment. Language, Speech, and Hearing Services in Schools, 36, 308–324.
Gillon G., Moriarty B. (2005). Integrated phonological awareness: An intervention program for preschool children with speech-language impairment. Canterbury, New Zealand: Department of Communication Disorders, University of Canterbury.
Goldman R., Fristoe M. (2000). Goldman-Fristoe Test of Articulation–Second Edition. Circle Pines, MN: American Guidance Service.
Harbers H., Paden E., Halle J. (1999). Phonological awareness and production: Changes during intervention. Language, Speech, and Hearing Services in Schools, 30, 50–60.
Haskill A., Tyler A. A. (2007). A comparison of linguistic profiles in subgroups of children with specific language impairment. American Journal of Speech-Language Pathology, 16, 209–221.
Haskill A., Tyler A., Tolbert L. C. (2001). Months of morphemes. Eau Claire, WI: Thinking Publications.
Hesketh A., Adams C., Nightingale C., Hall R. (2000). Phonological awareness therapy and articulatory training approaches for children with phonological disorders: A comparative outcome study. International Journal of Language and Communication Disorders, 35, 337–354.
Hesketh A., Dima E., Nelson V. (2007). Teaching phoneme awareness to pre-literate children with speech disorders: A randomized controlled trial. International Journal of Language and Communication Disorders, 42, 251–271.
Hoffman P. R., Norris J. A. (2005). Intervention: Manipulating complex input to support self-organization of a neuro-network. In Kamhi A. G., Pollock K. E. (Eds.), Phonological disorders in children: Clinical decision making in assessment and intervention (pp. 139–155). Baltimore: Brookes.
Hoffman P. R., Norris J. A. (2010). Dynamic systems and whole language intervention. In Williams A. L., McLeod S., McCauley R. J. (Eds.), Interventions for speech sound disorders in children (pp. 333–354). Baltimore: Brookes.
Johnson C. J., Beitchman J. H., Young A., Escobar M., Atkinson L., Wilson B., et al. (1999). Fourteen-year follow-up of children with and without speech/language impairments: Speech/language stability and outcomes. Journal of Speech, Language, and Hearing Research, 42, 744–760.
Larrivee L. S., Catts H. W. (1999). Early reading achievement in children with expressive phonological disorders. American Journal of Speech-Language Pathology, 8, 118–128.
Law J., Garrett Z., Nye C. (2004). The efficacy of treatment for children with developmental speech and language delay/disorders: A meta-analysis. Journal of Speech, Language, and Hearing Research, 47, 924–943.
Leitao S., Fletcher J. (2004). Literacy outcomes for students with speech impairment: Long-term follow-up. International Journal of Language and Communication Disorders, 39, 245–256.
Lewis B. A., Freebairn L. A., Taylor H. G. (2000). Academic outcomes in children with histories of speech sound disorders. Journal of Communication Disorders, 33, 11–30.
Miller J., Chapman R. (2000). SALT: Systematic analysis of language transcripts, version 6.1. Madison, WI: Language Analysis Laboratory, Waisman Center, University of Wisconsin.
Nathan L., Stackhouse J., Goulandris N., Snowling M. J. (2004). The development of early literacy skills among children with speech difficulties: A test of the “critical age hypothesis.” Journal of Speech, Language, and Hearing Research, 47, 377–391.
Nelson K. (1989). Strategies for first language teaching. In Rice M., Schiefelbusch R. (Eds.), The teachability of language (pp. 263–310). Baltimore: Brookes.
Pennington B. F., Bishop D. V. M. (2009). Relations among speech, language, and reading disorders. Annual Review of Psychology, 60, 283–306.
Peterson R. L., Pennington B. F., Shriberg L. D., Boada R. (2009). What influences literacy outcome in children with speech sound disorder? Journal of Speech, Language, and Hearing Research, 52, 1175–1188.
Raitano N. A., Pennington B. F., Tunick R. A., Boada R., Shriberg L. D. (2004). Preliteracy skills of subgroups of children with speech sound disorders. Journal of Child Psychology and Psychiatry, 45, 821–835.
Rapin I., Allen D. A. (1983). Developmental language disorders: Nostalgic considerations. In Kirk U. (Ed.), Neuropsychology of language reading, and spelling (pp. 155–184). New York: Academic Press.
Rapin I., Allen D. A. (1988). Syndromes in developmental dysphasia and adult aphasia. In Plum F. (Ed.), Language, communication, and the brain (pp. 57–75). New York: Raven Press.
Rice M., Wexler K., Cleave P. (1995). Specific language impairment as a period of extended optional infinitive. Journal of Speech and Hearing Research, 38, 850–863.
Rice M., Wexler K., Hershberger S. (1998). Tense over time: The longitudinal course of tense acquisition in children with specific language impairment. Journal of Speech, Language, and Hearing Research, 41, 1412–1430.
Robbins J., Klee T. (1987). Clinical assessment of oropharyngeal motor development in young children. Journal of Speech and Hearing Disorders, 52, 271–277.
Rvachew S., Ohberg A., Grawburg M., Heyding J. (2003). Phonological awareness and phonemic perception in 4-year-old children with delayed expressive phonology skills. American Journal of Speech-Language Pathology, 12, 463–471.
Shriberg L. D. (2010). Childhood speech sound disorders: From postbehaviorism to the postgenomic era. In Paul R., Flipsen P. (Eds.), Speech sound disorders in children (pp. 1–33). San Diego, CA: Plural Publishing.
Shriberg L. D., Austin D. (1998). Comorbidity of speech-language disorders: Implications for a phenotype marker for speech delay. In Paul R. (Ed.), The speech-language connection (pp. 73–117). Baltimore: Brookes.
Shriberg L. D., Kwiatkowski J. (1988). A follow-up study of children with phonologic disorders of unknown origin. Journal of Speech and Hearing Disorders, 53, 144–155.
Shriberg L. D., Kwiatkowski J. (1982). Phonological disorders III: A procedure for assessing severity of involvement. Journal of Speech and Hearing Disorders, 17, 256–270.
Snowling M., Bishop D. V. M., Stothard S. E. (2000). Is preschool language impairment a risk factor for dyslexia in adolescence? Journal of Child Psychology and Psychiatry and Allied Disciplines, 41, 587–600.
Tallal P., Ross R., Curtiss S. (1989). Familial aggregation in specific language impairment. Journal of Speech and Hearing Disorders, 54, 167–173.
Tyler A. A. (2008). What works: Evidence-based intervention for children with speech sound disorders. Seminars in Speech and Language, 29, 320–330.
Tyler A. A. (2010). Subgroups, comorbidity, and treatment implications. In Paul R., Flipsen P. (Eds.), Speech sound disorders in children: In honor of Lawrence D. Shriberg (pp. 71–92). San Diego, CA: Plural Publishing.
Tyler A. A., Lewis K. E., Haskill A., Tolbert L. C. (2002). Efficacy and cross-domain effects of a phonology and morphosyntax intervention. Language, Speech, and Hearing Services in Schools, 33, 52–66.
Tyler A. A., Lewis K. E., Haskill A., Tolbert L. C. (2003). Outcomes of different speech and language goal attack strategies. Journal of Speech, Language, and Hearing Research, 46, 1077–1094.
Van Kleeck A, Gillam R. B., McFadden T. U. (1998). A study of classroom-based phonological awareness training for preschoolers with speech and/or language disorders. American Journal of Speech-Language Pathology, 7, 65–76.
Willet H., Tyler A. A. (2001, November). Effects of treatment on finite morphemes in children with SLI. Presented at the Annual Convention of the American Speech-Language-Hearing Association, New Orleans, LA.
Cross-cultural; intervention; language impairment; phoneme awareness; speech sound disorders
© 2011 Lippincott Williams & Wilkins
What does "Remember me" mean?
By checking this box, you'll stay logged in until you logout. You'll get easier access to your articles, collections,
media, and all your other content, even if you close your browser or shut down your
To protect your most sensitive data and activities (like changing your password),
we'll ask you to re-enter your password when you access these services.
What if I'm on a computer that I share with others?
If you're using a public computer or you share this computer with others, we recommend
that you uncheck the "Remember me" box.
Highlight selected keywords in the article text.
Data is temporarily unavailable. Please try again soon.
Readers Of this Article Also Read