Skip Navigation LinksHome > Fall 2010 - Volume 22 - Issue 3 > A Longitudinal Study of Outcome Measures for Children Receiv...
Pediatric Physical Therapy:
doi: 10.1097/PEP.0b013e3181e94464
Research Report

A Longitudinal Study of Outcome Measures for Children Receiving Early Intervention Services

Eigsti, Heidi Johnson PT, PhD, DPT, PCS; Chandler, Lynette PT, PhD; Robinson, Cordelia PhD, RN; Bodkin, Amy Winters PT, PhD

Free Access
Article Outline
Collapse Box

Author Information

Rocky Mountain University of Health Professions, Provo, Utah (Dr Eigsti); Regis University, Denver, Colorado (Dr Eigsti); University of Puget Sound, Tacoma, Washington (Dr Chandler); JFK Partners, University of Colorado Denver, Aurora, Colorado (Dr Robinson); Center for Gait and Movement Analysis, University of Colorado Denver, Aurora, Colorado (Dr Bodkin).

Correspondence: Heidi Eigsti, PT, PhD, DPT, PCS, 3333 Regis Blvd, Regis University, Reuckert-Hartman College of Health Professions, School of Physical Therapy, Denver, CO 80210 (heigsti@regis.edu).

Grant Support: Section on Pediatrics, American Physical Therapy Association awarded to Dr Eigsti.

This work was completed while Dr Eigsti was a student at Rocky Mountain University.

Collapse Box

Abstract

Purpose: The purpose of this study was to compare the Pediatric Evaluation of Disability Inventory (PEDI) and the Mullen Scales of Early Learning (MSEL) as measures of change in children who received early intervention services.

Methods: Thirty-four children were stratified into 2 groups according to the presence of gross motor delay. The PEDI and MSEL were administered 3 times: at an average age of 18, 31, and 53 months of age. Data were analyzed using a repeated-measures multivariate analysis of variance.

Results: The findings suggest that PEDI Functional Skills Scaled Scores were capable of measuring change in both groups of children. The standard scores on the PEDI Functional Skills Social Scale were found to be more sensitive to change than the MSEL Receptive and Expressive Language scores for children with motor delays.

Conclusion: Using PEDI scaled scores may be an effective strategy for measuring change in children receiving early intervention services.

Back to Top | Article Outline

INTRODUCTION

Children with chronic health conditions and disabilities account for the highest prevalence of disabling conditions.1 Health care and early intervention providers must be able to document and assess the outcomes related to these conditions.1 In a recent report of the National Research Council and the Institute of Medicine, the Committee on Evaluation of Children's Health stated that there was a need for “a broader range of data on functioning across physical, cognitive, emotional, and social domains.”2 The United States Office of Special Education Programs is also calling for data as evidence of the effectiveness of federally supported intervention programs for children with disabilities.3 Outcome information obtained through longitudinal studies will provide information about the progress of children with disabilities. Additional benefits of outcome measurement include ensuring continuation of federal funding, improved program planning, informed decision making, and improved guidance in goal development.4 Unfortunately, documenting outcomes of children receiving Part C early intervention services presents challenges because it requires the use of measures that are sensitive to small changes in children with differing developmental and functional abilities. Kolobe et al5 suggested that a measure must be sensitive to change before it could be responsive. In other words, a measure must have the capacity to measure change over time before it can be compared with a judgment of meaningful change or a gold standard. The operational definition for sensitivity to change used in this report comes from DeBruin et al6 who state that sensitivity to change is the “accurate detection of change when it has occurred.” Comparative research is needed to determine the relative merits and sensitivity of commonly used outcome measures for specific populations of children. Data from different measures can then be judged using standardized indices of sensitivity to change. One of the responsibilities of local providers of early intervention services is to maximize the use of scientific evidence to advocate for valid, reliable, and responsive assessment practices in their local and state programs. Unfortunately, longitudinal studies of the functional abilities of children with disabilities in early intervention programs are few, providing stakeholders with little guidance in developing an outcome measurement system.7,8

According to Jette,9 the World Health Organization's International Classification of Functioning, Disability, and Health (ICF) provides a framework for outcome measurement by differentiating the various components of involvement of disease and injury from a medical and social perspective. The ICF identifies 3 domains of human function: body function, activity, and participation. Body function refers to physiological functions of body systems, activity is defined as the execution of an action, and participation places activities within the context of life situations.9 Jette reports that much work remains in developing outcome instruments that can be used to measure the various domains outlined in the ICF framework.9 Because early intervention has a long tradition of focusing on functional skills and increased child participation, it seems appropriate for these programs to base at least 1 dimension of an outcome measurement system on the ICF components of activity and participation. Only a few studies have addressed the ability of outcome measures based on multiple components of the ICF to detect change in children with disabilities. Wang et al10 investigated reliability, sensitivity to change, and responsiveness of an activity-based measure, the Peabody Developmental Motor Scales (PDMS-II) in 32 children with cerebral palsy (CP) (ages 27-64 months) over a 3-month time period. The results showed effect size (ES) indices of 0.2 (small) and responsiveness indices for subjects classified as having a clinically important change that ranged from 1.7 to 2.3 (small). The authors deemed these values as acceptable responsiveness and sensitivity and attributed the small ES to the small sample size and the heterogeneity of the children with CP in the study. Ostensjo and colleagues11 conducted a cross-sectional study using the Pediatric Evaluation of Disability Inventory (PEDI) and the Gross Motor Function Measure (GMFM) to investigate the relationship within and between the 2 ICF classifications of activity and participation. The study included 95 children diagnosed with CP at an average age of 58 months (SD = 18).11 Ostensjo et al11 reported that successful participation in daily activities was best predicted by the child's ability to perform gross motor tasks. The results of the study support the use of multidimensional assessment as an integral part of clinical practice.11 Dolva et al12 used the PEDI in a cross-sectional study with 43 children diagnosed with Down syndrome (DS) at an average age of 5 years. Using PEDI scores in functional areas, Dolva et al12 concluded that children with DS do not follow the same developmental profile in home and community function as their same age peers. The authors emphasize the need for more detailed information about functional skills that are important in daily life of children with DS. Kolobe et al5 compared the responsiveness of 2 activity-based outcome measures, the GMFM and the Peabody Developmental Gross Motor Scale (PDMS-GM). The authors hypothesized that the mean change scores for 24 infants with CP and 18 infants with motor delays would be greater for the GMFM than for the PDMS-GM over a 6-month period beginning at an average age of 13.9 months.5 Results of the study did not support the hypothesis but did reveal that infants with CP developed at a slower rate and made less change over time than their same age peers with motor delays. Shevell and colleagues13 designed a prospective study with a previously defined and assembled cohort of 67 children with global developmental delay, and 72 children with developmental language impairment. The children's initial developmental scores at 3.6 years of age on the Batelle Developmental Inventory (BDI) and their functional/activity scores on the Vineland Adaptive Behavior Scale (VABS) were compared with their scores at school entry (7-8 years).13 Interestingly, the children in both groups made more improvement in their VABS scores compared with their BDI scores, indicating that functional-based outcomes may be better indicators of progress for children with disabilities.

A review of the literature reveals a lack of activity- and participation-based outcome data for young children diagnosed with CP, DS, autism spectrum disorder, global developmental delay, or developmental language impairment, 5 of the most commonly encountered neuro-developmental disabilities of early childhood.1315 The purpose of this study was to compare 2 instruments based on different components of the ICF as measures of change in children who had received Part C early intervention services. Specifically, we investigated the differences in the capacity of the PEDI Functional Skills scales and Mullen Scales of Early Learning (MSEL) to measure change in children with and without motor delays over an average period of 35 months. The PEDI reflects the activity and participation components of the ICF and allows for the use of environmental modifications, whereas the MSEL reflects the body function and activity components of the ICF and does not allow for environmental modifications.16 Because of these differences in the 2 tests, we hypothesized that mean change scores and ES indices would be larger for the PEDI Functional Skills scales than for the MSEL for the group of children with motor delay.

Back to Top | Article Outline

METHODS

Subjects

Forty-one children, from a preestablished cohort of 70 infants who received early intervention services, were successfully contacted and agreed to participate in the study. One child was excluded because the family was non-English speaking and data from 6 other children were excluded because of incomplete data from their previous testing. The remaining 34 children were grouped according to the presence of motor delay. Group 1 included 17 children who had a MSEL Gross Motor score higher than 2 SD below the mean, and Group 2 included 17 children who had a MSEL Gross Motor score lower than 2 SD below the mean, at an average testing age of 20 months. To be eligible to participate in the study, children had to meet the following criteria:

1. Received early intervention services from a Part C service delivery program (ENRICH) in the Denver metropolitan area.

2. Less than 27 months of age at the time of initial enrollment into Part C intervention services.

3. Completed initial functional and developmental testing with the PEDI and MSEL over 2 data points at least 10 months apart during their Part C intervention.

4. Informed consent of a parent or guardian.

Exclusion from the proposed study was limited to:

1. Families who were unable or unwilling to participate in the planned evaluations.

2. Families who were not living with their children with no reliable informant available.

3. Families who were non-English speaking as the assessments were only available in English versions.

4. Families who had a child in the program who had undergone surgery within the last 3 months.

The characteristics of the 34 children are presented in Table 1. There were no statistically significant differences between the groups in gender, ethnicity, gestational age, or age at the time of testing. At the time of the third trial, Group 2 had a higher percentage of children in the school setting with disabilities (88%) who were receiving Part B services (special education, physical therapy, occupational therapy, and speech therapy) than Group 1 (41%).

Table 1
Table 1
Image Tools
Back to Top | Article Outline
Procedures

Project ENRICH, a program of John F. Kennedy Partners, assembled the cohort of children. As part of an ongoing effort to document the effects of early intervention, 3 researchers collected data on the children's development while the families were enrolled in Part C ENRICH services. The total number of assessments depended on the age at which the children were enrolled in the study, with assessments completed approximately every 10 to 12 months. The current study extended the data set with an additional assessment when the children were between the ages of 3 and 5.75 years. The combined phases of this prospective study resulted in the administration of the PEDI and MSEL 3 times on each child. First, at an average age of 18 months (SD = 6.7, range = 5-27), again approximately 13 months later at an average age of 31 months (SD = 6.5, range = 18-42), and at approximately 21 months later, at an average age of 53 months (SD = 11, range = 30-68). Both tests were administered during the same visit. The PEDI was administered first as it provided an opportunity for the child to become comfortable with the test administrator while the parent interview was conducted. The principal investigator was one of the test administrators and was a pediatric physical therapist with 22 years of experience. The other test administrator was a physical therapist with 1 year of pediatric clinical experience who served as a research assistant. Both test administrators were blinded to previous PEDI and MSEL test scores. The study was approved by the Colorado Multiple Institutional Review Board and the Institutional Review Board of Rocky Mountain University of Health Professions.

Back to Top | Article Outline
Measures

The PEDI and MSEL are widely used in clinical and research settings with children with a variety of developmental disabilities.1,1619 The PEDI17 was chosen to gather information on the ICF components of activity and participation in the areas of mobility, self-care, and social function. The PEDI has been normalized for children ages 6 months to 7.5 years.17 Multiple research efforts have demonstrated the internal consistency, inter-interviewer and test-retest reliability, and discriminative validity of the PEDI.1,1619 The PEDI consists of 3 Functional Skills scales: Self-Care (73 items), Mobility (59 items) and, Social Function (65 items).17 Items are scored as 0 (child is unable) or 1 (child is capable).17 Raw scores from the PEDI Functional Skills scales can be transformed into normative (t distribution is used for standard scores) or scaled scores.17 The standard scores provide a measure of the child's performance compared with the performance of same age peers, and the scaled scores provide an indication of the child's performance along a continuum of difficulty (scale 0-100) regardless of age.17

The MSEL18 is a measure of the ICF components of body function and activity in the areas of motor, perceptual, and language abilities. The assessment includes 5 scales: Gross Motor (35 items), Visual Reception (33 items), Fine Motor (30 items), Receptive Language (33 items), and Expressive Language (28 items).18 The test was normalized on 1849 children ranging in age from 2 days to 69 months.18 Scoring varies by item from 2-point scale (0 = does not meet criteria to 1 = meets criteria) to a 6-point scale.18 Results for each scale are described by t scores (M = 50, SD = 10), percentile ranks, and age equivalents.18 Internal consistency for each of the MSEL subscale ranges from a Cronbach's alpha of 0.95 to −0.97.18 The MSEL scale was compared with the Bayley scale,18 for concurrent validity with reported Mental Developmental Index (MDI) correlations of 0.53 to −0.60.18 In an additional study, the Bayley (MDI) and composite MSEL Mental Score Scale were correlated at 0.97.18 The Preschool Language Scale was correlated with the MSEL language scales with values of 0.9 to −0.92 at 18 months and 24 months, respectively.20 The concurrent validity studies of the standard scores on the Gross Motor and Fine Motor scales showed high correlations with the PDMS-2nd ed. (r = 0.80-0.91).21 The MSEL has been used extensively as a discriminative and evaluative measure in children with autism spectrum disorder, fragile X syndrome, and speech delays.22

Back to Top | Article Outline
Interrater Reliability

A pilot reliability study took place before the first month of testing, to compare the test scores derived from a former test administrator of the original cohort (via videotape) with the test scores derived by the principal investigator and research assistant in this investigation for 4 subjects on their second testing date. The interclass correlation coefficient, ICC (2,1), was used to analyze the test-retest reliability of the standard scores on all measures. According to Portney and Watkins,23 ICC values above 0.75 can be interpreted as indicating good reliability. ICC (2,1) values for this study were greater than 0.95 for all scales of the PEDI and MSEL.

Back to Top | Article Outline
Data Analysis

Raw scores for the PEDI and MSEL were transformed to standardized scores, and the PEDI Functional Skills scores were also converted to scaled scores. The repeated-measures multivariate analysis of variance (MANOVA) was used to determine the significance of within and between subject differences in the standard scores, on all of the tests across the 3 data points for the 2 groups. To compare similar constructs, 3 separate analyses were performed. The first repeated-measures MANOVA investigated measures related to the speech and language domain. It compared the PEDI Functional Skills Social Function scale standard scores, the MSEL Expressive Language, and the MSEL Receptive Language standard scores within and between groups across 3 trials. The second repeated-measures MANOVA compared the scales related to functional use of motor skills, the PEDI Functional Skills Mobility and Self-Care scales, and the MSEL Visual Reception and Fine Motor scales, within and between groups across 3 trials. The third repeated-measures MANOVA compared the scaled scores on the PEDI Functional Skills scales between the 2 groups across 3 trials. The MANOVA is considered a robust test that can withstand the risk of Type I and Type II error.24 Statistical analyses were performed with SPSS (Statistical Package for the Social Sciences) version 15.

Back to Top | Article Outline
Sensitivity Indices

The MANOVA relies on the statistical significance of the measure and depends on the magnitude of the observed change.25 For this reason, it is affected by the sample size and the variability of the measure and is limited in its generalizability when used with small sample sizes.25 Recent researchers have used additional methods that investigate the construct of variability separately from sample size. Therefore, to evaluate sensitivity we calculated 3 additional change indices, ES, standard response mean (SRM), and minimal detectable change (MDC) for standard and scaled scores on the PEDI Functional Skills scales and for standard scores on the MSEL:

* ES is defined as the difference between the mean baseline scores and the follow-up scores divided by the standard deviation of baseline scores:25

Equation (Uncited)
Equation (Uncited)
Image Tools

* SRM provides an estimate of change in the measure, standardized relative to the between patient variability in change scores:25

Equation (Uncited)
Equation (Uncited)
Image Tools

* MDC90 is the “minimal amount of change that is not likely to be due to chance variation in measurement.”26 It is computed by multiplying the z-score of a 90% confidence interval (CI) from a normal distribution by the standard deviation at the baseline measurement and by the square root of 2(1− r) where r is the ICC:26

Equation (Uncited)
Equation (Uncited)
Image Tools

We used ICC reliability coefficient (r) calculated on larger sample sizes from previous investigations.18,26 Change above the MDC90 value is considered (at a 90% confidence level) greater than measurement error and, therefore, likely a true change.27 The reliable change proportion (RCP) is the proportion of the sample that exceeded the minimum change score. The MDC90 is likely to change with settings and population.27 Therefore, we determined MDC90 indices for each group separately.

Back to Top | Article Outline

RESULTS

Group Comparisons in Mean Change Scores on the PEDI Functional Skills Scales and MSEL Over Time

The standard scores on the PEDI Functional Skills scales and MSEL were significantly different between groups across all trials (P < .01). Yet the differences in the groups' mean change scores, as measured by the group × time × measure interaction, was only statistically significant for the standard scores on the PEDI Functional Skills Social Function Scale (P = .021), whereas the MSEL Expressive and Receptive Language scales were significantly different between trials 1 and 3. The mean changes of the standard scores for the 2 groups of children were not significantly different from each other on the PEDI Functional Skills Self-Care and Mobility Scales and the MSEL Fine Motor, and Visual Receptive scales. Nor were the mean change scaled scores for the 2 groups of children significantly different from each other on the PEDI Functional Skills Self-Care, Mobility, and Social Function scales.

Back to Top | Article Outline
Sensitivity to Change Indices of Standard Scores

Table 2 shows the means, standard deviations, post hoc P values, and sensitivity to change indices, for the mean change in standard scores on the PEDI Functional Skills Social Function Scale and the MSEL Expressive and Receptive Language scales, across the 35 months of the study (trials 1-3). During this time, Group 1, the group without motor delays, made statistically significant improvements in their mean MSEL Expressive Language scores (P = .0001, ES = 1.62, 95%CI = ±.68). Group 2, the group with motor delays, made statistically significant improvements in their mean MSEL Expressive Language scores (P = .017, ES = .94, 95%CI = ±.75) and in their mean scores on the PEDI Functional Skills Social Function scale (P = .009, ES = .88, 95%CI = ±.30). The scales in which the groups made statistically significant change over time have large ES and SRM indices (>0.8).

Table 2
Table 2
Image Tools

Means, standard deviations, post hoc P values, and sensitivity to change indices for the standard scores on the PEDI Functional Skills Self-Care and Mobility scales, and the MSEL Fine Motor and Visual Reception scales are reported in Table 3. Group 1 demonstrated a statistically significant decrease in their mean scores on the MSEL Fine Motor Scale between trials 1 and 2 (P = .048, ES = −0.46, 95%CI = ±0.45). Group 2 made statistically significant improvements in their mean standard scores on the PEDI Functional Skills Mobility Scale between trials 2 and 3 (P = .003, ES = 0.55, 95%CI = ±0.35) and between trials 1 and 3 (P = .015, ES = 0.78, 95%CI = ±0.70). The scales in which the subjects demonstrated statistically significant change had large ES and SRM indices (range 0.46-0.78).

Table 3
Table 3
Image Tools
Back to Top | Article Outline
Sensitivity to Change Indices of Scaled Scores

Means, standard deviations, P values, and sensitivity to change indices for the scaled scores on the PEDI Functional Skills Self-Care, Mobility, and Social Function scales are reported in Table 4. Both groups demonstrated statistically significant changes in their scaled scores on all PEDI Functional Skills scales across all trials (P > .001). The ES and SRM indices for the PEDI Functional Skills scaled scores were greater than 0.8 across all trials.

Table 4
Table 4
Image Tools
Back to Top | Article Outline
Confidence Intervals

Table 5 shows the 95% CIs for the ES indices on all of the scales across all trials. The CIs for the standard scores on the PEDI Functional Skills scales and MSEL were wide, often crossing the value of “0.” The exceptions to this trend were the 95% CIs for the ES indices on the MSEL Expressive Language Scale between trials 1 and 2 and trials 1 and 3 for Group 1, and the PEDI Functional Skills Social Function Scale and MSEL Expressive Language Scale for Group 2 between trials 1 and 3. In these instances, the ES indices at the low end of the 95% CI were greater than 0.5. For all of the PEDI Functional Skills scaled scores, the ES indices at the low end of the 95% CI were greater than 1.00.

Table 5
Table 5
Image Tools
Back to Top | Article Outline

Minimal Detectable Change and Reliable Change Proportion at the Group Level

As shown in Table 6, the MDC90 index during the 35 months of the study for all PEDI Functional Skills standard scores ranged from 4.13 to 5.29 for Group 1, and from 3.66 to 5.37 for Group 2. The average value of the MDC90 index during the 35 months of the study for the MSEL standard scores ranged from 9.56 to 12.42 for Group 1, and from 8.88 to 12.42 for Group 2. The average value of the MDC90 index for the PEDI Functional Skills scaled scores during the 35 months of the study ranged from 3.91 to 9.92 for Group 1, and from 5.19 to 9.38 for Group 2. Table 6 also shows the RCP, the proportion of the sample that exceeded the minimum change score (MDC90), in each group for each scale across the 3 trials. The proportion of children who made true change in their standard scores on the PEDI Functional Skills scales from trials 1 to 3, ranged from 0.23 to 0.47 in Group 1 and from 0.41 to 0.70 in Group 2. The proportion of children who made true change in their standard scores on the MSEL scales from trials 1 to 3, ranged from 0.06 to 0.71 in Group 1 and from 0.18 to 0.59 in Group 2. The proportion of children who made true change in their scaled scores on the PEDI Functional Skills Self-Care, Mobility, and Social Function scales from trials 1 to 3 ranged from 0.94 to 1.0 in Group 1 and from 0.84 to 0.92 in Group 2.

Table 6
Table 6
Image Tools
Back to Top | Article Outline

DISCUSSION

The purpose of this study was to compare the capacity of both scaled and standard scores on the PEDI Functional Skills scales and standard scores on the MSEL, to detect change in 2 groups of children who received early intervention services. Because the PEDI Functional Skills scales allow for environmental modifications and emphasize different components of the ICF than the MSEL, we hypothesized that the PEDI would be more sensitive to change for the children in Group 2 who had motor delays. However, the results of the repeated-measures MANOVA revealed a group × measure × time interaction effect for only one scale, the PEDI Functional Skills Social Function Scale (P = .021). Post hoc analysis revealed that, on average, the children with motor delays, unlike children without motor delays, made statistically significant improvements; coming closer to peers of the same age who were typically developing, in their PEDI Functional Skills Social Function Scale scores over the 35 months of the study. This was an unexpected result as we intuitively thought that the differences between measures and groups would be most apparent in the scales that had items related to motor rather than language skills. However, the amount of change detected in mean scores on every other measure across the 3 trials was not significantly different between the groups. Post hoc P values revealed that both groups made statistically significant change in their mean standard MSEL Expressive Language scores (P < .018) over the 35 months of the study, and neither group made statistically significant changes in their mean standard MSEL Receptive Language scores. Group 1 had mean standard scores within 1 SD below the PEDI, and MSEL normative sample means on all scales with the exception of their mean score on the MSEL Expressive Language Scale (M = 34.53). This is the only standard score in which Group 1 demonstrated statistically significant improvement (P < .05). The results indicate that the standard scores on the PEDI Functional Skills Social Function Scale are better able to detect change than the standard scores on the MSEL Expressive and Receptive Language scales in children with motor delays compared with children without motor delays. More research is needed to determine how change in expressive language skill in the activity-based component relates to change in the participation component of social communication in children with and without motor delays. Because the children with motor delays in this study were already, on average, 18 months of age at the beginning of the study, they may have begun to plateau in their ability to catch up to their same age peers on the subtests related to motor development (Fine Motor, Visual Reception, Self-Care, and Mobility) but were continuing to show more rapid developmental progress in their social function and language skills. This is consistent with other research findings that suggest that there is an inverse relationship between age and the rate of motor development in children with physical disabilities.28 More research is needed to investigate the relationship between age and rate of developmental change of social function and language skills in children with and without motor disabilities receiving early intervention services.

The SRM and ES indices reported in the study revealed similar trends to those observed in the post hoc P values. In other words, the measures that had P values greater than .05, demonstrating statistically significant change, had larger ES and SRM values than the measures that did not demonstrate statistically significant change. Furthermore, calculated SRM values and ES indices tended to fall within the same Cohen's classification system of trivial (< 0.2), small (0.2 < x > 0.49), medium (0.5 < x > 0.79), and large (> 0.8) for all measures. In addition, both indices had similar CIs across measures, groups, and trials. The CIs for the standard scores on the PEDI Functional Skills scales and MSEL were wide, often crossing the value of “0,” indicating trivial to small change. The exceptions to this trend were the MSEL Expressive Language scores for Group 1 and the PEDI Functional Skills Social Function scores for Group 2 between trials 1 and 3. Even though these scales had wide CIs, the ES indices were large enough that the ES value at low end of the 95% CI for the MSEL Expressive Language Scale for Group 1 was medium (ES > 0.5), and the value at the low end of the 95% CI for the PEDI Functional Skills Social Function Scale for Group 2 was large (ES > 0.94). There was a high degree of variability in the study sample as indicated by the large range and standard deviation values for each subtest. This variability caused the ES indices for standard scores to range from small to medium and made the 95% CIs wide. The ES indices for the standard scores in this study were even smaller than those found in a meta-analysis by Ottenbacher in 1992, who examined the reported ES indices for 237 statistical tests from 59 early intervention studies.29 The results revealed that a large proportion of reported ES values were in the medium and small categories.29 Ottenbacher urged researchers in early intervention to report ES indices to enhance interpretation of statistical findings.29 For research studies with small sample sizes to demonstrate accurate ES indices with narrow CIs, the sample must have small variations in initial scores. Unfortunately, early intervention research involves a heterogeneous population with large variance, and thus, requires large sample sizes for ES accuracy.

The PEDI Functional Skills scaled scores provide an indication of the child's performance along a continuum of difficulty (scale 0-100) regardless of age.17 ES indices for scaled scores were larger than ES indices for standard scores. In fact, the ES indices at the low end of the 95% CIs were large (ES > 1.00) for all scaled scores on the 3 PEDI Functional Skills scales between trials 1 and 3 for both groups. Furthermore, the ES indices were large (ES > 0.08) and the P values were greater than .05 for all scaled scores on the 3 PEDI Functional Skills scales, between trials 1 and 2 for both groups indicating that PEDI Functional Skills scaled scores were capable of measuring change during the first 12 months that the children were enrolled in Part C services. Results indicate that using scaled scores from the PEDI Functional Skills scales can provide valuable information about how a group of children are progressing even when their standard scores reveal that they are not coming closer to the development of their same age peers.

The MDC indices from this study can serve as a starting point for interpreting group and individual changes on the PEDI Functional Skills scales and MSEL for children with disabilities in early intervention. The PEDI Functional Skills scaled score MDC90 values in this study ranged from 3.91 to 9.38, which encompassed the change score reported by Haley et al26 of 5.1 (0-100 scale), which appeared to be meaningful to clinicians during a child's or adolescent's inpatient rehabilitation. If we truly want tools that detect change in function in the activity and participation component, more research with a tool such as the PEDI is needed to inform us of expected MDC90 values and the usefulness of the tool for both program evaluation and tracking individual progress.

For comparing sensitivity among tools intended for evaluative purposes, the RCP of the MDC90 value has advantages. It has an inherent interpretation, the proportion of individual subjects in the sample that exhibit a true change, without placing undue emphasis on strict decision making based on the magnitude of a p-value. Although a larger proportion of children in Group 1 made true change in their standard MSEL Expressive and Receptive Language scores compared with Group 2, a smaller proportion of those children demonstrated true change in their standard scores on the PEDI Functional Skills Social Scale. These results may suggest that the relationship between receptive and expressive language skills and social communication may be different for children with different types of disabilities. The PEDI Functional Skills scaled scores showed significant changes beyond the MDC90 for over 94% of the children in Group 1 and for over 82% of the children in Group 2 on all 3 Functional Skills scales. The results suggest that using the RCP of the MDC90 of scaled scores on an activity/participation-based measure, such as the PEDI, is an effective strategy for detecting change across multiple domains of function in diverse groups of children receiving early intervention services. In addition, previously published research results have demonstrated that the PEDI Functional Skills scaled scores are responsive to change in young children with disabilities.14,28,3032

In this study, we addressed only one aspect of sensitivity to change, which was to measure the magnitude of change relative to the distribution of scores. A second aspect of sensitivity to change concerns the meaningfulness of change and requires an anchor-based statistic as a reference measure of perceived functional or quality of life status. This allows the clinician to determine if change in one outcome measure can be viewed as a perceived change in function. One of the most important yet controversial sources of anchor-based measurement is the family's perception of the child's change. Clearly, evaluating change over time can be enhanced by using the triangulation of measurement error and external anchor-based measures.

Back to Top | Article Outline
Limitations

The findings in this study may be limited in generalizability because the sample was small, heterogeneous, not representative of the larger population of children served in early intervention and not randomly selected. The small sample size may also increase the potential for a Type II error. Post hoc power analysis revealed power as P = .7, indicating a 30% chance of a Type II error. A sample size of 25 children in each group would have been necessary to achieve adequate power of 0.80 when performing a repeated-measures MANOVA. The internal validity of this study was threatened by the use of the parametric MANOVA when the parametric assumptions of sphericity and random sampling were violated.

Back to Top | Article Outline

CONCLUSIONS

Accountability for early intervention is our professional responsibility. Policy makers, tax payers, consumers, and third-party payers demand evidence of the efficacious use of time and resources in health care. The use of valid, reliable, and sensitive outcome measures is necessary when investigating treatment effectiveness. Outcomes can also guide clinical decision making and provide prognostic information to families. The findings from this investigation suggest that using sensitivity to change statistics such as ES, SRM, and the RCP of the MDC90 for PEDI Functional Skill scaled scores may be an effective strategy for assessing functional change in children who receive early intervention services. Data from different instruments can be judged using standardized indices of response mentioned in this study. More comparative research is needed to determine the relative merits and minimal clinically important change of various outcome measures for specific populations of children. When this information is available, clinicians and programs will have the means to evaluate the effectiveness of early intervention services and researchers will have tools to construct interval estimates and perform hypothesis-based investigations.33

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors acknowledge the contributions of Janet Houser, RN, PhD, and Megan Fibbe, DPT, to this work.

Back to Top | Article Outline

REFERENCES

1. Lollar DJ, Simeonsson RJ, Nanda U. Measures of outcomes for children and youth. Arch Phys Med Rehabil. 2000; 81(12 Suppl 2):S46–52.

2. Lollar DJ, Simeonsson RJ. Diagnosis to function: classification for children and youths. J Dev Behav Pediatr. 2005; 26(4):323–330.

3. Center ECO. Considerations related to developing a system for measuring outcomes for young children with disabilities and their families. [http://www.fpg.unc.edu/∼eco/pages/publications.cfm]. April 2004. Accessed February 27, 2006.

4. Thomas-Stonell N, Johnson P, Rumney P, Wright V, Oddson B. An evaluation of the responsiveness of a comprehensive set of outcome measures for children and adolescents with traumatic brain injuries. Pediatr Rehabil. 2004; 9(1):14–23.

5. Kolobe TH, Palisano RJ, Stratford PW. Comparison of 2 outcome measures for infants with cerebral palsy and infants with motor delays. Phys Ther. 1998; 78(10):1062–1072.

6. De Bruin AF, Diederiks JP, de Witte LP, Stevens FC, Philipsen H. Assessing the responsiveness of a functional status measure: the Sickness Impact Profile versus the SIP68. J Clin Epidemiol. 1997; 50(5):529–540.

7. Ostensjo S, Carlberg EB, Vollestad NK. Everyday functioning in young children with cerebral palsy: functional skills, caregiver assistance, and modifications of the environment. Dev Med Child Neurol. 2003; 45(9):603–612.

8. Ottenbacher KJ, Msall ME, Lyon N, et al. The WeeFIM instrument: its utility in detecting change in children with developmental disabilities. Arch Phys Med Rehabil. 2000; 81(10):1317–1326.

9. Jette AM. Toward a common language for function, disability, and health. Phys Ther. 2006; 86(5):726–734.

10. Wang HH, Liao HF, Hsieh CL. Reliability, sensitivity to change, and responsiveness of the Peabody Developmental Motor Scales-second edition for children with cerebral palsy. Phys Ther. 2006; 86(10):1351–1359.

11. Ostensjo S, Carlberg EB, Vollestad NK. Motor impairments in young children with cerebral palsy: relationship to gross motor function and everyday activities. Dev Med Child Neurol. 2004; 46(9):580–589.

12. Dolva AS, Coster W, Lilja M. Functional performance in children with Down syndrome. Am J Occup Ther. 2004; 58(6):621–629.

13. Shevell M, Majnemer A, Platt RW, Webster R, Birnbaum R. Developmental and functional outcomes at school age of preschool children with global developmental delay. J Child Neurol. 2005; 20(8):648–653.

14. Ketelaar M, Vermeer A, Hart H, van Petegem-van Beek E, Helders PJ. Effects of a functional therapy program on motor abilities of children with cerebral palsy. Phys Ther. 2001; 81(9):1534–1545.

15. Shevell MI, Majnemer A, Rosenbaum P, Abrahamowicz M. Profile of referrals for early childhood developmental delay to ambulatory subspecialty clinics. J Child Neurol. 2001; 16(9):645–650.

16. Ostensjo S, Bjorbaekmo W, Carlberg EB, Vollestad NK. Assessment of everyday functioning in young children with disabilities: An ICF-based analysis of concepts and content of the Pediatric Evaluation of Disability Inventory (PEDI). Disabil Rehabil. 2006; 28(8):489–504.

17. Haley SMCW, Ludlow LH, Haltiwanger J, Andrellos P. Pediatric Evaluation of Disability Inventory (PEDI). Version 1.0 ed. ed. Boston: Boston University; 1992.

18. Mullen E. Mullen Scales of Early Learning. AGS edition ed. Circle Pines, MN: American Guidance Service, Inc.; 1995.

19. Nordmark E, Jarnlo GB, Hagglund G. Comparison of the gross motor function measure and paediatric evaluation of disability inventory in assessing motor function in children undergoing selective dorsal rhizotomy. Dev Med Child Neurol. 2000; 42(4):245–252.

20. Bradley-Johnson. Test reveiws. Psychol Sch. 1997; 34(4):379–378.

21. Folio M, Fewell R. Peabody Developmental Motor Scales: Examiner's Manual. 2nd ed. Austin, TX: Pro-Ed, Inc; 2000.

22. Akshoomoff N. Use of the mullen scales of early learning for the assessment of young children with Autism Spectrum Disorders. Child Neuropsychol. 2006; 12(4-5):269–277.

23. Portney L, Watkins M. Foundations of Clinical Research Applications to Practice. 3rd ed. Upper Saddle River: Pearson Education, Inc.; 2009.

24. Warner RM. Applied Statistics: From Bivariate Through Multivariate Techniques. Los Angeles: Sage Publications, Inc.; 2008.

25. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000; 53(5):459–468.

26. Haley SM, Fragala-Pinkham MA. Interpreting change scores of tests and measures used in physical therapy. Phys Ther. 2006; 86(5):735–743.

27. Schmitt JS, Di Fabio RP. Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. 2004; 57(10):1008–1018.

28. Vos-Vromans DC, Ketelaar M, Gorter JW. Responsiveness of evaluative measures for children with cerebral palsy: the gross motor function measure and the pediatric evaluation of disability inventory. Disabil Rehabil. 2005; 27(20):1245–1252.

29. Ottenbacher KJ. Research methods practical significance in early intervention research: from affect to empirical effect. J Early Intervent. 1992; 16(2):181–193.

30. Oeffinger D, Gorton G, Bagley A, et al. Outcome assessments in children with cerebral palsy, part I: descriptive characteristics of GMFCS Levels I to III. Dev Med Child Neurol. 2007; 49(3):172–180.

31. Sakzewski L, Boyd R, Ziviani J. Clinimetric properties of participation measures for 5- to 13-year-old children with cerebral palsy: a systematic review. Dev Med Child Neurol. 2007; 49(3):232–240.
32. Wright FV, Rosenbaum PL, Goldsmith CH, Law M, Fehlings DL. How do changes in body functions and structures, activity, and participation relate in children with cerebral palsy? Dev Med Child Neurol. 2008; 50(4):283–289.

33. Liang MH, Fossel AH, Larson MG. Comparisons of 5 health status instruments for orthopedic evaluation. Med Care. 1990; 28(7):632–642.

activities of daily living; child; developmental disabilities; early intervention; infant; motor skills; psychometrics; sensitivity; speech-language pathology

© 2010 Lippincott Williams & Wilkins, Inc.

Login

Article Tools

Images

Share

Follow PED-PT on Twitter

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.