Skip Navigation LinksHome > Summer 2012 - Volume 24 - Issue 2 > Concurrent Validity and Reliability of the Pediatric Evaluat...
Pediatric Physical Therapy:
doi: 10.1097/PEP.0b013e31824c94ca
Research Article

Concurrent Validity and Reliability of the Pediatric Evaluation of Disability Inventory-Computer Adaptive Test Mobility Domain

Dumas, Helene M. PT, MS; Fragala-Pinkham, Maria A. PT, DPT, MS

Free Access
Article Outline
Collapse Box

Author Information

Research Center for Children with Special Health Care Needs, Franciscan Hospital for Children, Boston, Massachusetts.

Correspondence: Helene M. Dumas, PT, MS, Research Center, Franciscan Hospital for Children, 30 Warren St, Boston, MA 02135 (hdumas@fhfc.org).

Grant Support: This study was supported by a Clinical Research grant from the Section on Pediatrics, American Physical Therapy Association.

The authors declare no conflict of interest.

Collapse Box

Abstract

Purpose: To examine concurrent validity, item-specific reliability, and score distributions of the new Pediatric Evaluation of Disability Inventory-Computer Adaptive Test (PEDI-CAT) Mobility domain with the original PEDI Functional Skills (FS) Mobility Scale.

Methods: Thirty-five parents of children with neurodevelopmental disabilities completed the PEDI-CAT on a computer and the paper PEDI FS via interview.

Results: Strength of association between the PEDI-CAT Mobility domain and PEDI FS Mobility Scale scores was good to excellent (r = 0.82; P < .001). Intraclass correlation coefficients ranged from .3390 to 1.000, and agreement ranged from 60% to 100% for 8 specific items. No child had the minimum score on either test, whereas 9 children (26%) had a maximum score on the PEDI FS Mobility Scale.

Conclusions: This study provides evidence for potential users that the concurrent validity, reliability, and score distribution for the PEDI-CAT Mobility domain are adequate for use with children with varied diagnoses and throughout the pediatric age span.

Back to Top | Article Outline

INTRODUCTION

The Pediatric Evaluation of Disability Inventory-Computer Adaptive Test (PEDI-CAT)1 is a new clinical assessment for children and youth from birth to 20 years of age that was developed on the basis of years of experience, feedback, and formal research with the original PEDI.26 Completed by using preinstalled software on any computer, the PEDI-CAT was designed to be used across all diagnoses, conditions, and settings. The PEDI-CAT item bank contains 276 activities and measures function in 4 domains: (1) Daily Activities, (2) Mobility, (3) Social/Cognitive, and (4) Responsibility. As with the original PEDI, the PEDI-CAT can be completed by parent/caregiver report or professional judgment of clinicians who are familiar with the child. The PEDI-CAT is intended to enable clinicians to construct a description of a child's current functional status or progress in acquiring functional skills that are part of everyday life.1

The PEDI-CAT software uses statistical models to estimate a child's abilities from a minimal number of the most relevant items or from a predetermined number of items within each domain. All respondents begin with the same item in each domain in the middle of the range of difficulty or responsibility and the response to that item then dictates which item will appear next (a harder or easier item), thus customizing the items to the child and minimizing the number of irrelevant items. With administration of each subsequent item, the score is reestimated along with the confidence interval and the computer algorithm determines whether the stopping rule (an acceptable level of precision or a set number of items) has been satisfied. If satisfied, the assessment ends. If not satisfied, new items are administered until the stopping rule is satisfied. The PEDI-CAT program then displays the results, including an item map, a scaled score, and an age percentile, instantly.1

The original PEDI,2 published in 1992, is a functional assessment used by physical therapists and other rehabilitation professionals to measure self-care, mobility, and social function and can be administered using a paper-and-pencil version or using the PEDI software program.2 Strengthening its appeal as a clinical tool is the long history of published research examining the psychometric properties including reliability2,79 and validity.2,811 Although the PEDI is one of the most widely used measures of children's function worldwide,7,1117 its limitations include a small age range for normative scoring (6 months to 7.5 years), limited content, and the length of administration time.1820

The PEDI-CAT, just as the original PEDI, must offer clear clinical value to clinicians and researchers if it is to be adopted for use, and thus, evaluation of its psychometric properties is imperative. To date, content and construct validity of the PEDI-CAT have been reported.4 Discriminant validity has been evaluated using simulated versions, based on the initial normative and disability samples,5 in a prospective study of 102 children with and without disabilities6 and in a study examining the scores of children who use a walking aid or wheelchair.21 Test-retest reliability was examined during a small prospective study in which parent respondents completed the PEDI-CAT twice within 1 month.6 Additional validity and reliability studies are warranted to determine that the PEDI-CAT Mobility domain is consistent and measuring what is intended to be measured with varied samples and populations and at different times.

The purpose of this study was to (1) examine concurrent validity of the new PEDI-CAT Mobility domain with the original PEDI Functional Skills (FS) Mobility Scale, (2) evaluate item-specific reliability between the PEDI-CAT Mobility domain and PEDI FS Mobility Scale and, and (3) assess score distributions for floor and ceiling effects.

Back to Top | Article Outline

METHODS

Participants

Parents of children with neurodevelopmental disabilities (n = 35) and with ages from birth to 20 years and who were English speaking were recruited to participate in this study. Participants were generally mothers (89%), and children ranged in age from 3.93 to 19.87 years. Children had the following diagnoses: autism (n = 4); cerebral palsy, Gross Motor Function Classification Scale levels I to V (n = 20); genetic disorders (eg, Down syndrome, Prader-Willi syndrome) (n = 5); and other neurologic disorders (eg, attention deficit disorder, developmental delay) (n = 6). Twelve of the children used a walking aid (crutches, cane, or walker), and 14 children used a wheelchair some or all of the time. Demographic characteristics of the study sample are depicted in Table 1.

Table 1
Table 1
Image Tools
Back to Top | Article Outline
Instrumentation

Each of the PEDI-CAT domains (Daily Activities, Mobility, Social/Cognitive, and Responsibility) is self-contained and can be used separately or in combination with the other domains. Using a 4-point response scale, the PEDI-CAT measures level of difficulty (easy, a little hard, hard, and unable) in the 3 functional domains of Daily Activities, Mobility, and Social/Cognitive. The PEDI-CAT Responsibility domain measures the extent to which the caregiver or child takes responsibility for managing complex, multistep life tasks. The PEDI-CAT Mobility domain addresses 5 content areas: Basic Movement and Transfers, Standing and Walking, Steps and Inclines, Running and Playing, and Wheelchair. Seventy-five items address early mobility and physical functioning activities such as head control, transfers, walking, climbing stairs, and playground skills, whereas an additional 22 items are specifically for children who use mobility devices such as walking aids (canes, crutches, walkers) and/or wheelchairs. Photographs accompany each item to further depict the activity being assessed.1

There are currently 2 versions of the PEDI-CAT: the “Speedy CAT,” the quickest way to get a precise score estimate while administering 15 or fewer items per domain, and the “Content-Balanced CAT,” in which approximately 30 items per domain are administered and include a balance of items from each of the content areas within each domain. Each version generates a summary (scaled) score on a 20- to 80-point metric and a normative score in percentiles.1

The original PEDI has 2 scales, Functional Skills and Caregiver Assistance, to assess capability and level of independence within the functional domains of Self-Care, Mobility, and Social Function. The PEDI FS Mobility Scale includes 59 items and is used to measure a child's capability with basic functional mobility skills in the following 13 item groups: toilet transfers, chair/wheelchair transfers, car transfers, bed mobility/transfers, tub transfers, method of indoor locomotion, distance/speed indoors, pulls/carries objects, method of outdoor locomotion, distance/speed outdoors, outdoor surfaces, upstairs, and downstairs. Items are scored as capable (1) or unable (0). A raw score (sum of all item scores) is converted to a normative score (based on chronological age using a t-distribution) or a scaled score (not age-dependent) score along a 0- to 100-point scale.2

Neither the PEDI nor the PEDI-CAT requires any special environment, materials, or activities to administer. The focus on typical performance at the present time is assessed and, as such, the child's parent(s) or professionals who currently provide services for the child are the most likely respondents. The PEDI and the PEDI-CAT can be completed on multiple occasions for the same child (eg, admission, interim assessment, discharge, and follow-up), and there is no minimum time that must pass between assessments (manuals).

Back to Top | Article Outline
Procedure

For this study, participating therapists were trained in administering the PEDI-CAT Content-Balanced Mobility domain and the PEDI FS Mobility Scale. Following Institutional Review Board approval, parents of children with neurodevelopmental disabilities and ages from birth to 20 years were recruited through a private school for children with special health care needs and outpatient clinics in a pediatric rehabilitation hospital in the northeast United States. Parents provided written informed consent and completed a demographic form to provide information about themselves and their child. Parent participants then completed the PEDI-CAT Content-Balanced Mobility domain (version 2.21) on a laptop computer as well as the original paper PEDI FS Mobility Scale via caregiver interview. Both tests were completed on the same day. Presentation was varied to avoid a test-order effect such as fatigue, practice, or motivation. Parent participants received a small honorarium in the form of a gift card to thank them for their participation. Raw scores were converted to normative and scaled scores for the PEDI FS Mobility Scale using the PEDI manual2 and combined in to a project-specific database with the normative and scaled scores generated by the PEDI-CAT software.

Back to Top | Article Outline
Data Analysis

Data analysis and interpretation of results was performed using the Statistical Package for the Social Sciences. Concurrent validity was assessed by examining the strength of association between the 2 tests using Pearson product moment correlations for the scaled scores. To examine item-specific reliability between the PEDI-CAT Mobility domain and PEDI FS Mobility Scale, intraclass correlation coefficients and percentage agreement (how often respondents scored an item the same on both tests) were calculated. Each participant responded to all 59 items on the PEDI FS Mobility Scale and 30 items on the PEDI-CAT Content-Balanced Mobility domain. For this analysis, we identified 17 items on the PEDI-CAT that were identical in wording to the original PEDI or asked the participant about the same functional mobility activity as an item on the original PEDI. It should be noted, however, that because of the nature of the CAT, not all 35 participants responded to all 17 of these items. Only 8 of these 17 items had 20 or more responses, thus limiting the comparison. In addition, to conduct this analysis, responses on the 4-point PEDI-CAT response scale (0, unable; 1, hard; 2, a little hard; 3, easy) were recoded (0, unable/hard; 1, a little hard/easy) as the PEDI has a 2-point response scale (0, unable; 1, capable). To analyze the score distribution for each test, percentages of scaled scores less than and greater than 50 (midpoint for each scale) were calculated. In addition, scaled scores for each individual case were plotted and compared visually. Finally, the number and percentage of children with minimum (floor) and maximum scores (ceiling) along each scale were computed.

Back to Top | Article Outline

RESULTS

The strength of association between the PEDI-CAT Mobility domain scaled scores and the PEDI FS Mobility scaled scores and was good to excellent (r = .82; P < .001), indicating a strong correlation between the scores on both tests. Intraclass correlation coefficients ranged from .3390 to 1.000 and agreement results ranged from 60% to 100% for the 8 items that had at least 20 responses on the PEDI-CAT (Table 2).

Table 2
Table 2
Image Tools

Scores on the PEDI FS Mobility Scale ranged from 18.20 to 100 (maximum score). On the PEDI-CAT Mobility domain, scaled scores ranged from 33.07 to 76.75 (maximum score). Eight children (23%) had a scaled score less than 50 on the PEDI FS Mobility Scale, whereas 27 children (77%) had a score greater than 50 points. Three children (9%) had a score less than 50 on the PEDI-CAT Mobility domain, whereas 32 children (91%) had a score greater than 50. No child had the minimum score on either test, whereas 1 child (3%) had a maximum scaled score on the PEDI-CAT and 9 children (26%) had a maximum scaled score on the PEDI FS Mobility Scale (Figure).

Fig. Score distribut...
Fig. Score distribut...
Image Tools
Back to Top | Article Outline

DISCUSSION

There is a need for valid, reliable, and responsive functional mobility measures appropriate for a wide age range and varied diagnoses for use in clinical and research settings. In addition, new tests must be efficient and capable of integrating with electronic medical records. The PEDI-CAT was developed to respond to these needs, and it is thus imperative to determine whether the PEDI-CAT is worthy of use. The purpose of this study was to examine the concurrent validity, item-specific reliability, and score distribution of the PEDI-CAT Mobility domain when compared with the original PEDI FS Mobility Scale.

The strong correlation between scaled scores indicates strong agreement between the original PEDI and the new PEDI-CAT. Children who had a higher scaled score on the PEDI-CAT also had a higher scaled score on the PEDI FS Mobility Scale, and vice versa for lower scores. The PEDI FS Mobility Scale has previously been shown to have concurrent validity with the Battelle Developmental Inventory Screening Test,22 the Functional Independence Measure for Children,10 and the Gross Motor Function Measure.17 Concurrent validity is particularly important for a new, untested assessment such as the PEDI-CAT that is being proposed as an alternative to the current standard.

Both measures identified children with limitations in functional mobility. This is encouraging, as physical therapists will get similar results regardless whether they use the PEDI FS Mobility Scale or the new PEDI-CAT for younger children with physical disabilities. However, the PEDI-CAT identified a larger percentage of older children with limitations in functional mobility and may be more useful to pediatric therapists who provide services to a wide age range of children and youth. Normative standard scores are also available but only for children younger than 7.5 years on the original PEDI. The PEDI-CAT broadens this through the age of 20 years. In this sample, only 9 of the 35 children were 7.5 years or younger, making comparison difficult and thus limited this analysis to scaled scores.

Seventeen PEDI-CAT Mobility items that were presented to respondents were similar to original PEDI FS Mobility items.4 As the PEDI-CAT software selects which items to administer, not all 35 participants responded to all 17 identified items. We chose the 8 items for which we had at least 20 responses on the PEDI-CAT to assess item-specific reliability. Six of the 8 items had good to excellent reliability. The 2 items that did not have strong agreement were both related to stair climbing. It is possible that the wording was dissimilar enough for participants to respond differently, as the original PEDI can include the use of handrails and the PEDI-CAT specifies “without holding on to handrail.”

The PEDI FS Mobility Scale has a score range of 0 to 100 points, whereas the PEDI-CAT score metric is 20 to 80, allowing for the addition of both lower- and higher-level items over time. The scores on both the PEDI FS Mobility Scale and the PEDI-CAT Mobility domain were distributed along the score metrics for this sample of children and youth. The majority of children, however, scored above the midpoint on both scales. A large majority scored between 55 and 65 points on the PEDI-CAT, whereas there was a greater point spread on the PEDI FS Mobility Scale, as illustrated in the Figure. Of note is that the standard errors for the PEDI-CAT scores in this 55- to 65-point score range are approximately 0.5, indicating a high level of precision and a very small degree of sampling error, whereas the standard errors for the PEDI FS Mobility Scale are approximately 2.0 to 3.6 in this score range. Evaluation of responsiveness of the PEDI-CAT Mobility domain is needed to determine whether the PEDI-CAT scale is precise enough to show clinically relevant change over time.

Although no children had the lowest score on either test, 9 children had maximum scores (100) on the PEDI FS Mobility but not on the PEDI-CAT Mobility domain. This is encouraging because one of the criticisms of the original PEDI was the limited content of more challenging mobility activities.3 This criticism appears to be addressed with the PEDI-CAT. For example, on the original PEDI, the most difficult mobility item is “Walks down full flight with no difficulty.” On the PEDI-CAT, more challenging mobility items include “Walks 3 miles/5 kilometers,” “Goes up and down an escalator,” and “Climbs step ladder to put a heavy box on a high shelf.” As we found no indication of a floor effect, content coverage appears to have been adequate on the PEDI-CAT Mobility domain versus the PEDI FS Mobility Scale for this particular sample.

A threat to the validity of our study design was that the respondents completed 2 tests in 1 day and this may not generalize to a population in which only 1 test is given.23 In addition, respondents may have encountered the same item(s) twice and thus this increases the potential for a practice or memory effect. Although our sample was diverse in age and functional ability (which is indicative of the intended population for use of the PEDI-CAT), it was small. Further studies with larger samples are warranted to establish the PEDI-CAT Mobility domain as a reliable, valid, and responsive assessment for physical therapists. Also, as mathematical equations are developed to link original PEDI scores to PEDI-CAT scores so that a child's progress over time may be tracked,1 comparability studies should be conducted.

Back to Top | Article Outline

CONCLUSIONS

This study provides preliminary evidence for potential users that the concurrent validity, reliability, and score distribution for the PEDI-CAT Mobility domain are adequate for use with children with varied diagnoses and throughout the pediatric age span.

Back to Top | Article Outline
ACKNOWLEDGMENTS

This article is dedicated to Stephen M. Haley, PT, PhD, FAPTA, primary author of the Pediatric Evaluation of Disability Inventory (PEDI) and the Pediatric Evaluation of Disability Inventory-Computerized Adaptive Test (PEDI-CAT).

Back to Top | Article Outline

REFERENCES

1. Haley S, Coster W. PEDI-CAT: Development, Standardization and Administration Manual. Boston, MA: CRECare LLC; 2010.

2. Haley SM, Coster WJ, Ludlow LH, Haltiwanger JT, Andrellos PA. Pediatric Evaluation of Disability Inventory: Development, Standardization and Administration Manual. Boston, MA: Trustees of Boston University; 1992.

3. Haley SM, Coster WJ, Kao YC, et al. Lessons from use of the Pediatric Evaluation of Disability Inventory (PEDI): where do we go from here? Pediatr Phys Ther. 2010;22:69–75.

4. Dumas HM, Fragala-Pinkham MA, Haley SM, et al. Item bank development for a revised Pediatric Evaluation of Disability Inventory (PEDI). Phys Occup Ther Pediatr. 2010;30:168–184.

5. Haley SM, Coster WJ, Dumas HM, et al. Accuracy and precision of the Pediatric Evaluation of Disability Inventory Computer Adaptive Test (PEDI-CAT) for children 0 to 21 years. Dev Med Child Neurol. 2011;53(12):1100–1106.

6. Dumas HM, Fragala-Pinkham MA, Haley SM, et al. Computer adaptive test performance in children with and without disabilities: prospective field study of the PEDI-CAT [published ahead of print October 12, 2011]. Disabil Rehabil. 2012;34(5):393–401.

7. Berg M, Jahnsen R, Froslie K, Hussain A. Reliability of the Pediatric Evaluation of Disability Inventory (PEDI). Phys Occup Ther Pediatr. 2004;24:61–77.

8. Nichols D, Case-Smith J. Reliability and validity of the Pediatric Evaluation of Disability Inventory. Pediatr Phys Ther. 1996;8:15–24.

9. Erkin G, Elhan A, Aybay C, Sirzai H, Ozel S. Validity and reliability of the Turkish translation of the Pediatric Evaluation of Disability Inventory (PEDI). Disabil Rehabil. 2007;29;1271–1279.

10. Ziviani J, Ottenbacher K, Shephard K, Foreman S, Astbury W, Ireland P. Concurrent validity of the Functional Independence Measure for children (WeeFIM) and the Pediatric Evaluation of Disabilities Inventory in children with developmental disabilities and acquired brain injuries. Phys Occup Ther Pediatr. 2001;21:91–101.

11. Berg M, Aamodt G, Stanghelle J, Krumlinde-Sundholm L, Hussain A. Cross-cultural validation of the Pediatric Evaluation of Disability Inventory (PEDI) norms in a randomized Norwegian population. Scand J Occup Ther. 2008;15(3):143–152.

12. Chen KL, Tseng MH, Hu FC, Koh CL. Pediatric Evaluation of Disability Inventory: a cross-cultural comparison of daily function between Taiwanese and American children. Res Dev Disabil. 2010;31(6):1590–1600.
13. Galvin J, Lim BC, Steer K, Edwards J, Lee KJ. Predictors of functional ability of Australian children with acquired brain injury following inpatient rehabilitation. Brain Inj. 2010;24(7–8):1008–1016.

14. Stahlhut M, Christensen J, Aadahl M. Applicability and intrarespondent reliability of the Pediatric Evaluation of Disability Inventory in a random Danish sample. Pediatr Phys Ther. 2010;22(2):161–169.

15. Han T, Gray N, Vasquez MM, Zou LP, Shen K, Duncan B. Comparison of the GMFM-66 and the PEDI functional Skills Mobility domain in a group of Chinese children with cerebral palsy. Child Care Health Dev. 2011;37(3):398–403.

16. Stahlhut M, Gard G, Aadahl M, Christensen J. Discriminative validity of the Danish version of the Pediatric Evaluation of Disability Inventory (PEDI). Phys Occup Ther Pediatr. 2011;31(1):78–89.

17. Wren TA, Sheng M, Bowen RE, et al. Concurrent and discriminant validity of Spanish language instruments for measuring functional health status. J Pediatr Orthop. 2008;28(2):199–212.

18. Kothari DH, Haley SM, Gill-Body KM, Dumas HM. Measuring functional change in children with acquired brain injury (ABI): comparison of generic and ABI-specific scales using the Pediatric Evaluation of Disability Inventory (PEDI). Phys Ther. 2003;83:776–785.

19. McCarthy ML, Silberstein CE, Atkins EA, Harryman SE, Sponseller PD, Hadley-Miller NA. Comparing reliability and validity of pediatric instruments for measuring health and well-being of children with spastic cerebral palsy. Dev Med Child Neurol. 2002;44:468–476.

20. Østensjo S, Bjorbaekmo W, Carlberg EB, Vollestad NK. Assessment of everyday functioning in young children with disabilities: an ICF-based analysis of concepts and content of the Pediatric Evaluation of Disability Inventory (PEDI). Disabil Rehabil. 2006;30:489–504.

21. Dumas HM, Fragala-Pinkham MA, Tian F, Haley SM. A preliminary evaluation of the PEDI-CAT mobility item bank for children using walking aids and wheelchairs. J Pedi Rehabil Med. In press.

22. Feldman AB, Haley SM, Coryell J. Concurrent and construct validity of the Pediatric Evaluation of Disability Inventory. Phys Ther. 1990;70(10):602–610.

23. Wan L, Keng L, McClarty K, Davis L. Methods of comparability studies for computerized and paper-based tests. Test Measurement Res Serv. 2009; (10):1–4.

activities of daily living/psychology; adolescent; child; preschool child; computer assisted diagnosis/methods; disability evaluation; disabled children; female; health surveys; humans; male; physical therapy/methods; reliability and validity; young adult

© 2012 Lippincott Williams & Wilkins, Inc.

Login

Article Tools

Images

Share

Follow PED-PT on Twitter

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.