Wiepert, Susan L. MS, PT; Mercer, Vicki Stemmons PhD, PT
The focus of this study was on examining the effects of permitting an increased number of practice trials on scores obtained on the Gross Motor Scale of the Peabody Developmental Motor Scales (PDMS). 1 The PDMS generally allows minimal opportunity for children to practice task items before performance is formally scored. The question of interest was whether allowing preschoolers with typical development to perform an increased number of trials per task would improve their scores on the PDMS Gross Motor Scale as compared with scores obtained during testing with the standard number of allowable trials. A change in scores with an increased number of practice trials would suggest that the current method of administration of the PDMS Gross Motor Scale results in underestimation of gross motor abilities in preschool children.
Research related to an appropriate amount of practice needed to demonstrate ability within specific domains of gross motor skill is limited. Most researchers seem to have focused on the variability of practice hypothesis derived from Schmidt’s schema theory, 2 not on changes related to the amount of allowable practice. The universal assumption linking theoretical views of motor skill acquisition is that learning through practice is a consequence of the acquisition of more appropriate representations of action. 3 How these representations of action occur through schema formation, neural synapse recruitment, or improved problem solving/predictive capabilities is not yet fully understood.
The literature is sparse with regard to how many practice trials, or repetitions, are appropriate to exhibit representative skill level and ability in any movement task. Lee et al. 4 questioned what is actually involved in a repetition and emphasized that cognitive processes are a significant part of repetition. They argued that a repetition could be considered an attempt to solve a problem with a goal in mind, given that the same problem has been previously encountered. Higgins, 5 interested in motor skill acquisition, stressed that as physical therapists we should be interested in development of motor competence in our patients. This competence includes problem-solving ability. Higgins 5 viewed skillful movements as task specific and conforming to individual as well as environmental constraints.
Higgins 5 dichotomized skill into high and lesser degrees of consistency in goal attainment. High level skill is defined as a high degree of consistency in goal achievement with strong generalizability of the optimal solution. Lesser degrees of skill include varying degrees of consistency and generalizability of goal achievement. She stated that most people demonstrate moderate-level skill rather than high-level skill. Athletes may be considered highly skilled performers within their specific sport domain. A baseball player may be highly skilled at baseball as a result of consistent exposure to the task but may be moderately skilled at basketball. Performance can be limited by current knowledge of the task as well as other individual factors. Higgins 5 stated that through practice the learner is able to refine solutions to specific problems.
The concept of practice is of primary importance when considering the process of gross motor skill assessment for children. Most pediatric assessment tools currently in use are performance based. This means that the child’s ability in a variety of gross motor skill domains is represented by the child’s performance during a specific test administration. Although frameworks for pediatric assessment are evolving from an emphasis on the attainment of normal movement patterns to the attainment of functional skills, 6 few tools are currently available for assessing function in children of preschool age with minimal motor delay or deficit. Most assessment tools for this population focus only on motor development.
The PDMS is a motor skill assessment designed for use with children from birth to 83 months of age. It is the only standardized, norm-referenced assessment tool that encompasses the gross and fine motor skills of children from 2.5 to 4.5 years of age. 7 The developers of the PDMS are special educators. 1 According to the test manual, the PDMS was developed to enable special educators to plan instructional programs based on encouraging the development of skills not already completely developed in a child. 1 Current use of the test by physical therapists for screening and evaluative purposes seems inconsistent with the original intent of the test developers.
The PDMS is criterion referenced as well as norm referenced, giving this test dual capabilities. Norm-referenced tests are used when the purpose of the testing is to identify children with motor handicaps or to establish a specific age-equivalent level for a child. Criterion-referenced tests are used when a physical therapist wants to plan and implement an intervention program or to determine the effectiveness of an existing intervention program. 7,8 Criterion-referenced tests allow for examination and documentation of a child’s individual performance in relation to the child’s past performance or to a domain of specific tasks.
The PDMS scoring system includes a three-point scale (score of 0 to 2), allowing credit for partial success (score of 1) and complete success (score of 2) on task items. The motor skill domains of reflexes, balance, nonlocomotor, locomotor, and receipt/propulsion skills are included in the test. Reflex status is assessed in younger children as an indicator of the integrity of the developing nervous system. Dynamic and static balance skills are assessed by using activities such as standing on one foot and walking across a balance beam. Nonlocomotor skills include muscular strength (eg, sit-ups and push-ups) and postural changes (eg, bending and moving from sitting to standing). Locomotor skills include activities such as running, jumping, hopping, and skipping. Receipt and propulsion skills include catching, throwing, and kicking a ball. The PDMS may be used to assess all five of these skill areas during a single test administration.
Scale levels in the PDMS are broken down into age-specific intervals based on typical motor development. These scale levels are used to determine a child’s basal and ceiling ages. When administering the PDMS, the examiner determines the child’s basal age as a starting point. Basal age is defined as the scale level at which the child can successfully complete all tasks to full criterion. Each subsequent level of the scale is then administered in its entirety until a ceiling age is reached. The ceiling age is that level at which the child cannot successfully complete more than one of the task items to full criterion (score of 2). After the test is administered, the child’s raw scores for each skill area are transformed into standardized scores. Scaled scores, age equivalents, and developmental motor quotients can be calculated to provide a broader view of the child’s present motor abilities. 1
The PDMS manual details verbal instructions to be given to the child before each task is attempted, the number of allowable practice trials for each task, and whether the examiner is allowed to demonstrate the task to the child. Many of the task items do not allow any practice trials because performance is scored after the child’s first attempt at the task. The number of practice trials per task varies. The authors of the PDMS do not justify the number of trials per task in the test manual.
If most individuals demonstrate moderate skill as suggested by Higgins, 5 then some degree of inconsistency during performance of a task would be expected. A child may have difficulty generalizing the task to different contexts, including a testing situation in which the child may be unfamiliar with the examiner. A child with a moderate level of skill may need additional trials in this context to demonstrate ability to successfully complete the task.
In previously published pilot work for this study, a boy aged 50 months with typical development demonstrated improvement with up to five trials in four of the five gross motor skill domains on the PDMS Gross Motor Scale. 9 The largest improvement in Z scores occurred in the domain of receipt and propulsion, in which the score was initially −1.18 and improved to 0.05 with up to five trials on each task. The subject’s age equivalence score increased three months when up to five trials were allowed. The results from this pilot work encouraged us to explore further the effect of the number of practice trials on PDMS Gross Motor Scale scores.
The purpose of this study was to determine whether increasing the number of practice trials for each of the skill tasks on the PDMS Gross Motor Scale would produce significant improvements in scores of preschoolers with typical development. Our specific research questions were as follows: 1) How does additional practice affect mean scores in the gross motor skill domain? 2) Will total scores for the group of preschoolers increase significantly when increased practice is allowed? 3) Will total scores after practice for any of the children tested fall outside the 95% confidence interval of total scores obtained after standard administration?
This study was approved by the Committee on the Protection of the Rights of Human Subjects, University of North Carolina at Chapel Hill. Nineteen children, 12 boys and seven girls, between the ages of 48 and 68 months (mean age = 58.63 ± 7.21 months) participated in the study. Subjects were recruited through letters sent home to parents of all children between the ages of four years, zero months, and five years, 11 months, who were enrolled at a church-owned nursery school in Ransomville, NY. All of the subjects were white and from middle-income families. Two families declined participation because their children had been tested previously using the PDMS. All children whose parents returned recruitment forms expressing interest in participating were subsequently enrolled in the study.
The age range of four to six years was selected for the study because children of this age are frequently referred for gross and/or fine motor assessment. Referrals may result from concerns about a child’s motor development that parents and professionals consider potentially detrimental to the child’s success in kindergarten. Commonly children have to pass general motor assessment tests to attend kindergarten classes.
Parents of potential subjects were interviewed by telephone to determine whether the child met the criteria for participation. Inclusion criteria were parental consent and parental report of general ability to keep up with peers during gross motor play activities. Exclusion criteria were previous administration of the PDMS; history of developmental delay; diagnosis of orthopedic, cardiovascular, or neurological disorder; diagnosis of attention deficit disorder or hyperactivity; any parental concern about overall development; visual impairment not correctable by lenses; hearing impairment; and preterm birth (less than 37 weeks of gestation).
The principal investigator administered the PDMS Gross Motor Scale to each child after the child’s parent or caregiver read and signed an informed consent form approved by the Committee on the Protection of the Rights of Human Subjects at The University of North Carolina at Chapel Hill. At the time the study was conducted, the principal investigator had five years of experience as a pediatric physical therapist and three years of experience in administering and using the PDMS. The principal investigator discussed the testing procedure with each child before testing to familiarize the child with the investigator and procedures. The test was administered to all subjects in accordance with the instructions and equipment specifications in the PDMS manual. The investigator followed the instructions in the manual concerning number of demonstrations of each task and repetition of verbal instructions at the beginning of each trial.
The PDMS was administered once to each child in a 40 × 30-foot room at the nursery school; the room was equipped with a one-way mirror and was set up solely for standardized testing. The flooring of half of the room was linoleum, and the other half was covered with a rug approximately 0.25 inch thick. The room was separate from other rooms in the building and was therefore relatively free of outside distractions. Although the parents of three subjects chose to be present in the room during test administration, the parents of the other subjects watched from the other side of the one-way mirror. One subject’s four-year-old brother was also present with a parent in the room during test administration. A research assistant was present during all testing. The children were generally eager to participate after acclimating to the testing environment, principal investigator, and research assistant. The average time for total test administration was 45 minutes.
Separate scores were recorded for each gross motor task trial. Because up to five trials were allowed per task, up to five scores were recorded for each task. The decision to allow up to five trials per task was based on results of the pilot study 10 and on concerns about the generalizability of the results if additional trials were permitted. We thought that five trials for each task, requiring a total of approximately 45 minutes for test administration, would be feasible in most clinical settings. Items in which the standard test protocol allowed for scoring the best of two trials included the first two trials as the standard administration. Items in the receipt and propulsion category that allowed for the best two of three trials included the first three trials as the standard administration.
Scoring was based on the ordinal scoring procedure described in the test manual. 1 Each trial was scored using a three-point (score of 0 to 2) scale. A score of 0 was given when the child could not or would not attempt the task or when the attempt did not demonstrate that the skill was clearly emerging by 50% or greater. A score of 1 was given for each trial in which the child’s performance showed a clear resemblance to the item criterion of 50% or greater but did not entirely meet this criterion. A score of 2 was given when the child performed the task successfully according to 100% of all item criteria.
The lowest age level on the PDMS Gross Motor Scale used in testing our subjects was 24 to 29 months. Analysis of test items beginning at the 24- to 29-month level revealed that only one trial was allowed during standard administration for the majority of the items. For the balance skill domain, 44% of the tasks allowed for two trials. None of the tasks in the nonlocomotor skill domain allowed for more than one trial. For the locomotor skill domain, 12% of the tasks allowed for two trials. Receipt and propulsion was the only skill domain in which a maximum of three trials was permitted. Twenty-six percent of the tasks for receipt and propulsion allowed for up to two trials, and 37% allowed for up to three trials.
To address the first two research questions concerning the effects of additional practice on gross motor skill domain scores and total scores, we calculated the percentage of change in scores between standard test administration and administration with up to three trials per task and between standard test administration and administration with up to five trials per task. For items in which standard test administration included three trials per task, the scores for the first comparison were identical, and the percentage of change was 0%. Mean percentage change scores were determined for raw gross motor skill domain scores and raw total scores.
SYSTAT software (Systat Inc., Evanston, Ill) was used to perform all statistical analyses. A multivariate analysis of variance was conducted to examine the effects of gender, skill domain, and type of test administration (standard, maximum of three trials, and maximum of five trials). Gender was a between-subjects factor, whereas skill domain and type of test administration were within-subject factors. After a significant omnibus F test, separate two-way (gender by type of test administration) repeated-measures analyses of variance were conducted for each raw domain score and total score. Post hoc comparisons were conducted using Tukey honest significant difference tests. The significance level for these tests was set at p < 0.05.
To address the third research question concerning the occurrence of clinically significant changes in total scores, clinically significant change was operationally defined as scoring outside the 95% confidence intervals of the standard error of measurement as described by Hinderer et al. 7 Confidence intervals were determined for each of the raw domain scores and for the total scores based on standard PDMS protocols. The 95% confidence intervals were ±8.8 raw score points for subjects 48 to 53 months of age, ±12.2 raw score points for subjects 54 to 59 months of age, and ±6.3 raw score points for subjects 60 to 71 months of age.
Additional practice positively affected all individual gross motor skill domain raw scores in terms of mean percentage of change (Figs. 1–3). The largest percentage of change in all domain and total scores occurred when the children were allowed up to three trials, as compared with the standard number of trials. The addition of up to two more trials (for a total of five allowable trials) resulted in smaller changes. In comparing scores obtained using standard administration procedures vs administration with a maximum of three trials (Fig. 1), the largest percentage of change occurred in the receipt and propulsion domain. This change in scores occurred despite the fact that 37% of the tasks within the receipt and propulsion domain allowed for three trials during standard administration and therefore had identical scores for the two types of test administration. When the maximum number of trials increased from three to five, the percentage of change in receipt and propulsion nearly doubled, from 4.22% to 8.29% (Fig. 2).
Results of the multivariate analysis of variance revealed a significant main effect of type of test administration (Wilk’s λ = 0.16; F = 41.41;df = 2, 16;p < 0.001). Results of the separate analyses of variance revealed that all gross motor skill raw domain scores and total scores increased significantly with three to five practice trials (Table 1). Post hoc analyses indicated that mean scores obtained with three trials were significantly higher than those obtained with the standard number of trials for all domain and total scores (p < 0.05). A significant difference was present between three and five trials only for the mean receipt and propulsion domain scores and total scores. No significant gender or interaction effects were present.
Clinically significant change was determined by using the 95% confidence intervals based on standard PDMS protocol proposed by Hinderer et al. 7 Clinically significant change in total raw scores occurred for four (21%) of the subjects when given up to three allowable trials and for nine (47%) of the subjects when given up to five allowable trials. Clinically significant change did not occur in individual domain scores except for one subject in the receipt and propulsion domain.
Receipt and propulsion seems to be the gross motor domain most sensitive to increased allowable trials. Mean percentage of change for three vs five allowable trials was less than one percent for all domains except receipt and propulsion. There may be a variety of reasons for this sensitivity to practice with receipt and propulsion activities. These tasks may require the greatest amount of coordination and attention; therefore, there may be more room for improvement with practice. It may be more difficult to create a motor memory for receipt and propulsion tasks, and more practice may then be needed for optimal performance.
Observation of the children’s behavior during data collection indicated that five practice trials may not be advisable for some tasks. Children often were successful before the fifth trial, too fatigued to complete five trials, or unwilling to attempt the task after fewer than five trials. Children had the most difficulty completing five trials for tasks within the locomotor and nonlocomotor domains, and little change occurred between three and five trials in these domains. On the other hand, a significant difference was observed between three and five trials in the receipt and propulsion domain and total score. Therefore, up to five trials may be appropriate for preschool children for receipt and propulsion tasks, and up to three trials may be appropriate for tasks within the other gross motor domains. The absence of any interaction effects suggests that boys and girls of preschool age respond similarly to increased practice trials during administration of the PDMS.
The significance of these changes in test scores is important for pediatric evaluation and intervention. Changes outside a 95% confidence interval represent actual changes rather than changes caused by random fluctuations in scores. Almost 50% of the subjects demonstrated clinically significant improvements in raw total scores when up to five trials were allowed. The majority of changes occurred in the domain of receipt and propulsion. Possible reasons for the large effect on receipt and propulsion tasks may be that these tasks depend heavily on visual feedback and eye-hand/foot coordination. The children may have had less familiarity with receipt and propulsion tasks, with practice then leading to improvement in scores.
Changes in PDMS raw total scores of preschoolers who are typically developing with increased practice suggests that ability for task success is not captured by the standard number of trials. The authors of the PDMS have recently modified and renormed the assessment tool, creating the PDMS II. 10 The new edition allows up to three trials for each task. Other changes include specific definitions of the scoring criteria for each task (in terms of zero, one, or two) and grouping of similar tasks for greater ease of administration. The final version of the PDMS II was not available at the time this study was completed. We suggest that the number of trials for each test item be determined very carefully on the basis of empirical evidence to increase test specificity. This issue will remain an important consideration as long as performance-based assessment tools such as the PDMS are used.
Limitations of this study include the small number of subjects and the absence of more descriptive information on the subjects. Suggestions for possible future study include replication with a larger number of subjects, further reliability testing, and examination of the effects of number of demonstrations and the content of verbal instructions. Investigation of the effects of practice on scores obtained during administration of the PDMS Fine Motor Scale is also needed.
The results of this study are applicable to preschoolers who are typically developing. The study should be repeated with children with atypical development as well because evaluative scores influence medical and educational decisions. In view of the fact that the addition of three to five allowable trials influenced the PDMS scores of these preschoolers who are typically developing, the possibility exists that even more dramatic effects would be observed in preschoolers with atypical development, who often have limited movement experience.
The PDMS has been reported to have a higher frequency of identification of motor delays than other assessment tools. 6,11 Some children who score poorly on the PDMS may not have true motor deficits, but may simply lack experience with the tasks. Perhaps the PDMS should be renormed with decisions concerning the number of allowable trials for each task based on empirical evidence. Allowing an adequate number of practice trials during gross motor skill assessment may result in improved test specificity.
In this study the effects of increased allowable trials on PDMS scores of preschoolers with typical development were analyzed. The results of this study revealed that all domain and total scores improved significantly with increased trials, with the largest change in the domain of receipt and propulsion. The only statistically significant difference between three and five trials was in the receipt and propulsion domain. This suggests that five trials for the receipt and propulsion domain and three trials for the other domains may be necessary to determine optimal performance for task success in preschoolers.
The authors thank the children, their families, and Pastor Scott Clark of ABC Nursery School for making this study possible. The authors also thank Lynne Clark, Merton Wiepert, Gloria Wiepert, and Erin Woods for providing invaluable research assistance, and Pro-Ed for providing information on the PDMS II. The authors thank Drs. Darlene Sekerak, Cynthia Lewis, and Marie Reilly, who were part of the research committee. Lastly, the authors thank all faculty and staff at the University of North Carolina, Chapel Hill, who provided informal but helpful assistance for this study.
This study was completed by Ms. Wiepert in partial fulfillment of the requirements for the Master of Science degree at the University of North Carolina, Chapel Hill.
© 2002 Lippincott Williams & Wilkins, Inc.