Ibey, Robyn J. MScPT; Chung, Rochelle MScPT; Benjamin, Nicole MScPT; Littlejohn, Shannon MScPT; Sarginson, Andrea MScPT; Salbach, Nancy Margaret PT, PhD; Kirkwood, Gail PT; Wright, Virginia PT, PhD
Head injuries are the second most common reason for injury-based hospitalization in children and youth.1 Resulting impairments from an acquired brain injury (ABI) are multifactorial and can negatively affect gross motor activity performance.2–7 The recovery of advanced motor skills that support participation in sporting and recreational activities is often an important goal in the rehabilitation of children and youth who have an ABI.3,8 The underlying components of advanced gross motor skills that can be affected postinjury include balance and postural control (static and dynamic), coordination, speed and agility, and strength.2,7 These components are required for high-level gross motor activities such as running, hopping, jumping, and reversing direction quickly as well as for skills that require combined use of arms and legs, for example, running to catch a ball or jumping with a skipping rope. An outcome measure that assesses advanced motor skills is needed to effectively identify and treat residual deficits, and to evaluate response to treatment in these children and youth during their recovery. This measure could also be used to detect risk for injury and guide activity recommendations as youth with an ABI re-engage in school and community activities.
A number of outcome measures have been used to evaluate high-level gross motor skills within the pediatric ABI population, but none comprehensively capture an individual's progress postinjury through the rehabilitation and community phases of recovery.9 The best known of these measures are the Bruininks-Oseretsky Test of Motor Proficiency (BOTMP),10 the Community Balance and Mobility (CB&M) scale,11 the High-Level Mobility Assessment Tool (HiMAT),12 and the Gross Motor Function Measure (GMFM).13
The BOTMP was designed originally as a discriminative measure for use in screening of children with suspected motor impairments (eg, clumsiness, developmental coordination disorder, motor delay). Use of the full BOTMP, short form, and individual subscales has been extended to other patient populations (eg, cerebral palsy [CP], ABI, children with acute leukemia).10,14 The BOTMP's gross motor and fine motor scales evaluate various aspects of body coorduaination, balance, speed and agility, strength, visual-motor control, and fine motor dexterity.10,15 Normative data for the BOTMP have been established for children aged 4 to 21 years,10 and clinical studies have been conducted with individuals with CP16 and mild brain injury.17,18 The BOTMP discriminated between children with a mild ABI versus age-matched children developing typically with respect to their gross and fine motor abilities.17 However, published evidence of success in its use as an outcome measure in pediatric ABI is limited to work with the BOTMP fine motor scales18 and use of the balance subscale for youth with a mild ABI.19,20 The speed- and repetition-based response option format with the focus on normative scores and the lack of real-life activities limit its use as a functional outcome measure.
The CB&M scale was developed for adults with an ABI to measure postural stability in advanced-level tasks.11 Clinicians in pediatric rehabilitation centers have started to administer it to higher-functioning children and youth with an ABI, and there is evidence of excellent test-retest and interrater reliability with this population.21 The CB&M scale focuses on evaluation of the balance component of gross motor function, with some emphasis on assessment of performance speed. However, it does not directly evaluate other high-level gross motor concepts such as agility and strength, nor are the items tested entirely relevant to the advanced motor skills required in children's recreation and school contexts.
The HiMAT was developed to assess advanced mobility skill deficits in youth and adults with a traumatic brain injury (TBI).12,22 The item-generation process included pediatric outcome scales and the expert opinion of pediatric clinicians, but content validity was assessed only with adults. The HiMAT assesses lower-limb activities (walking, running, skipping, hopping, bounding tasks, as well as the ability to negotiate stairs)23 but lacks integrated multidimensional dynamic tasks, for example, activities that combine speed and agility, such as reversing direction quickly or integration of use of the upper limbs. Scoring is based entirely on time required to complete the task; that is, no quality of movement/performance considerations are taken into account. Although the HiMAT has strong psychometric properties with adults,23 there are no studies that validate and support its use in pediatric ABI.
The GMFM is a capacity-based outcome measure used internationally to evaluate gross motor abilities of children with CP and those with an ABI.13,24,25 Although it has strong psychometric properties in both diagnostic groups, children who have higher gross motor competence achieve top-level scores on it, despite persistent physical impairments and decreased participation in gross motor activities and sports. Indeed, the GMFM was designed to evaluate the foundations of gross motor function rather than high-level skills, efficiency or quality of performance, or the factors that affect participation. For example, there are no high-level timed tasks in the GMFM other than several static balance items in the Stand dimension. Furthermore, there are only 2 tasks that incorporate combined upper- and lower-limb use (carrying an object and picking something up from the floor).
The GMFM's ceiling effect was demonstrated in a study of youth with a TBI in which 80% achieved a maximum score on the GMFM at a 6-month follow-up, despite continued difficulties with balance, and the quality and efficiency of movement.9 Diminished responsiveness in high-functioning individuals was also observed in a study in which some children with a TBI continued to improve in gait velocity and cadence while GMFM scores remained constant.26 Notably, 20% of participants had maximum GMFM scores at baseline, making it impossible to track further improvements in gross motor abilities.26 Similar issues have been identified with high-functioning children with CP, and accordingly an advanced version of the GMFM is being developed by this article's senior author to allow for more sensitive testing of their skills.27
The purpose of this study was to develop a challenge assessment (CA) for outcome evaluation with high-functioning school-aged children and adolescents with an ABI, and to assess the feasibility of its administration. The ABI-CA is intended to measure changes in performance of advanced gross motor skills that represent balance and postural control (static and dynamic), coordination, agility and speed, and strength. Since the GMFM is well-accepted as a strong measure of foundational gross motor skills and is used in the early phases of rehabilitation of youth with an ABI,9 the ABI-CA was modeled after and designed to extend the walk/run/jump dimension of the GMFM. The majority of children and youth with an ABI have experienced typical development up until their injury, and they and their families have hopes that they will return to their previous activities and sports. This makes them different from children who have a chronic condition such as CP and are working throughout the rehabilitation process to acquire new physical skills. Because of this, we anticipated that skills that would be of importance for children with an ABI to perform might be quite different from those for children with CP. Consequently, a decision was made to develop a separate advanced gross motor test that is targeted specifically for children and youth who have an ABI.
This study consisted of a multistep process involving 3 measurement development phases: item generation, item selection, and pilot-testing of the ABI-CA.
Initially, a literature review was completed by using peer-reviewed publications (PubMed and CINAHL database searches), physical therapy and human kinetics textbooks, and published outcome measures. The goal was to compile a comprehensive list of high-level gross motor items/tasks for school-aged children and youth, and to identify the gross motor components (categories) in which they belonged. We recruited 4 physical therapists (PTs) to serve on the expert panel. Each had specialized training and clinical experience in the field of pediatric ABI (defined as having worked with this population for 5 years or more). Physical therapists who participated in an earlier GMFM challenge module (CM) development study for children with CP27 were excluded from this panel as it was felt that knowledge of the final items from the CM for children with CP might introduce a tendency to identify and select similar items for the ABI-CA. The task of the expert group was to discuss priorities for the content of the measure, review the formulated list of items, categorize them into the gross motor activity categories provided, and then evaluate the suitability of each item for inclusion in the ABI-CA.
Prior to the panel session, each panel member was sent an information package containing examples of items within 4 predetermined activity categories (balance and postural control, coordination, speed and agility, and strength). This allowed them to familiarize themselves with the item concepts and do independent thinking about these and other potential items before the session. At the start of the panel session, the group reviewed the investigators' list of preliminary items and together decided for each item whether it was (1) safe to test on a child with an ABI, (2) feasible to administer in the rehabilitation setting, and (3) important for reintegrating the child into home, school, and community activities. The panel also suggested other challenging gross motor activities to add to the list. Review of the panel's comments was undertaken later by the investigators to identify other item content considerations prior to the selection of the final set of items for the web survey.
Physical therapists in clinical practice with ABI expertise from Canada and other countries were identified from contact information listed on web sites of various physical therapy associations, pediatric hospitals, and brain injury networks, and were invited to participate in the item-selection web survey. Eligibility requirements for participation in this survey were a minimum of 1 year of recent clinical experience working with the pediatric ABI population and familiarity with the GMFM. Physical therapists who had participated in the previous study to develop a CM for the GMFM for children with CP were excluded.27 The web survey was carried out by sending an initial e-mail inviting PTs to take part in the study. Two reminder e-mails were sent 5 and 6 weeks later to encourage survey participation.28,29 In these e-mails, the PTs were given a link to the survey questionnaire with the inclusion/exclusion criteria indicated both within the e-mail and on the web survey. Physical therapists were on the honor system as to whether they met these criteria and undertook the survey anonymously. Participants were asked to rate the safety, feasibility, and importance of each listed item using a 5-point Likert scale (strongly disagree to strongly agree). Items were described in terms of the task to be performed and proposed method for scoring, for example, “Standing on toes on an uneven surface (soft exercise mat) holding for 20 seconds and progressing to eyes closed: scored on ability to hold each position for 20 seconds.” Respondents were asked to provide comments regarding listed items and to identify additional gross motor activities that they felt were relevant.
Survey responses were analyzed separately for each item. The number of strongly agree and agree responses were summed and then divided by the total number of respondents for each item to give an inclusion endorsement percentage score. This was performed for each criterion of safety, feasibility, and importance. As a first step, only items that achieved a minimum score of 80% for safety were considered for inclusion, as this was the primary testing concern identified by the expert panel. Next, the shortlisted items achieving a minimum score of 70% on both feasibility and importance were considered. Finally, the investigators reviewed the respondents' survey comments to determine the final pool of unique items for inclusion in the pilot version of the ABI-CA. Items that met inclusion criteria but were considered to be difficult to standardize or test in a PT setting, or were similar to other items, were modified or eliminated. Items that did not meet the minimum inclusion score for feasibility and importance were considered only if the group of items that was selected did not appear to fully represent a key skill set.
The investigators then developed a scoring response scale for each item on the ABI-CA that was modeled after the ordinal 4-point scale (range, 0-3 points) of the current GMFM. Response categories were individualized to the item and designed to capture task completion as well as speed or quality of movement (eg, performance style) associated with these advanced motor tasks. A score of “0” on the ABI-CA represented inability to initiate or partial completion of the desired task, whereas a score of “3” indicated optimal performance of the task (Figure 1).
ABI-CA Pilot Study Phase
A convenience sample of children from the ABI caseloads of 5 PTs at Holland Bloorview Kids Rehabilitation Hospital was recruited. The eligibility criteria are outlined in Table 1. The investigators were involved in obtaining assent/consent from each child and parent according to the study's Research Ethics Board-approved protocol and then administered the ABI-CA. Testing was conducted in a 40-m hallway and a stairwell, and tasks requiring parallel bars and balance beam were tested in the physical therapy gymnasium. An investigator provided instructions to the child and demonstrated the items prior to an attempt being made. A second investigator and V.W. were also at the test session and took observational notes. Allowing each investigator to be present for at least 2 test sessions meant that everyone on the team was able to experience the administration of the test firsthand and make comments on it. This helped to maximize the feedback for revisions. The child was given a maximum of 3 attempts for each item, similar to the GMFM administration guidelines. The ABI-CA scoring manual outlined the general administration procedures, as well as specific instructions and response scales for each item.
Following the test, the children were asked by the test administrator for verbal feedback on tasks they found easy/difficult, tasks they enjoyed/did not enjoy, suggestions for future revisions to the current tasks, and tasks they felt should be added. They were also given a verbal fatigue scale where they rated how tired they were immediately after the test (0 = not tired at all to 10 = extremely tired).
Each participant's test session was videotaped. Immediately after the session, the 2 investigators reviewed the video-footage, notes, and feedback from the participant to gather ideas about how items and scoring parameters could be modified and improved. At the end of the pilot phase, videos and field notes were reexamined to establish the final version of the item-specific response scales for the entire ABI-CA. Each participant's performance on the ABI-CA items was then formally scored from their test videotape, and these scores were used in the results summary that follows. Individual item scores were summed to produce the ABI-CA total score. Item mean scores were also calculated, and those that had a mean score of “0” or “3” were identified to give an idea of skills that were potentially too easy or too difficult to perform. This assisted in formulating recommendations for elimination, addition, or modification of the ABI-CA items.
Item generation from the literature review yielded 78 items, which investigators classified within (1) balance and postural control (static and dynamic), (2) coordination, (3) agility and speed, or (4) strength. The expert panel confirmed the relevance of these 4 predetermined activity categories and did not suggest additional categories.
Item-Selection Phase (Expert Panel and Web Survey)
The 4 PTs on the panel provided ideas on the basis of their extensive work with pediatric ABI (ie, they had a combined total of greater than 45 years of experience in this area). The expert panel rating process resulted in reducing the investigators' list from 78 to 47 items. Fourteen of these remaining items were in the balance category, 14 in coordination, 8 in agility and speed, and 11 in strength. The panel recommended eliminating items that were of low functional relevance (eg, side plank), required equipment not widely accessible to all clinicians (eg, tasks using a Bosu ball or trampoline), or simulated children's games (eg, musical chairs) that might not be widely understood across cultural groups. They also suggested limiting the number of sport-specific drills because a child who had not been exposed to the sport prior to their injury would be at a disadvantage when trying these. There was consensus to accept items involving a combination of gross motor components including dynamic movements, or altered sensory elements such as closing one's eyes during a task while standing on an uneven surface.
Eighty individuals and 15 organizations were contacted via e-mail or telephone to invite their participation in the web survey. Of the 86 PTs who began the questionnaire, 75 (87%) provided complete data. Analysis of their Likert scores revealed that 31 of the 47 items (65.9%) achieved inclusion endorsement scores of 80% for the safety criteria and 70% for feasibility and importance and thus were shortlisted for consideration for the pilot-test version of the ABI-CA. One other item, wobble board balance, was slightly below the safety cutoff score (endorsement score of 73.3%) while satisfying the feasibility and importance criteria (scores of 77.3% and 77.0%, respectively). Because of its unique focus on maintaining balance on an unpredictable surface, the wobble board was included in the pilot-test ABI-CA in a revised format requiring that testing be done within parallel bars to increase the testing safety.
The ratings for safety, feasibility, and importance for the items that were not accepted in the preliminary version of the ABI-CA are presented in the Appendix. Although it was important that ABI-CA's pilot-test items represented a broad skill range, it was critical to keep the test manageable in terms of length and the effort and focus required for children to complete it. Accordingly, the 32 provisionally accepted items were reviewed by the investigators to see whether, from a redundancy standpoint, any could be removed. Eight items were eliminated (see the Appendix) on the basis of their similarity to other accepted items that were more comprehensive or functionally based or had received higher importance scores. Thus, at the end of the entire item-selection review process, 24 of 47 items rated in the experts' survey were included in the ABI-CA pilot-test version (Table 2). Four are tested bilaterally, resulting in a total of 28 scored items for the ABI-CA pilot version.
ABI-CA Pilot-Study Phase
Four female and 2 male participants, aged 8 to 17 years, gave informed consent to participate in the pilot study. As per the study's eligibility requirements, they demonstrated strong abilities on foundational gross motor skills as shown by a mean score of 99.0% on the GMFM's walk/run/jump dimension (range of scores from 96% to 100%). Each ABI-CA testing session took approximately 1 hour to complete. As was expected with individuals with an ABI, participants displayed differences in their cognitive processing speed, attention span (increased distractibility), and understanding of each task during testing.30 This resulted in ongoing revisions to the instructions to make them more concise, as well as incorporation of a demonstration into the verbal explanation of the task. Counting aloud during timed tasks was added to the administration instructions to give participants constant feedback. Participants typically required repetitive cueing to complete some of the positional tasks, such as ensuring that their leg was fully extended while sitting on an exercise ball or in bird dog position.
Table 2 shows participants' scores on each ABI-CA item. One item, stand and reach, was eliminated after evaluating the second participant as it was predicted, on the basis of the ease of performance by the first 2 participants, that the maximum score for this item would be achieved by everyone in the sample (Table 2). This elimination resulted in 23 unique items being pilot tested. Because 4 items were tested bilaterally, 27 items were scored altogether. The group's ABI-CA total mean score was 50.7 out of 81.0 points (63.0%, SD = 17.4), and the item mean scores varied from 1.2 to 3.0 points (Table 2).
Following the test, participants provided comments about their thoughts on the items tested. New skill suggestions included reverse jumping jacks and running to kick a soccer ball. However, there was no consensus as to whether any of the items should be revised or eliminated or whether there were certain skills that should be added. All participants rated their level of fatigue as minimal (no more than 2 points out of 10 on the fatigue scale).
The item-generation, item-selection, and pilot-study phases resulted in a 23-item ABI-CA that was feasible to administer in a rehabilitation setting. The ABI-CA comprises high-level tasks that challenged the 6 participants, and the item scores demonstrated its ability to detect impairments in gross motor function not captured by the GMFM. The ABI-CA has the potential to expand the ability of rehabilitation professionals to evaluate and monitor gross motor function as children and adolescents with ABI transition into home, school, and community environments.
Discussion with the expert panel assisted in identifying the best candidate items for the item-reduction survey. The expert panel's confirmation of the 4 activity categories supported the conceptual framework for gross motor function that was used for the ABI-CA.2,7 The balance/postural control category was acknowledged to be the gross motor function component that is most pertinent to a child's independence. This is consistent with other high-level outcome measures used in pediatric ABI such as the CB&M scale that focus on balance. However, other components of gross motor function were acknowledged to be important for reintegration, supporting inclusion of items that target coordination, speed and agility, and strength.
For analysis of the web survey results, items were first judged with respect to safety because of the increased risk of falling and injury in this population. Next, items were assessed for their importance for reintegration into home, school, and community environments as well as their testing feasibility. To ensure the feasibility of applying the CA within rehabilitation centers and outpatient facilities, equipment and space demands were kept to a minimum, requiring inexpensive equipment and a test track that is easy to construct.
The web survey allowed the investigators to obtain input from Canadian and international PTs and was a critical step in establishing the ABI-CA's content validity. Few new items were generated in the web survey phase, indicating that the item-generation methods used at the outset were effective in obtaining a comprehensive set of items. In addition, the web survey provided an opportunity to confirm the clarity of the wording of the proposed items.
ABI-CA Pilot-Study Phase
The ability of the children in the sample to obtain at least partial accomplishment scores on the ABI-CA's items suggests that these skills are within the capabilities of high-functioning children and adolescents with an ABI. The total mean score of 50.7/81.0 (SD = 17.4) indicates that the ABI-CA has greater underlying potential to detect change with these participants than the GMFM (ie, all had scored > 95% on the GMFM's walk/run/jump dimension). Testing was completed within a reasonable testing time (ie, 1 hour) and without participant fatigue.
The item testing resulted in various revisions to the items, testing procedures, instruction manual, and rating scale, using input from the investigators and the participants. These youth had the most difficulty with repetitive unilateral heel lifts, likely due to strength issues as well as trouble in understanding what was required in the task. The item was modified several times during the sessions to more effectively test the skill. Of the 24 tasks tested, 3 appeared to be too easy (Table 2). Specifically, the stand and reach task was eliminated from testing early on as described earlier. Although 6 participants achieved the maximum score on the prone lying to running task, it was retained given the potential to increase its difficulty by adapting the response scale to include transitional movement speed. The wall squat strength task also had high scores, with 5 of 6 participants achieving the maximum score. This suggested the need for task modification such as increasing the duration of the position hold. Testing of a larger sample will show whether these latter 2 tasks when modified will be better at highlighting strength and stability issues experienced by children with an ABI.
The exposure to different sports before sustaining an ABI appeared to have an impact on performance ability on the ABI-CA. Sport-specific tasks such as bouncing a ball while walking and kicking a soccer ball were better performed by those who indicated that they had played basketball or soccer prior to their accident. The ability differences observed may also be attributed to differences in age and stage of development for refined sports-specific skills.31
As testing progressed, it became evident that consideration of movement or postural compensations for certain items might be a critical part of the response scale. Typical compensations for some of the strength items involved altered postural control, such as excessive lumbar lordosis or pelvic rotation, suggesting difficulty in core stability. For balance tasks, typical performance deviations included excessive arm use, forward trunk lean, and protective stepping. Inclusion of movement quality in the response scale would help to enhance the evaluation of change in skill performance over time because movement refinements are often addressed within PT sessions.
Timing was done for 7 items to determine ability to hold the prescribed static position, whereas speed was recorded for 9 other items to capture the swiftness of performance. Timing cut-points were first determined through testing by the investigators. As testing progressed, time cut-points were revised on the basis of participants' abilities. A task could have up to 4 time cut-points or intervals in its response set. In future development of the response scales, upper and lower time cut-points need to be refined. These could be derived from a combination of mean values from children and adolescents who are able-bodied for upper cut-point(s), and observed mean or lower quartile scores from children and adolescents with an ABI for lower cut-point(s).
The observed scores of items that rated both speed and accuracy reflected the difficulty children with an ABI have with dual or multitask items. Performance quality decreased with the introduction of a difficult coordination element into a timed task, and it was clear that an accuracy versus speed trade-off existed. Tasks such as ins and outs, jumping jacks, forward hopping, step-ups, and bouncing ball while walking are examples illustrating that the child's focus on speed affected the execution of a coordinated rhythmical pattern. Conversely, the negative effects of accuracy requirements on speed were shown in the walk and pick up bean bags task that was done between 2 parallel lines. These lines were used to evaluate the participant's ability to walk in a straight, narrow path. Participants focused on remaining between the lines rather than walking forward in a continuous and rapid fashion. It is difficult to know to what extent these performance issues were a result of the injury or the result of unfamiliarity with the task, that is, performance might improve with repeated trials. Further testing with a larger sample of children with an ABI as well as able-bodied youth is required to resolve this issue.
Several study limitations should be noted. The item-generation panel involved PT experts from 1 facility, possibly resulting in a bias in the ideas and views expressed. However, the use of the large international group for the web survey helped to counter any content validity limitations that this may have caused. The study sample size was small and participants were selected from 1 facility, and as such, conclusions cannot be drawn regarding the applicability of the measure to all high-functioning children with an ABI. In addition, the evolving instructions during testing (ie, the iterative measurement development process used) may have influenced the children's performance of the items. Item-rating consistency was optimized by having the team do the scoring following pilot-testing from the children's videos using the revised rating guidelines. Finally, while the 23-item ABI-CA was reasonably balanced in items per category (ie, 6 items in balance, 7 in coordination, 6 in agility, and 4 in strength), the categorization of items was preliminary and needs to be confirmed in future psychometric testing.
The next phase in the development of the ABI-CA will focus on further refinement of the response scales for the ABI-CA's 23 items. Revisions based on feedback obtained in this study as well as additional testing of youth with an ABI and youth who are able-bodied in various age cohorts will allow establishment of expected values for time, distance, and performance abilities that can then be used to further refine the response sets.
This research created a 23-item initial version of the ABI-CA for high-functioning children and adolescents with an ABI. A wide range of activities, some incorporating the upper extremity, are included in this item set. The existing items reflect high-level gross motor activities that PTs consider safe, important, and feasible to test. This work has demonstrated that within this population it is possible for a clinically based measure to capture performance limitations of advanced motor skills that are beyond the scope of the GMFM. The final stage of development of the ABI-CA will be to determine its reliability, validity, and responsiveness and to examine its scaling by using Rasch methods and factor analysis.32 These psychometric properties should be established before the ABI-CA is recommended for clinical and research use. The measurement development and pilot-testing approach taken in this study may provide a useful model for others to follow for the design of observational measures for use in physical therapy.
The authors thank the physical therapists who took part in the item-generation and item-reduction phases of the study, as well as the children who participated in the pilot-testing of the measure. The authors also thank Susan Cohen (research assistant, Bloorview Research Institute) for her assistance with the recruitment and Holland Bloorview Kids Rehabilitation Hospital for its support of the research activities.
APPENDIX Items Rejected for Acquired Brain Injury-Challenge Assessment Pilot-Test Version: Item Endorsement Percentage From Physical Therapists Web-Survey Cited Here...
Listed by category i...Image Tools