Pediatric physical therapy interventions have the ability to affect both immediate and long-term outcomes for children and their families. An eclectic mix of interventions, specifically designed for each child and family is typically used to maximize functional outcomes.1–3 Furthermore, interventions may differ based on the context in which the services are delivered. For example, the plan of care and goals established by a school physical therapist (PT) may be written and implemented differently than by a private-practice PT for the same 6-year-old child with Down syndrome. Consequently, traditional therapeutic interventions present measurement challenges due to the multidimensionality and multidisciplinary interaction that is inherent in providing services to children and their families.4 Our inability to “disaggregate” traditional interventions results in a therapeutic approach that produces numerous confounding variables for clinical outcomes research.4
Although randomized controlled trials (RCTs) are considered the highest level of evidence on which to base practice, these designs are difficult to complete due to smaller numbers of potential consenting participants and difficulty in matching the control and treatment groups.5 When the scope is narrowed to the practice of pediatric physical therapy, there are even fewer options for RCTs given the ethical confines that accompany the use of children with disabilities in clinical research. Yet, the fact remains that proof of the effectiveness and efficiency of therapeutic interventions is still of great value to children and their families, healthcare payers, the physical therapy profession, and society as a whole.4
As an alternative to RCTs, the clinical practice improvement (CPI) approach provides a naturalistic view of treatment by examining what actually happens, but does not alter the treatment itself for purposes of evaluating the efficacy of an intervention.5,6 An inductive approach, or “experience-driven, bottom-up approach,” is used in the CPI model, which asks front-line clinicians to describe and characterize what they actually do during therapy.4 This information may then be organized and categorized to formulate a taxonomy to “systematically characterize the treatments, procedures, and interventions used in rehabilitation.”4 Using an observational study design, the CPI approach has the ability to uncover best practices, which can later be tested in validation studies or RCTs.7 In addition, unproductive activities and interventions can be “weeded out” in advance so that time is not wasted on further investigation.7
As part of their work on the Post-Stroke Rehabilitation Outcomes Project, Gassaway et al8 used the CPI approach to record exactly which interventions were being used with patients poststroke during inpatient rehabilitation admissions. All patients studied were in their first year of rehabilitation after a stroke.8 The main research question of the Post-Stroke Rehabilitation Outcomes Project was “What impact does each stroke rehabilitation activity or intervention, both individually or collectively, have on patient outcomes on discharge, controlling for patient differences including medical and functional status on admission?”7
During a 6-month period, multidisciplinary teams from 7 inpatient rehabilitation facilities met via conference call to develop a documentation form for each discipline within the rehabilitation team. The group of PTs agreed to use functional activities as the organizational theme since functional activities are a critical component of many therapeutic approaches.4 To assemble a comprehensive list of interventions and address regional differences in treatment approaches, all interventions used in any of the participating facilities were listed.4 The result was the development of the Physical Therapy Rehabilitation Activities (PTRA) form.
The initial version of the PTRA form was used by the site-representatives for 1 month. Based on feedback from site-representatives, the first revision was completed. Then the PTRA forms were pilot tested by participating facility therapists during treatment with patients poststroke, over the course of 3 months. At the conclusion of this pilot test, the PTRA went through its final revision, approximately 9 months after the process was initiated.4 The final version of PTRA was then used between 2001 and 2003 in 7 inpatient rehabilitation facilities (1161 patients in 6 facilities in the United States) and the data have produced several subsequent research studies on poststroke outcomes.8
In an attempt to begin developing a similar data bank of intervention-specific information from pediatric PTs, the logical first step was to develop a comprehensive data collection form. With the CPI model in mind, a Pediatric Physical Therapy Intervention Activities (PPTIA) data form was developed to record what pediatric PTs were actually doing in therapy. Susan D. Horn, PhD shared the final draft of the PTRA form along with definitions, instructions, and case studies used in the training manual for the Post-Stroke Rehabilitation Outcomes Project with the authors. These documents were modified by the authors to develop the first draft of the pediatric-specific PPTIA data form and Version 1 of the PPTIA Training Manual. A deliberate decision was made to keep the form to 1 page to elicit details of the therapy session without becoming cumbersome.
The PPTIA data form was designed to capture specific information regarding (1) the types of functional and/or developmental activities being addressed in therapy; (2) the specific intervention(s) used to address each type of activity; (3) the amount of time spent addressing each type of activity during the session (in 5-minute increments); (4) the service-delivery model; and (5) the setting in which the child was seen.
Establishing the validity of any instrument used in clinical practice is a requirement of the APTA Task Force on Standards for Measurement in Physical Therapy (APTA Task Force on Standards) to infer meaningful or useful information from the measurement.9 Portney et al10(p. 81) contend, “validity is not inherent to an instrument, but must be evaluated within the context of the test's intended use and specific population.” Validity, “cannot be directly observed, palpated, or measured. Rather, it is inferred from research findings and applied experience using personal as well as generally accepted standards.”11
Establishing face validity is the first step toward validation of the PPTIA, questioning whether or not the PPTIA “appears to test what it is supposed to and that it is a plausible method for doing so.”10(p. 82) Next, content validity is explored to determine the comprehensiveness of the item pool and the extent to which pediatric PTs believe the PPTIA truly reflects the content of their therapy sessions.12 Because content validity is not established by statistical measures, one of the most common methods to assess and judge the appropriateness of a tool is the use of a panel of experts who compare the objectives and purposes of the measurement tool with its actual content.11,13,14
The purpose of this study was to validate the PPTIA data form when used by pediatric PTs to describe the interventions applied in the treatment of children with developmental disabilities. Specific study objectives were to (1) establish face validity for the PPITA data collection form based on expert panel feedback, (2) assess content validity of the PPTIA data collection form using a sample of pediatric PTs from geographically diverse areas around the United States and clinically diverse settings (early intervention, school-based practice, and clinic or hospital-based practice), and (3) produce a final revision of the PPTIA Training Manual and data collection form based on feedback from therapist participants.
This study was approved for expedited review by the University of Puget Sound Institutional Review Board.
To first establish face validity, a nonrandom sample of 9 experienced pediatric PTs were asked to review the PPTIA Training Manual and data collection form. The authors determined this panel of experts based on our knowledge of pediatric PTs within the United States and Canada and our involvement with therapists through participation in research endeavors.
The expert panel received the study documents via e-mail. Each member of the expert panel reviewed the PPTIA Training Manual and data collection form individually and then completed an electronic survey, which was e-mailed back to the authors. Consent to participate was implied when the expert panel survey was returned to the authors.
The first part of the survey consisted of a table asking participants to provide feedback on each section of the PPTIA Training Manual related to clarity and accuracy. The second part of the survey asked participants to provide feedback on each section of the PPTIA data collection form related to clarity and relevance. Finally, each expert was asked if they believed that the PPTIA data form was “a plausible way to record what pediatric PTs are doing during therapy sessions.” Feedback received from the expert panel was collated for the first revision (Version 2) of the PPTIA Training Manual and data collection form.
Participants for the content validity phase of this study were recommended by the expert panel therapists or by colleagues of the authors. Each recommending therapist was asked to identify a few practicing pediatric PTs known to work in early intervention, schools, or clinics/hospitals. No other inclusion or exclusion criteria were used. An e-mail introduction and short description of the study was sent to all identified therapists. As a result, a nonrandom sample of 41 pediatric PTs contacted the authors and requested a participant packet, which was sent by mail. Therapists' consent to participate was implied by returning the PPTIA data collection forms and content validity survey to the authors in a preaddressed postage paid envelope.
To assess content validity, study participants were asked to identify up to 5 children (and a minimum of 2 children) from their active caseload for this study. The authors requested that at least 2 of the children had a diagnosis of cerebral palsy or were between the ages of 1 and 5 years. This was requested because the authors wanted to evaluate this form for potential use within a study of interventions for young children with cerebral palsy. The remainder of the children could be of any age or have any diagnosis. To preserve the anonymity of the children, the PPTIA data forms for each participant were prenumbered. The therapist participants were asked to read and study the PPTIA Training Manual and then use the PPTIA data form with the children they had identified. Study participants were not asked to change their interventions used with the children identified, but rather just use the PPTIA form to record what they did with the children during 1 treatment session.
After using the PPTIA data form with children during a 2- to 3-week period, participants were asked to complete a survey. The survey requested demographic information from each therapist participant as well as information about the age and diagnosis of each child whose therapy session was documented using the PPTIA. To assess content validity, participants were asked to complete an 11-item content-validity questionnaire (Likert scale, with 1 = strongly disagree to 5 = strongly agree) and provide feedback on how long it took them to complete each PPTIA form. At the end of the survey was a space for open-ended comments where therapists could make suggestions for additions, deletions, or modifications to the PPTIA Training Manual and data collection form.
Face Validity Phase
A total of 7 therapists consented to participate by returning the expert panel survey via e-mail. Therapists who provided input on the PPTIA as part of the expert panel were all female, with an average age of 53.7 years, (SD, 9.0; median, 55; range, 41–68 years). The expert panel had an average of 32.0 years of total clinical experience (SD, 8.5; median, 34; range, 20–45 years) and 30.6 years of practice in the area of pediatrics (SD, 7.6; median, 32; range, 20–43 years). All participants in the expert panel had advanced degrees and various specialty certifications. Table 1 describes the expert panel in more detail.
The most significant concern from the expert panel was the “severity rating” included on the original version of the PPTIA form. All 7 experts reported that this rating was problematic because it could not be operationally defined for all ages, diagnoses, and practice settings. Panel members suggested using the Gross Motor Function Classification System as a guide to determine severity level. However, this classification system is intended only for children who have a diagnosis of cerebral palsy, and the PPTIA was intended for use to document therapy with children of any age and diagnosis who receive physical therapy services.15 The authors chose to omit the severity rating from the first revision of the PPTIA data form since the intention of the form was to capture what was done within a therapy session rather than describe in detail the child who was being treated.
Additional feedback from the expert panel focused on 3 main areas: (1) communication and coordination, (2) transdisciplinary service delivery, and (3) developmental play. There were also recommendations for additional interventions. Details about concerns and modifications made to the form are described below.
The first issue was how to reflect the time pediatric PTs spend communicating and consulting with the child and family and other professionals on behalf of the family. Communication and consultation may occur within a therapy session and other times it occurs outside the session. Based on the feedback received, “Communication and Coordination” was moved from the “Type of Activity” section to the “Intervention Codes” section. In addition, documentation time was extracted in the “Service Delivery” section to purely reflect the time spent writing notes, reports, and other documentation related to the child's care.
The second issue was around the use of a transdisciplinary model where the PT is the primary interventionist who implements the family service plan. For example, a 9-month-old child who qualified for Early Intervention services in the area of adaptive skills, motor skills and cognitive skills might only be seen by a PT, who would provide intervention and address goals related to adaptive and cognitive development in addition to the more traditional focus on motor skills. This is a recommended model for early intervention providers in some areas of the country, and as a result, intervention codes for transdisciplinary interventions were added to the PPTIA form.16
Third, feedback was provided regarding the concept of developmental play. When working with children, almost all therapeutic activities have developmental play embedded in them. Expert panel participants were unsure whether developmental play should be considered a Type of Activity or an Intervention. Based on feedback from the expert panel, the operational definition of developmental play was expanded to include activities such as reach, grasp, and release, functional play with objects, and exploratory, physical, and constructive play; however, it remained in the Type of Activity section.
Finally, a few other interventions were added (oral motor interventions, reciprocating gait orthosis/parapodium, and heat/cold modalities) and operational definitions were modified based on expert panel feedback to improve clarity, accuracy, and relevance of both the PPTIA Training Manual and data collection form. The first revision (Version 2) of the PPTIA Training Manual and data collection form were used for the content validity phase of this study.
Of the 7 expert panel participants who returned the survey, 5 therapists answered the final question, “Do you feel the PPTIA data form is a plausible way to record what pediatric PTs are doing during therapy sessions?” All 5 therapists answered “yes” to the plausibility question, leading the authors to conclude that face validity had been supported.
Content Validity Phase
Of the 41 content validity participant packets mailed out, 27 were completed and returned. Table 2 describes the content validity sample. Participating PTs (24 women, 3 men) had an average age of 41.8 years (SD, 11.9; median, 38.5; range, 26–60 years). The sample had an average of 18.4 years of total clinical experience (SD, 12.1; median, 17; range, 1–36 years) and 15.3 years of practice in the area of pediatrics (SD, 11.7; median, 11, range, 1–36 years). The sample included 7 therapists with Pediatric Clinical Specialist certification and 6 therapists with Neurodevelopmental Treatment certification. Therapists from 12 states participated in the content validity portion of this study; however, the largest group of participants (48.2%) was from Washington State. A variety of clinical settings were represented, with 11 therapists who work primarily in a clinic or hospital setting (41%), 10 therapists who work in a school setting (37%), and 6 therapists who work in early intervention (22%). Therapists were asked to indicate the percentage of time spent in different therapy settings and the highest percentage setting was used for purposes of describing the sample.
Although participants were asked to use the PPTIA on at least 2 children who either had a diagnosis of cerebral palsy or who were between the ages of 1 and 5 years, treatment data from 126 pediatric physical therapy sessions were documented for children with an array of diagnoses between 6 months of age and 18 years of age (mean, 4.5 years of age; SD, 3.43). Table 3 provides descriptive statistics for the 4 most commonly reported diagnoses.
The results of the 11-item content validity questionnaire yielded ordinal level data and the results have been organized by question in Table 4. Using the criteria for agreement as responses marked either “strongly agree” or “agree” and the criteria for disagreement as responses marked either “disagree” or “strongly disagree,” over 96% of the participants agreed that the training manual provided adequate instruction to complete the PPTIA form. Therapists also agreed that the list of functional or developmental activities, which could be selected to describe the therapy session, was comprehensive (89% agreement), that it was obvious which activity best fit the actual therapeutic activity (93% agreement), and that the PPTIA accurately reflected the activities addressed (92% agreement).
On the basis of the feedback received in the comments section, 4 therapists reported that developmental play was difficult to quantify, stating that it could be any number of developmental or recreational activities, but it could also be an intervention used to facilitate functional mobility, weight shifting, etc. Confusion around accounting for documentation time was also noted by 3 participants who were unclear how it should be coded to be a part of an “Other Activity.”
Therapists were also in agreement that the intervention codes listed on the PPITA were comprehensive (82% agreement), that it was obvious which intervention code best fit the actual therapy technique (78% agreement), and that the PPTIA form accurately reflected the interventions used (85% agreement).
Two therapists provided feedback under the comments section about expanding the motor learning definition to include intervention for coordination and timing. There was some confusion as to whether the assistive devices should be listed if the child owns them or if they were used during therapy. There were also additional comments about the lack of positioning devices listed (Wheelchairs, standers, and ambulation devices were listed, but not seating options, sidelyers, etc.). Two therapists reported that many interventions overlap with others and that at times it might be difficult to choose the most important or primary intent of the intervention.
Overall, based on questionnaire results, therapists agreed that the service delivery section was not difficult to complete (89% agreement) and that they became more proficient completing the PPTIA form after using it several times (93% agreement). Therapists took an average of 9 minutes to complete the PPTIA on the first trial, an average of 7 minutes to complete the second trial, and an average of 5 minutes to complete the third and fourth trial. The fifth trial took 6 minutes on average. It is important to note that 22 of the 27 participating therapists actually completed all 5 trials.
When asked whether they could realistically complete the PPTIA after each treatment session, only 59% agreed that they could do so without undue burden (with 26% in disagreement). Participants were split as to whether they would prefer to complete the PPTIA form on the computer rather than on paper (41% agreed, 22% were neutral, and 37% disagreed). This preference seemed to vary largely depending on the environment that each therapist practiced in and their access to computers.
On the basis of all the feedback received, the PPTIA Training Manual and PPTIA data collection form underwent a final revision, which was primarily aimed at providing more specific operational definitions for the activities and interventions listed on the data collection form (see Appendix for the PPTIA Training Manual Version 3, available online). Figure 1 depicts the final revision of the PPTIA data collection form.
The results of this study suggest that the PPTIA data collection form may be a valid tool to reflect the interventions and activities used by pediatric PTs in their treatment of children with developmental disabilities. In further examination of the results of the content validity questionnaire, a few limitations should be noted (1) there were 4 different therapists who answered disagree to 1 or 2 questions on the Type of Activity and Intervention Code sections; however, no one disagreed uniformly across all sections of the PPTIA; (2) there were only 2 therapists who marked strongly agree with the statements in questions 6 and 7 regarding the Intervention Code section; and (3) there was 1 therapist who marked disagree to question 9 regarding becoming more efficient after using the PPTIA several times.
Because the PPTIA is a newly developed tool, the results from this study should be interpreted cautiously. Face validity and content validity are generally subjective concepts; therefore, further and ongoing assessment of validity will be needed on subsequent revisions of the PPTIA. Also, the sample used in this study was recruited based on colleague-recommendation and no inclusion or exclusion criteria were used. Although attempts were made to sample from geographically and clinically diverse areas, our sample may not be representative of pediatrics PTs as a whole because 48% of the content validity sample was from Washington State. Regional variations in therapist to child ratios within the educational environment have been noted across the country with the Pacific Region identified as having one of the lowest ratios of therapists to children.17 Almost half of the sample of PTs who used the PPTIA to assess content validity were school-based therapists from the Pacific Region, which may have influenced the service-delivery model or interventions used.
Also since we asked that therapists try to use the PPTIA on at least 2 young children with cerebral palsy of the 5 children they reported on, we may have more information on the validity of this form for this population rather than all children seen by PTs. Finally, the authors operationally defined the terminology used for activities and interventions, then adjusted those definitions based on the expert reviewers and finally the content reviewers, but we may not have defined these terms in a manner that is agreeable to all PTs.
One participant suggested the possibility that participating therapists may have actually changed their therapy sessions in terms of complexity or variety in anticipation of documenting the session with the PPITA during this study. The influence of such a “testing effect” may need to be controlled in further studies. Future research should be aimed at determining the interrater reliability of the PPTIA. This might be done by videotaping therapy sessions and then comparing the treating therapist's report on the PPTIA form with those of observing therapists.
The PPTIA data collection form has the potential to provide a wealth of information spanning both clinical and administrative contexts. Clinically, the PPTIA could be revised for use with specific diagnostic groups (cerebral palsy, muscular dystrophy, Down syndrome, Rett syndrome, etc.) to reflect interventions and activities that may be unique to that population. The specific type of information yielded from the PPTIA could aid researchers in the determination of treatment taxonomies for these diagnosis groups, using the CPI approach. In addition, a wealth of data could be collected for outcome studies to determine if there are specific therapeutic activities or interventions that are associated with improved outcomes.
Another application might be to revise the PPTIA form for use within a specific setting (early intervention center, school district, or clinic/hospital) so that the activities and interventions could be tailored to the facility or organization, and not include superfluous codes. For example, an early intervention setting data collection form might ask therapists to indicate if they are providing direct therapy or parent coaching and an inpatient hospital unit data collection form might ask therapists to describe pain management interventions.
The PPTIA data collection form could also be used as a clinical tool to help student therapists or new therapists isolate and hone their clinical decision-making skills. For example, does a student therapist or young clinician know why they are doing a particular activity and what functional goal it is addressing? Clinically, the PPTIA will allow a therapist to track the complexity and variety of therapeutic interventions over time and help identify when an intervention is becoming stagnant.
Administratively, the PPTIA could be used to track the amount of time therapists spend with the child/family and on other activities when the child/family are not present, such as completing the necessary documentation across ages and diagnostic groups. For example, do children with medical fragility or more severe motor/cognitive involvement require more nonbillable time for writing letters of medical necessity for durable medical equipment and coordinating with the child's medical team, compared with a child with general developmental delays? Or do younger children require more parent training and family education than older children? This information may help to determine more realistic productivity expectations, caseload or workload numbers, and caseload mix within a department.
The PPTIA could also provide administrators with information regarding the number of cotreatments, group therapy sessions, and consultations occurring during any given period of time. Finally, the PPTIA could be revised and used as the sole documentation form to document a therapy session and could also be coded to include billing information. Administratively, this could cut down on the overall time a therapist is spending on documentation and or billing each day.
The PPTIA, a form used to document interventions and activities that pediatric PTs use with children and families, was developed and analyzed for face and content validity. Two subsequent revisions were made based on the information from pediatric PTs who were determined to be experts and from practicing clinicians who used the form. The overall opinion of the experts and the clinicians was that the PPTIA was a valid form for documenting intervention with children. The PPTIA form has the potential to act as a template, which could be modified to suit clinical, administrative, and research needs, which will ultimately benefit the children and families who receive pediatric physical therapy services.
The authors thank all the therapists who generously gave their time and expertise to this study. Special gratitude is extended to Susan D. Horn, PhD, who willingly shared information and documents that were crucial to the completion of this study.
© 2009 Lippincott Williams & Wilkins, Inc.