Secondary Logo

Journal Logo

RESEARCH REPORTS: MEASUREMENT

Psychometric Properties of a Standardized Observation Protocol to Quantify Pediatric Physical Therapy Actions

Sonderer, Patrizia PT, MAS; Akhbari Ziegler, Schirin PT, MSc; Gressbach Oertle, Barbara PT, MAS; Meichtry, André PT, MSc; Hadders-Algra, Mijna MD, PhD

Author Information
doi: 10.1097/PEP.0000000000000398

INTRODUCTION AND PURPOSE

Multiple early pediatric physical therapy (PPT) intervention strategies are used to treat infants with or at risk for neuromotor disorders or developmental delay.1 The interventions, such as neurodevelopmental treatment, Vojta, general developmental programs using infant stimulation, and conductive education,2–4 aim to support motor development and functional abilities and to reduce neuromotor dysfunction.5,6 The interventions, however, use different physical therapy actions. Evidence suggests that especially the infant stimulation programs are associated with improved developmental outcome,7 an effect that is present despite the large heterogeneity within these programs.2 This suggests that our understanding of the effective elements of PPT is limited. New intervention strategies have been developed, such as constraint-induced movement therapy (CIMT)8 and specific motor training programs, for example, action-observation training.9

Pediatric physical therapists (PTs) choose from a repertoire of actions.3,10,11 They implement those techniques with which they have had good results and positive experience in the past.12,13 This means that PPT in infants is characterized by large variations in practice,3 which makes it extremely difficult to evaluate the effect of PPT intervention.

Infant physical therapy varies with the PT and within regional area and countries because of differences in the education, health policies, family culture, and the context where therapy is provided. Therapy can be provided in a clinical setting or at home.3

To promote the development of evidence-based practice in PPT, we need to know how PPT is practiced and what actions the PT uses during a PPT session. Knowledge of the specific contents of PPT is a prerequisite in understanding the effect or the effects of PPT. The Groningen Observation Protocol (GOP) was developed to quantitatively assess the contents of PPT sessions. This standardized protocol is based on the literature and from clinical practice.12,14 GOP is a video-based method that classifies physical therapy actions observed during a PPT session into main categories, such as “neuromotor actions” and “communication.” Some main categories are divided into subcategories. For example, subcategories of neuromotor actions are “facilitation techniques” and “sensory experience.”

Blauw-Hospers et al14 reported that the GOP was a reliable instrument to assess PT actions in Dutch infant PPT sessions. And, in the Dutch context, the GOP has also been shown to be an appropriate and thorough assessment tool, as it classified greater than 97% of the PT actions into protocol categories and the categories were to a large extent mutually exclusive.14 Moreover, the GOP was able to document theoretically expected differences between 2 PPT approaches3 and revealed that specific PPT actions were related to infant developmental outcome.12,15 This indicates that the GOP has an appropriate construct validity. Recently, the GOP was updated to GOP 2.0 (Supplemental Digital Content 1 [https://links.lww.com/PPT/A172] and 2 [https://links.lww.com/PPT/A173]) to (a) include new developments in PPT, such as constraint-induced movement therapy (Baby-CIMT),8 (b) expand the PT items of the GOP for application in other countries, for example by adding specific actions of pediatric Vojta therapy, and (c) allow for a reorganization of categories, especially by grouping neuromotor actions into the main category of “neuromotor actions.” The latter reorganization resulted in 5 instead of 8 main categories.

The aim of this study is to assess the psychometric properties of the GOP 2.0 by testing (1) the inter- and intrarater reliability of GOP 2.0 in PTT sessions, using interclass correlation coefficients (ICCs) with 95% confidence interval and (2) the completeness of GOP 2.0 defined as its ability to classify the majority (>90%) of the PT actions.

Design

This study is part of a larger study on the quantification of the contents of PPT in different countries. For that study, 60 PPT sessions were video recorded in Switzerland, 30 videos of PTT sessions of infants aged 4.3 to 8.0 months, and 30 videos of PTT sessions of infants 8.3- to 14.0-month corrected age. The current study addresses GOP 2.0's completeness on the sample of 60 videos and GOP 2.0's reliability on 2 random samples of 10 videos (Figure 1). To determine intrarater reliability, 1 assessor (masked) assessed the 10 videos with an interval of 3 weeks. To assess interrater reliability, 2 assessors (masked) rated 10 videos independently.

F1
Fig. 1.:
Flow diagram of video selection.

The study design was approved by the ethical committee of the cantons St Gallen (EKSG) and Zürich (KEZ). Signed consent was received from all therapists and the parents of the infants.

PARTICIPANTS

Infants

All infants received PPT between June 2013 and August 2014 on the basis of referral of a pediatrician. Inclusion criteria were the presence of neurological dysfunction or/and developmental delay as diagnosed by the pediatrician. Infants diagnosed with plagiocephaly or/and torticollis only, or with congenital hip dysplasia, congenital heart diseases, or cystic fibrosis were excluded from the study, as the interest of the study was especially PPT actions in infants at high risk for neurodevelopmental disorders. The clinical and social characteristics of the infants are summarized in Table 1.

TABLE 1 - Characteristic of Infants
All Interrater Reliability Intrarater Reliability
YI OI All YI OI All YI OI All
Number of videos 30 30 60 6 4 10 5 5 10
Infant age in months CA, median (range) 6.1 (4.3-8.0) 11.3 (8.3-14.0) 8.1 (4.3-14.0) 7.3 (5.5-7.8) 11.3 (10.8-11.5) 7.5 (5.5-11.5) 7.3 (5.8-8.0) 11.5 (11.3-14.0) 9.6 (5.8-14.0)
Sex: male/female 15/15 22/8 37/23 3/3 3/1 6/4 1/4 4/1 5/5
Gestational age at birth in weeks, median (range) 37 (25-42) 38 (27-42) 37 (25-42) 39 (37-41) 39 (30-40) 39 (30-41) 40 (26-41) 35 (27-40) 40 (26-41)
Birth weight in grams, median (range) 2720 (545-4000) 2770 (690-4280) 2720 (545-4280) 3380 (2460-4000) 2652 (1200-3500) 3073 (1200-4000) 3720 (1030-4000) 1670 (690-3005) 2300 (690-4000)
Maternal education (n; H/M/L),a missing data, n 6/12/11, 1 10/11/8, 1 16/23/19, 2 2/3/1, 0 2/0/2, 0 4/3/3, 0 1/1/2, 1 2/2/1, 0 3/3/3, 1
Abbreviations: CA, corrected age; OI, older infants; YI, younger infants.
aLevel of education: L = low (compulsory schooling), M = medium (high school), and H = high (university).

Pediatric PTs

To be included in the study, pediatric PTs had to be working with infants between June 2013 and August 2014. The study did not have exclusion criteria for pediatric PTs. Forty PTs were involved in the study. The participating PTs had between 1 and 34 years (median 14 years) of experience in PPT. Table 2 summarizes the characteristics of the PTs.

TABLE 2 - Characteristics of Pediatric Physical Therapists
In Interrater Reliability Videos (n = 10) In Intrarater Reliability Videos (n = 10) In 60 Videos
Number of therapists involveda 9 8 40
Years of experience as PT
Median (range) 23 (9-33) 24 (9-36) 18 (3-40)
Years of experience as pediatric PT
Median (range) 21 (3-30) 19 (3-34) 14 (1-34)
Sex: male/female 0/9 0/8 4/37
Additional education
NDT, bobath, n 36
NDT, baby, n 10
Vojta, n 6
Sensory integration, n 15
Abbreviations: NDT, neurodevelopmental treatment; PT, physical therapist.
aSome PTs treated more than 1 infant (ie, a younger and an older infant).

METHODS

Video Recording of the PPT Session

The treatment session was recorded in the child's usual therapy setting and at its regular treatment time. The camera person (a physical therapist) influenced the treatment as little as possible while recording the session. The camera was positioned as far away as possible from the infant and therapist. Care was taken to have the infant and therapist in full view of the video, so that all therapeutic and infant actions were observed. The children's and therapist's data were coded and treated as confidential information.

GOP 2.0

The GOP 2.0 manual describes PT actions in 5 main categories: neuromotor actions, educational actions, communication, position, and situation (Supplemental Digital Contents 1 [https://links.lww.com/PPT/A172] and 2 [https://links.lww.com/PPT/A173]). Each category contains various subcategories (“behaviors”). For instance, examples of subcategories of the main category “neuromotor actions” are “facilitation techniques,” “self-produced motor behavior,” and “passive motor behavior,” and examples of subcategories of the main category “communication” are “information exchange” and “instruct.” The PT actions of the subcategories (“behaviors”) were further specified by modifiers; for example, the subcategory “facilitation techniques” was further subdivided into the modifiers “handling,” “pressure techniques,” and “tapping techniques.”

The training of the assessors (n = 2) included a 2-day training in Groningen by one of the developers of the GOP and GOP 2.0. The training consisted of intensive personal tutoring and practicing assessment of videos of PPT sessions with GOP 2.0 and the Observer XT program.16 After the 2 days of training, the assessors scored 5 videos of PPT sessions (Supplemental Digital Content 2, available: at https://links.lww.com/PPT/A173). Supervision by staff in Groningen was provided via Skype and e-mail. Finally, another day of “live” tutoring and practicing was performed. Training was considered completed when interassessor agreement with a trained Groningen assessor on the last 3 videos assessed reached a median ICC of more than 0.80.

The PT actions, as defined by GOP 2.0, were scored quantitatively with The Observer XT (Version 11.5; Noldus, Wagingen, the Netherlands) (Supplemental Digital Content 3, available at: https://links.lww.com/PPT/A174).14The Observer XT allows for assessing different actions taking place simultaneously during a PTT session with an accuracy of 0.01 second (Supplemental Digital Content 4, available at: https://links.lww.com/PPT/A175). Actions were measured in full seconds and were calculated from The Observer XT in percentage of the duration of the PPT session

Statistical Analysis

Three sequences of 5 minutes were selected from each video starting at 100 to 200 seconds, 500 to 800 seconds, and 1000 to 1300 seconds from the onset. Reliability was quantified with ICCs' 95% confidence intervals (CIs). For the interrater reliability, we used ICC2 (2-way random model); for the intrarater reliability, we used ICC1 (1-way random model). According to Shrout and Fleiss,17 values more than 0.75 reflect excellent agreement, values between 0.4 and 0.75 sufficient to good agreement, and values less than 0.4 insufficient agreement. PPT actions observed in less than 2% of time were excluded from the reliability analyses. Completeness was defined when at least 90% of PT actions could be scored with GOP 2.0. The R statistical software R version 2.14.118 was used for analyses.

RESULTS

The interrater and intrarater ICCs and their associated 95% CIs are summarized in Table 3. Interrater reliability was calculated for 17 behaviors. In 16 of the 17 behaviors, the ICCs indicated an excellent agreement. In the remaining behavior “transition,” the ICC indicated sufficient to good agreement (ICC, 0.70; 95% CI, 0.45-0.84). Interrater agreement was calculated for 26 modifiers. The ICCs indicated an excellent agreement in 15 modifiers (58%), a sufficient to good agreement in 9 modifiers (35%) and an insufficient agreement in 2 modifiers (8%). The latter occurred in the modifier “little variation” of the behavior “challenged to self-produced motor behavior” and in the modifier “minimal postural support” of the position “prone.”

TABLE 3 - Interrater and Intrarater Agreement on PPT Actions and Situations Determined With GOP 2.0
Intervention Interrater Agreement ICC (95% CI) Intrarater Agreement ICC (95% CI)
Neuromotor actions
Facilitation techniques 0.79 (0.61-0.89)a 0.99 (0.99-1)
Handling 0.75 (0.55-0.87) 1 (0.99-1)
Pressure technique n.o. 0.99 (0.99-1)
Sensory experience, state event 0.89 (0.79-0.95) 1 (0.99-1)
With the aim of body awareness 0.83 (0.55-0.88) n.o.
Passive motor experience 0.76 (0.56-0.88) 1 (0.99-1)
SPMB 0.87 (0.67-0.94) 0.98 (0.97-0.99)
Challenged to SPMB 0.86 (0.72-0.93) 0.99 (0.97-0.99)
Large variation 0.86 (0.73-0.93) 0.97 (0.94-0.99)
Little variation 0.39 (0.06-0.65) 0.89 (0.79-0.95)
Just at the verge of the infant's ability 0.82 (0.61-0.91) 0.99 (0.97-0.99)
CSPM, activity flows into facilitation, sensory, or passive experience 0.88 (0.75-0.94) 0.98 (0.96-0.99)
Large variation 0.46 (0.13-0.70) 1 (0.99-1)
Little variation 0.47 (0.15-0.70) 0.98 (0.95-0.99)
Handling techniques 0.80 (0.58-0.90) n.o
Sensory experience 0.67 (0.41-0.83) 0.94 (0.88-0.97)
Just at the verge of the infant's abilities 0.88 (0.76-0.94) 0.96 (0.91-0.98)
Without challenge 0.86 (0.73-0.93) n.o.
Not specified neuromotor action n.o. 0.96 (0.91-0.98)
Educational actions toward caregiver
Caregiver training 0.80 (0.62-0.90) 1 (1-1)
Not specified education 0.80 (0.62-0.90) 1 (1-1)
Communication
Information exchange 0.93 (0.86-0.97) 1 (1-1)
Instruct n.o. 0.66 (0.40-0.82)
Strict instruction n.o. 0.66 (0.40-0.82)
Provide feedback 0.83 (0.68-0.92) 1 (0.99-1)
Share information 0.89 (0.96-0.99)
Evaluating the procedure 0.62 (0.32-0.80) 0.98 (0.95-0.99)
Not specified communication 0.82 (0.66-0.91) 0.99 (0.97-0.99)
No communication 0.89 (0.78-0.95) 0.98 (0.95-0.99)
Positions
Supine 0.99 (0.99-1) 1 (1-1)
On flat surface 1 (0.99-1) 1 (1-1)
On lap PT/CG 1 (1-1) n.o.
Other surface n.o. 1 (1-1)
Pelvis not lifted 0.92 (0.83-0.96) 0.80 (0.62-20.90)
With imposed pelvis lift 0.95 (0.90-0.98) 0.72 (0.50-0.86)
Adaptive equipment 0.94 (0.89-0.97) 1 (1-1)
Not adaptive equipment 1 (1-1) 1 (1-1)
Prone 0.99 (0.98-0.99) 1 (1-1)
Across leg/arm of PT/CG 0.98 (0.95-0.99) 1 (1-1)
On flat surface 0.92 (0.84-0.96) 1 (1-1)
Other surface n.o. 1 (1-1)
Clear postural support n.o. 1 (1-1)
Minimal postural support 0.09 (−0.23 to 0.41) 0.69 (0.45-0.84)
No postural support 0.76 (0.56-0.88) 0.77 (0.58-0.88)
Adaptive equipment 0.58 (0.29-0.77) 0.99 (0.98-1)
No adaptive equipment 0.69 (0.44-0.84) 1 (1-1)
Side 0.99 (0.98-1)
On flat surface n.o. 0.99 (0.99-1)
Minimal postural support n.o. 0.99 (0.98-1)
No adaptive equipment n.o. 0.99 (0.99-1)
Sitting n.o. 1 (1-1)
On flat surface n.o. 0.93 (0.86-0.97)
On lap PT/CG n.o. 1 (1-1)
Clear postural support n.o. 0.85 (0.72-0.93)
Minimal postural support n.o. 0.81 (0.63-0.90)
No postural support n.o. 0.99 (0.97-0.99)
No adaptive equipment n.o. 1 (0.99-1)
Standing n.o. 1 (0.99-1)
On flat surface n.o. 0.99 (0.99-1)
Minimal postural support n.o. 1 (0.99-1)
No adaptive equipment n.o. 1 (0.99-1)
Transition 0.70 (0.45-0.84) 0.95 (0.91-0.98)
With a handling technique 0.72 (0.50-0.86) 0.94 (0.89-0.97)
No adaptive equipment 0.71 (0.35-0.87) 0.96 (0.91-0.98)
Situation of treatment
Motor activity/play 1 (0.99-1) 0.90 (0.80-0.95)
Dressing 0.99 (0.98-1) 1 (0.99-1)
Carrying n.o. 0.89 (0.78-0.95)
Abbreviations: CI, confidence interval; CSPM, challenged to self-produced motor; GOP, Groningen Observation Protocol; ICC, intraclass correlation coefficient; n.o., not observed action; PPT, pediatric physical therapy; PT/CG, physical therapist/caregiver; SPMB, self-produced motor behavior.
aValues in italics denote the ICCs of the behaviors, leaving the values of the modifiers in nonitalicized face.

Intrarater reliability and 95% CIs were calculated for 23 behaviors. All but one had excellent intrarater agreement; this was “instruct” in the domain of communication for which agreement was sufficiently to good. Intrarater agreement and 95% CIs were calculated for 40 modifiers. Thirty-seven had an excellent agreement (93%) and 3 had sufficient to good agreement. These were the modifier “strict instruction” of the behavior “instruct,” the modifier “with imposed pelvis lift” of the position “supine” and the modifier “minimal postural support” of the position “prone.”

To evaluate completeness of GOP 2.0, we analyzed the prevalence of the behavior “not specified neuromotor action.” This behavior was observed in 50 of the 60 videos and occurred with a median value of 3% (standard deviation: 2%) and a maximum of 6.7% of time. PT actions could be classified in more than 90% of the treatment time. The most prevalent action during the PPT sessions was motor activity occurring in 90.4% of time (median value; range 59.3%-100%). Within the main category of “neuromotor actions,” the various behaviors occurred in the following relative median frequencies: “facilitation” 20.6% (range 0%-77.4%), “sensory experience” 9.6% (range 0%-44.7%), “passive motor experience” 0.3% (range 0%-22.5%), “self-produced motor behavior” 19.1% (range 3.0%-53.53%), “challenged to self-produced motor behavior (CSPMB), allowed to continue activity by himself” 9.6% (range 0%-77.0%), and “CSPMB flows into facilitation, sensory, or passive experience” 24.8% (range 0%-61.8%). The time on educational actions was spent either on “caregiver training” (median value 21.3.0% of time, range 0%-100%) or “not-specific educational actions” 78.7% of time (median value, range 0%-100%). In about 90% of time communication occurred (median 89.9%; range 53.7%-100%); major part (60.5%; range 25,7%-98.1%) of the communication consisted of “not specified communication,” other types of communication consisted of 1.2% (range 0%-24.7%) “information exchange,” 5.1% (range 0%-38.0%) “instruct,” and 13.2% (range 0.2%-51.3%) “provide feedback.”

DISCUSSION

The good to excellent results on reliability and completeness of GOP 2.0 are consistent with previous research of Blauw-Hospers et al14 on the basis of the first version of the GOP. Blauw-Hospers et al14 reported that the GOP is a good tool to assess PPT actions in a standardized and reliable way. The present replication study, using a larger video sample than the Blauw-Hospers et al14 study, confirmed the appropriateness of GOP 2.0 for quantitative video analysis of PPT sessions.

Exceptions to the good to excellent reliability, interrater agreement for the modifier “little variation” of the behavior “challenged to self-produced motor behavior” and in the modifier “minimal postural support” of the position “prone” was insufficient. Both parameters require observation experience, as it is difficult to describe in words the exact criteria for “little variation” and “minimal postural support.”

The completeness of GOP 2.0 for PPT was very good, as more than 90% of the content of the PPT sessions could be classified by GOP 2.0. The percentage of neuromotor actions that could be classified was even higher than expected (97%). This high value of neuromotor behavior that could be classified matches the one of Blauw-Hospers et al.14 This means that GOP 2.0 is a good instrument to quantitatively describe the contents of PPT in infants with neuromotor disorder or developmental delay in Switzerland. As the content of PPT for high-risk infants is heterogeneous,3 a quantitative analysis of the contents of PPT is required to understand the effects of PPT.12,15 GOP 2.0 allows for the evaluation of the contents of PPT across countries. This may facilitate an understanding what other PPT colleagues do during therapy and promote the development of increasingly effective early intervention strategies.

A major strength of the study was the analysis of PPT sessions of a heterogeneous group of high-risk infants treated by a heterogeneous group of pediatric physical therapists who had varied levels of experience and education. The heterogeneity of the infants was larger than in Blauw-Hospers et al,14 who studied a selective group of infants who had been hospitalized in the Neonatal Intensive Care Unit of the University Medical Center Groningen and had shown definitely abnormal general movements19,20 at 10 weeks' corrected age. Our study indicates that GOP 2.0 is applicable in a selective sample of high-risk infants, and also in the more heterogeneous population of infants treated in PPT.

It may be considered a limitation that participation in the study was the therapist's choice. Thus, the findings of this study may not be generalized to all PPT, as the therapists who participated were characterized by an open mindedness to new knowledge and changes in therapy practice. Another limitation of the study is the artificial presence of a video camera during the PPT session. Even though it has been reported in literature that the presence of a video camera in general does not affect the behavior of people being filmed,21 some of the therapists told the researcher that they worked less “freely” in the presence of a video camera and a colleague therapist holding that camera.

In conclusion, GOP 2.0 is a reliable instrument to quantify the contents of PPT. GOP 2.0's completeness indicates that it is a good instrument to classify virtually all PPT actions observed during PPT session in Switzerland. We recommend 2 types of future studies: studies that apply GOP 2.0 in other countries, such that we obtain information on similarities and differences in PPT across the world, and studies using GOP 2.0 to evaluate the effects of PPT on developmental outcome.

ACKNOWLEDGMENTS

The authors thank all the infants and their parents who participated in the study as well as the participating therapists.

REFERENCES

1. Spittle A, Orton J, Anderson PJ, Boyd R, Doyle LW. Early developmental intervention programmes provided post hospital discharge to prevent motor and cognitive impairment in preterm infants. Cochrane Database Syst Rev. 2015;11:CD005495. doi:10.1002/14651858.CD005495.pub4.
2. Blauw-Hospers CH, Hadders-Algra M. A systematic review of the effects of early intervention on motor development. Dev Med Child Neurol. 2005;47:421–432.
3. Dirks T, Blauw-Hospers CH, Hulshof L, Hadders-Algra M. Differences between the family-centered “COPCA” program and traditional infant physical therapy based on neurodevelopmental treatment principles. Phys Ther. 2011;91:1303–1322.
4. Benzies KM, Magill-Evans JE, Hayden KA, Ballantyne M. Key components of early intervention programs for preterm infants and their parents: a systematic review and meta-analysis. BMC Pregnancy Childbirth. 2013;13(suppl 1):S10.
5. Sweeney JK, Heriza CB, Blanchaard Y. Neonatal physical therapy. Part I: clinical competencies and neonatal intensive care unit clinical training models. Pediatr Phys Ther. 2009;21:296–307.
6. Cameron EC, Maehle V, Reid J. The effects of an early physical therapy intervention for very preterm, very low birth weight infants: a randomized controlled clinical trial. Pediatr Phys Ther. 2005;172:107–119.
7. Blauw-Hospers CH, de Graaf-Peters VB, Dirks T, Bos AF, Hadders-Algra M. Does early intervention in infants at high risk for a developmental motor disorder improve motor and cognitive development? Neurosci Biobehav Rev. 2007;31:1201–1212.
8. Eliasson A-Ch, Sjöstrand L, Ek L, Krumlinde-Sundholm L, Tedroff K. Efficacy of baby-CIMT: study protocol for a randomised controlled trial on infants below a 12 month, with clinical signs of unilateral CP. BMC Pediatr. 2014;14:141.
9. Guzzetta A, Boyd RN, Perez M, et al. UP-BEAT (Upper limb baby early actio-observation training): protocol of two parallel randomised controlled trials of action-observation training typically developing infants and infant with asymmetric brain lesion. BMJ open. 2013;3:e002512.
10. Mayston M. Bobath Concept: A View From the United Kingdom. Laguna Beach, CA: NDTA Network; 2005:12(2).
11. Howle J. NDT in the United States. Laguna Beach, CA: NDTA Network; 2005:12(2).
12. Blauw-Hospers CH, Dirks T, Hulshof L, Bos A, Hadders-Algra M. Pediatric physical therapy in infancy: from nightmare to dream? A two-arm randomized trial. Phys Ther. 2011;91:1323–1338.
13. Mayston M. Editorial: bobath concept: [email protected]: mid-life crisis—what of the future? Physiother Res Int. 2008;13:131–136.
14. Blauw-Hospers CH, Dirks T, Hulshof LJ, Hadders-Algra M. Development of a quantitative tool to assess the content of physical therapy for infants. Pediatr Phys Ther. 2010;22:189–197.
15. Hielkema T, Blauw-Hospers C, Dirks T, Drijver-Messelink M, Bos AF, Hadders-Algra M. Does physiotherapeutic intervention affect motor outcome in high-risk infants? An approach combining a randomized controlled trial and process evaluation. Dev Med Child Neurol. 2011;53:e38–e15.
16. The Observer XT, version 11.5. Noldus Information Technology b.v. International Headquarters Wageningen, The Netherlands. www.noldus.com
17. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
18. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2015.
19. Prechtl HF, Einspieler C, Cioni G, Bos AF, Ferrari F, Sontheimer D. An early marker of neurological deficits after perinatal brain lesions. Lancet. 1997;349:1361–1363.
20. Hadders-Algra M. General movements: a window for early identification of children at high risk for developmental disorders. J Pediatr. 2004;145:S12–S18.
21. Albrecht TL, Ruckdeschel JC, Ray FL III, et al. Portable, unobtrusive device for videorecording clinical interactions. Behav Res Methods. 2005;37:165–169.
Keywords:

Groningen Observation Protocol; infants; pediatric physical therapy; quantify contents; reliability

Supplemental Digital Content

© 2017 Wolters Kluwer Health, Inc. and Academy of Pediatric Physical Therapy of the American Physical Therapy Association