In the full Hempel examination, neuromotor function is evaluated in 5 domains: fine motor function, gross motor function, posture and muscle tone, reflexes, and visuomotor function. Qualitative aspects of motor functions (ie, prehension, sitting, crawling, standing up, standing, and walking), muscle tone, reflexes, and visuomotor function can be assessed. The classification of neurological dysfunction is based on specific criteria for single items in the various domains. Items are scored in qualitative categories, denoting the type and/or severity of deviancy using a precoded ordinal measurement scale.7,8
The Hempel screening7,8 consists of 21 items from the full neurological examination technique of Hempel. In the current project, only the items dealing with spontaneous motor behavior (n = 18) were taken into account. The sum of these 18 items resulted in the total screening score, with a range of measures from 16 to 58. The sequence of the items of the screening is more or less fixed, depending on the cooperation of the child. The interrater reliability of this screening is satisfactory.7,8 Only limited data are available on the validity of the screening.8 In a small group of children (n = 42) at risk of developing neuromotor disorders, the discriminating potential of this instrument was evaluated.7 The predictive value of the screening has not been tested.
The screening also results in the clinical classification “typical development” (“typical”) or “suspect for atypical development” (“suspect”) based on Gestalt perception. This is more subjective than the judgement of the items of the screening in accordance with the protocol.
Agreement between live observation of a child and his/her observation on a video recording using Hempel screening has not been studied.
The live observation was carried out by 2 physiotherapists (C.H. and G.V.), who observed the screening through a 1-way screen (Figure 3). Clinical details of the child were unknown to the observers. The full screening was recorded on video. For the assessment of agreement between the live observation and the observation based on the full video recording, both assessors reassessed the children after an interval of 2 months on the basis of the full recording. In the study of Heineman,10 the observers evaluated the video recordings of the same child twice, with an interval of 4 weeks for evaluating intrarater agreement of the Infant Motor Profile. Heineman's study, however, only used observation of a video recording. In our teleconsultation study, the interval between the 2 assessments (live observation of the child and observation of the full video recording) was chosen on the personal advice of Hadders-Algra (Professor of Developmental Neurology, University Medical Center Groningen, the Netherlands). Her advice was to wait at least 6 weeks between the live observation and that of the video recording, to avoid recall bias. This interval was sometimes longer for practical reasons (availability of the observers) and to ensure that there was no possibility of recall bias.
A short video summary was produced from the full video recordings of the screening of all the children from the teleconsultation study group by sampling 6 one-minute video clips from the full video recording, which includes the items of the screening (“6act”), and another short video summary, which shows the child at 6 one-minute intervals (2nd, 5th, 7th, 10th, 12th, and 15th minute (“6time”). Agreement was determined for both the total screening scores and the clinical classification “typical” or “suspect” between the full video recording and the short video clips (“6act”) and between the full video recording and the other short video clips (“6time”). All recordings were presented randomly (both the child and the type of recording) and were assessed by 3 observers: 1 pediatric physiatrist (C.M.) and 2 trainees in pediatric physiatry (M.H. and A.E.), with an interval of a minimum of 6 weeks. The clinical details of the children were unknown to the observers, except their ages.
Also agreement was determined between live observation of the child and observation on the full video recording for both the total screening scores and the clinical classification “typical” or “suspect.”
Exploration of Validity of the Clinical Classification Based on the Short Video Clips
The 21 children from the explorative validity study of the clinical classification also underwent Hempel screening the first time they came to our center for observation, which was videotaped. The 9 children with typical development also underwent Hempel screening. From the video recording of these 30 children, short summaries were sampled in a similar way to the teleconsultation study group. Agreement was determined between the clinical classification “typical” or “suspect” and the child's final diagnosis, which was determined at a later stage and after a full clinical workup, and typical development. The video clips were assessed by 2 pediatric physiatrists (C.M. and R.W.), who classified the neuromotor development of the children as “typical” or “suspect” (Figure 4). The clinical details were unknown to the observers, except their ages. The video clips were shown at random, with a period of 2 to 3 months between 2 observations.
The choice of observers fits with the practical situation of our center, in which the pediatric physical therapist assesses the children by means of Hempel screening and questions about suspect neuromotor development are put to medical specialists. An important factor is that there was no funding for this research, and thus it had to be carried out with the observers available.
Agreement between the total screening scores of the full video recording and the video clips was measured with the intraclass correlation coefficient (ICC). Agreement between the total screening scores of the live observation and the full video recording was measured with the ICC. Agreement in the clinical classification “typical” or “suspect” between the various types of assessment (live observation, full video recording, and both types of video clips) was determined with Cohen's kappa.
In the explorative validity study of the clinical classification agreement between the clinical classification based on the short video clips sampled from the full video recording of the first observation, or the full video recording of the screening of the children with typical development, and the presence of a final diagnosis or classification as typical development, was determined with Cohen's Kappa.
Statistical analyses were performed using SPSS version 16. Statistical significance was considered achieved at a α value less than .05. The kappa value indicates the strength of agreement, where 1 is complete agreement and 0 is chance agreement.11–13
The international terms for agreement (almost perfect, substantial, moderate, fair, slight, poor)11 are related to the associations between groups and are not reducible to individuals.
Agreement between the total screening scores based on observation of the child on the full video recording (“film15”) and his/her observation on video clips including the items of the screening (“6act”) was almost perfect; the ICC was 0.83 (Figure 5). The mean difference was −0.04 (SD = ±2.69); the range of measures was 16 to 58. The Bland Altman plot (Figure 6) shows that 95% of the score difference was between −5.31 and 5.24. Agreement between the clinical classification “typical” or “suspect” based on observation of the full video recording (“film15”) and observation of the same child on video clips including the items of the screening (“6act”) was substantial (Cohen's kappa: 0.63).
Agreement between the total screening scores based on observation of the child on the full video recording (“film15”) and his/her observation on video clips showing the child at 6 one-minute intervals (“6time”) was substantial; the ICC was 0.80 (Figure 7). The mean difference was 0.15 (SD = ±2.99); the range of measures was 16 to 58. The Bland Altman plot (Figure 8) shows that 95% of the score difference was between −5.71 and 6.01. Agreement between the clinical classification “typical” or “suspect” based on observation of the full video recording (“film15”) and his/her observation on video clips showing the child at 6 one-minute intervals (“6time”) was moderate (Cohen's kappa: 0.56).
Agreement between the total screening scores based on live observation of the child and observation of the full video recording (“film15”) was almost perfect; the ICC was 0.87 (Figure 9). The mean difference was −1.28 (SD = ±3.07); the range of measures was 16 to 58. The Bland-Altman plot (Figure 10) shows that 95% of the score differences was between −7.30 and 4.74.
Agreement between the clinical classification “typical” or “suspect” based on live observation and on the full video recording (“film15”) was substantial (Cohen's kappa: 0.66). Perfect agreement occurred for 91% of the children (in 49 of 54 cases), but the prevalence of the classification “typical” was low, which reduced Cohen's kappa.
In the explorative validity study of the clinical classification agreement between the clinical classification “typical” or “suspect,” based on the short video clips, and the presence of a neuromotor developmental disorder was substantial (Cohen's kappa: 0.75).
The present teleconsultation study suggested a satisfactory agreement between evaluation of a child's motor behavior on the basis of spontaneous motor behavior assessed with Hempel screening and evaluation on the basis of short video clips sampled from this screening.
There was a stronger agreement between the total screening scores of the full video recording of the screening and of both video clips than between the clinical classification “typical” or “suspect” based on the full video recording and on both video clips. An explanation might be that the criteria of the clinical classification “typical” or “suspect” of Hempel screening are based on Gestalt perception, which is more subjective than the use of clear criteria for the items of Hempel screening.
The results of this study are in line with the results of Craig et al,2 which showed that neurological examination on video recording compared favorably with face-to-face examination, and the study of Salih et al,3 which showed that the use of video clip examination is probably as good as live examination. Both studies were conducted with adults.
Evaluation of the scores in our teleconsultation study indicated that the items of Hempel screening had been scored more strictly for the video recording than for the live observation. This is in line with the results of Craig et al,2 showing that examination using video recording is more sensitive in detecting abnormalities than face-to-face examination.
The explorative validity study of the clinical classification showed that agreement on the clinical classification “typical” or “suspect” based on the short video clips and the presence of a neuromotor developmental disorder was substantial. This is in line with the findings of the systematic review of Hersch et al,1 describing relatively good sensitivity and specificity of physical examinations performed with teleconsultation, that is, short video clips. The explorative validity study of the clinical classification showed that screening with short video clips could have predictive validity and may form a good basis for teleconsultation.
A strength of this study lies in the fact that the children participating in both study groups were a representative group of children who were potentially eligible for teleconsultation. The diagnoses of the children in the explorative validity of the clinical classification portion of the study indicate a neuromotor developmental disorder. In most cases, this is not immediately obvious at birth but becomes apparent in the first few years. This group of children was thus eligible for teleconsultation. Other strengths were that the children's clinical details and final diagnoses were known to none of the observers and that the children were examined in an age-appropriate way and with a method applicable in pediatric rehabilitation.
The study also has several limitations. First, only limited data on the validity of Hempel screening were available. As the discriminative validity was tested in a small and heterogeneous group of children who were considered to be at risk of developmental disorders, interpretation of the differences in the quality of motor behavior with that of the children with typical development had to be done carefully. Second, data from the Hempel screening were limited and information on the construct validity of the screening was lacking. A third limitation is that the study group of the explorative validity study of the clinical classification was small and selective, because the children were included retrospectively on the basis of their diagnoses. Therefore, the results cannot be generalized to the population of children referred for teleconsultation. This also implies that sensitivity and specificity could not be determined. Further research with larger samples is needed to support the predictive validity of the video screening.
In the future, we hope that children with a suspected neuromotor developmental disorder can be detected earlier through teleconsultation.
Assessments based on short video clips of the spontaneous motor behavior part of the Hempel screening, as used in teleconsultation, agree in a satisfactory way with assessments based on the entirely spontaneous motor behavior part of the Hempel screening. The results of this teleconsultation study indicate that Hempel screening enables reliable and valid observation of video clips of spontaneous movement behavior, for which video clips including the items of the screening are preferred.
The explorative validity study of the clinical classification suggested that the assessment of short video clips may be a valid indicator of the presence of a neuromotor developmental disorder. This suggestion certainly needs to be tested in larger samples.
The authors thank their colleagues in the Division of Physical Therapy for Children at the Roessingh Rehabilitation Center (Cécile Hudepohl and Gerben Visser) and at the University Medical Center Groningen (Miriam Helmus and Agnes Elema) for performing part of the assessment; and Jitse Jagersma and Peter Kroon for their technical assistance with information and communication technology.
1. Hersch W, Helfand M, Wallace J, et al. A systematic review of the efficacy of telemedicine for making diagnostic and management decisions. J Telemed Telecare. 2002;8(4):197–209.
2. Craig JJ, McConville JP, Patterson VH, Wootton R. Neurological examination is possible using telemedicine. J Telemed Telecare. 1999;5(3):177–181.
3. Salih SA, Wootton R, Beller E, Gray L. The validity of video clips in the diagnosis of gait disorder. J Telemed Telecare. 2007;13(7):333–336.
4. Hadders-Algra M. The neuromotor examination of the preschool child and its prognostic significance. Ment Retard Dev Disabil Res Rev. 2005;11:180–188.
5. Bloo H, Grobbe FA. Telefysi, zorg op afstand dichterbij. De breedbandtoepassing video-teleconsultatie in de kinderrevalidatie. 2006. Publication number 06ET01.
6. Palisano R, Rosenbaum P, Walter S, Russell D, Wood E, Galuppi B. Development and reliability of a system to classify gross motor function in children with CP. Dev Med Child Neurol. 1997;39(4):214–223.
7. Hempel MS. The Neurological Examination of Toddler Age. Groningen; 1993. Thesis.
8. Hempel MS. Neurological development during toddling age in normal children and children at risk of development disorders. Early Hum Dev. 1993;34(1-2):47–57.
9. Hadders-Algra M. Variation and variability: keywords in human motor development. Phys Ther. 2010;90(12):1823–137.
10. Heineman KH. The Infant Motor Profile: a standardized and qualitative method to assess motor behavior in infancy. Dev Med Child Neurol. 2008;50(4):275–282.
11. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174.
12. Feinstein AR. Clinimetrics. New Haven, CT: Yale University Press; 1987.
13. Altman DG. Practical Statistics for Medical Research. London, UK: Chapman & Hall; 1991.
child; child development/physiology; feasibility studies; motor skills/physiology; neuropsychological tests/standards; preschool; reliability; teleconsultation; validity© 2013 Wolters Kluwer Health | Lippincott Williams & Wilkins and the Section on Pediatrics of the American Physical Therapy Association.