INTRODUCTION AND PURPOSE
Children with cerebral palsy (CP) are the largest group of children with disabilities with a prevalence of 1.8 to 3.2 per 1000 live births.1,2 Cerebral palsy is a permanent, but not unchanging, disorder of movement and/or posture and motor function, due to a nonprogressive lesion or abnormality of the immature brain.3 Although the brain abnormality is nonprogressive, the movement and neuromusculoskeletal-related functions can change and cause limitations in relation to activity and participation.4
Muscle contractures and joint deformities are common in children and adolescents with CP, thus monitoring joint range of motion (ROM) is important.5 The development of contractures is multifactorial. From the outset, muscle shortening is dynamic and over time, fixed contractures may develop because of structural changes in the skeletal muscle in CP. Muscle-tendon units in children with CP do not lengthen enough to keep up with the growth of the bones. Contractures correspond to changes in muscle sarcomere length, fiber type, hypertrophy of extracellular matrix, fiber and fiber bundle stiffness, and stem cell numbers.4,6 When monitoring ROM longitudinally as well as monitoring progress and improvement after treatment and rehabilitation, an accurate and reliable measurement is crucial. It can be challenging for a child to participate in ROM measurements, since it requires the ability to relax and cooperate during the examination. In clinical practice, ROM measurements are often performed with a universal goniometer, a procedure associated with substantial sources of measurement error in both children with and without neurological deficiencies.7 The reliability of the universal goniometer has been studied in children with CP using different study designs and different results ranging from fair to excellent intratester reliability and poor to excellent intertester reliability for different joints.7–18 Intersessional errors are greater than intrasessional errors,7,11 and the precision of the measurement decreases when performed by different raters.11
The use of downloadable applications for smartphones in health care is increasing.19 There are different applications for measuring joint ROM using the inbuilt smartphone magnetometer, accelerometer, and camera to record angle measurement. However, this technology has not yet been examined as a clinical instrument for measuring passive joint ROM in children with CP, although it might offer a faster and more reliable assessment of ROM.20–24 Research to determine the reliability and validity of smartphone applications in children with CP is necessary prior to implementation into clinical practice.
The aim of this study was to examine the intrarater and interrater reliability, agreement, concurrent validity, and time spent using the smartphone photography-based goniometer application (DrGoniometer, CDM S.r.L, Milano, Italy) compared with the universal goniometer in the measurement of hip abduction, popliteal angle test, and ankle dorsiflexion in children with CP.
This was a cross-sectional study designed to determine the intrarater and interrater reliability and agreement of ROM measurements with a universal goniometer and a smartphone photography-based goniometer application in hip abduction, popliteal angle, and ankle dorsiflexion in children with CP. Concurrent validity between the 2 measurement methods was examined as well as the time consumption using both methods. The reporting of the study conforms to the Guidelines for Reporting Reliability and Agreement Studies.25
A total of 50 children were assessed by physical therapists in an examiners room at Department of Pediatrics, Aalborg University Hospital, before or after they attended their interdisciplinary medical appointment or at a separate occasion.
Inclusion criteria for this study were a confirmed diagnosis of CP. Children aged 4 to 15 years at all Gross Motor Function Classification Systems (GMFCS) levels were included. The GMFCS is a standardized ordinal classification tool of gross motor function capacity in children and adolescents with CP. The severity of disability is reflected in 5 levels where level I includes children who walk without limitations and levels IV and V include nonambulators.26 Exclusion criteria were surgery in the lower extremity within the last 6 weeks; any expression of pain or discomfort prior to the examination; significant joint deformity including fixed contractures of more than 20° from neutral position when placed in supine position; and any disability that made it difficult for the child to cooperate during the examination.
Before study commencement, ethical approval was obtained from the North Denmark Region Committee on Health Research Ethics (N-20160056). Parents and children were informed in writing about the study and parents of the participants gave their written informed consent prior to participation and data collection in accordance with the Declaration of Helsinki II. The study was also approved by the Danish Data Protection Agency.
Range-of-motion measurements of passive joint ROM at end range were performed using a standard 360° plastic universal goniometer with 20-cm movable arms, unit scale 1° following a predefined protocol, and using a photography-based goniometer application (DrGoniometer, CDM S.r.L, Milano, Italy, version 1.9) on an iPad.
The universal goniometer is commonly used in clinical practice. The reliability of the universal goniometer has been studied in children with CP with different study designs and different results ranging from fair to excellent reproducibility for intratester reliability and poor to excellent for intertester reliability for different joints.7–18
The goniometer application was originally designed for examination of joint ROM by use of a smartphone camera and has shown excellent reliability in elbow, knee, shoulder, and metatarsophalangeal joint goniometry,20–24 and concurrent validity22,24 in an adult population with and without impairments.
The children were assessed in an examination room in a relaxed, supine position on an examination bed at the Department of Pediatrics, Aalborg University Hospital, according to a predefined test protocol. During the test procedure, the parent and the child were allowed to read a book or watch a movie on an iPad together if it was necessary to keep the child preoccupied during the examinations. All ROM measurements of the individual child were completed on the same day to reduce measurement error and avoid measurement variability in day-to-day passive ROM. If the child had unilateral CP, measurements were performed on the more affected leg. If the child had a diagnosis of bilateral CP, 1 lower limb was chosen randomly by drawing lots.
The procedures for measuring hip abduction, popliteal angle test and ankle dorsiflexion followed guidelines established by Norkin and White,27 and the measurements were completed during 3 successive sessions using a universal goniometer and a photography-based application using the iPad camera. To avoid the ROM measurement of one session leading to an increase in the ROM of the subsequent session due to stretching of soft tissue structures around the joint, a 5-minute break between the second and third sessions was included.
Prior to the study, a pilot study was performed with rehearsal of both universal goniometer and smartphone photography-based goniometer following the predefined test protocol used in the main study on 6 children aged 5 to 16 years without neurological deficiencies at the Department of Pediatrics, Aalborg University Hospital. Subsequently, adjustments were made to the test protocol to separate the first and second measures performed by rater 1. This resulted in a break between the intrarater measurements. To ensure blinding, removable marks were used for anatomical landmarks and results were covered when performing the on-screen measurement.
Two physical therapists performed ROM measurements individually using the universal goniometer and the application. Both were trained clinicians with 14 and 25 years of clinical experience for rater 1 and rater 2, respectively. A third clinician, a trained occupational therapist, assisted and obtained the measurements from the universal goniometer and took the photographs with the application to secure blinding of the 2 raters. Each child was assessed on 3 successive occasions intraday; first session by rater 1; second session by rater 2; and 5 minute break and session 3 by rater 1 (Figure 1).
Each rater was alone in the room with the assistant and performed passive ROM measurement of hip abduction, popliteal angle and ankle dorsiflexion with the universal goniometer, and the application on each occasion. Prior to each session, each rater marked the anatomical landmarks according to the protocol to facilitate correct alignment of the universal goniometer arms and the markers in the application. The markers were removed completely after each session. Landmarks for the universal goniometer and application were as follows for hip abduction: Spina iliaca anterior superior at the left and right sides and the center of patella of the tested hip, popliteal angle: lateral epicondyle of the femur, greater trochanter and lateral malleolus, and ankle dorsiflexion: lateral malleolus and head of fibula.27
The universal goniometer was placed on the specific joint, and the universal goniometer arms were aligned with the landmarks for each joint measurement. When the individual rater was satisfied and approved the universal goniometer position, the assistant read and recorded the joint angle.
Marks were placed on the floor and the wall prior to the assessments to ensure that photographs were taken from the same distance. In the application, a virtual goniometer was placed by dragging cursors on the photographed landmarks directly on the screen on a photograph taken by the iPad camera. Each rater was blinded of the measured angle by covering the on-screen angle measurement result, when placing the virtual goniometer. The results were noted after the measurement, at least 1 day apart, by rater 1 in a separate data sheet without results from the child's other sessions. Subsequently, photographs and data were stored on a computer and joint angles were recorded.
Time spent was recorded at session 1 with a smartphone timer when the universal goniometer measurement was initiated by rater 1 until the goniometer was removed by the assistant. For the application, time spent with the patient was recorded from the start of the photograph until saving the photograph and subsequently from finding the photograph on the iPad, placing the markers and until saving the information.
According to de Vet et al,28 a sample of 50 children is required anticipating an intraclass correlation coefficient (ICC) of 0.8 and a confidence interval of 95% (95% CI ± 0.1 [0.7-0.9]).
Descriptive data, including mean measurement angles with standard deviations, were calculated for each session.
Two-way random-effects ICCs, 2,1-type absolute agreement, were used to determine intra- and interrater reliability and concurrent validity.29,30 Since ICC values are context specific and dependent on the heterogeneity of the study population, the interpretation of ICCs must be done with the specific study population in mind.31 In this study, we interpreted the ICC values based on guidelines offered by Portney and Watkins,29 whereby a value less than 0.40 indicates poor reliability, an ICC in the range 0.40 to 0.75 indicates fair to good reliability, and an ICC value greater than 0.75 indicates excellent reliability.
Agreement/measurement error expresses the variation of repeated measurements and provides an indication of to what extend the variation between 2 measurements can be attributed to measurement error.
Standard error of measurement (SEM) was calculated as SEMagreement = √σ2error = √ (σ2pt + σ2residual) as a measure of agreement/measurement error. This value is expressed on the actual scale of measurement, which is an advantage for clinical interpretation. Since ICC 2,1-type absolute agreement was applied, the smallest detectable change (SDC) was calculated on the basis of SEMagreement using the formula: SDC = 1.96 × SEM × √2 to determine the magnitude of change that would exceed the threshold of measurement error at 95% confidence level.30
Bland-Altman plots were produced, which gives a visual presentation of the differences plotted against the mean score for each subject between 2 measurements and the universal goniometer and the application.
Differences in time were calculated as time spent with the child performing the measurement with the universal goniometer and the application, respectively, and total time, which included time spent completing the measurement in the application in addition to time spent with the child. Data had a normal distribution and differences between the 2 methods were compared using a paired sample t test. The α level was set at P value less than .05 and all analyses were performed in R Statistics (R Foundation for Statistical Computing, Vienna, Austria, version 3.5.0 https://www.R-project.org/).
Fifty participants were enrolled in this study between November 18, 2017, and August 27, 2018, from the Department of Pediatrics at Aalborg University Hospital.
Two raters assessed the children with CP (25 females and 25 males), mean age 8.9 (3.2) years and GMFCS levels I to V, for participant characteristics (Table 1). Data were available for 49 children, while 1 child was not able to complete the ROM measurements. Of the 49 children, 2 children had missing values on a single joint ROM measure. Measures on this joint were excluded, while the remaining data were included.
TABLE 1 -
|Sex, n (%)
|Age, mean (SD), y
|Diagnosis (subtype), n (%)
|GMFCS level, n (%)
Abbreviation: GMFCS, Gross Motor Function Classification System.
Summary statistics from the intra- and interrater reliability and concurrent validity analysis are presented in Table 2. All ROM measurements, hip abduction, popliteal angle, and ankle dorsiflexion had excellent intrarater reliability while interrater reliability showed good to excellent ICC values.
TABLE 2 -
Results of the Intrarater and Interrater Reliability
and Concurrent Validity
of the Universal Goniometer and the Application
||Mean Difference (SD)
||ICC (95% CI)
|Intrareliability universal goniometer
|Interreliability universal goniometer
|Concurrent validity (rater 1)
|Concurrent validity (rater 2)
Abbreviations: CI, confidence interval; ICC, intraclass correlation coefficient; SDC, smallest detectable change; SEM, standard error of measurement.
The concurrent validity between the 2 measurement methods ranged from 0.67 to 0.76 indicating good to excellent between device ICC for rater 1 and excellent between device ICC 0.75 to 0.85 for rater 2.
The SDC for the intrarater assessment indicates that a change of 12.00° or greater in hip abduction, 16.52° in popliteal angle, and 13.99° in ankle dorsiflexion for the universal goniometer would be required to be 95% certain that the change is not due to measurement error. For the application, it was 10.26° in hip abduction, 15.47° in popliteal angle, and 9.92° in ankle dorsiflexion, respectively (Table 2).
The SDC values were higher for interrater reliability. Figure 2 shows the agreement between rater 1 and rater 2 for the application. On average, rater 2 scores 1.85° and 2.97° higher than rater 1.
The Bland-Altman plots show the agreement between the 2 methods for rater 1 (Figure 3). Outliers were present in all ROM positions. Systematic differences between the application and universal goniometer were seen in hip abduction and ankle dorsiflexion. Systematic differences were also seen in ankle dorsiflexion for rater 2 (data not shown).
Visual inspection of Bland-Altman plots for intrarater and interrater reliability indicated no systematic differences over the range of examinations, indicating no greater precision with practice from the first to the last patient (plots not shown).
Time Spent Using the 2 Measurement Methods
As shown in Table 3, time spent with the child was lower when using the application as compared with the universal goniometer for all 3 ROM measurements (P < .05). While there were no significant differences between the time spent in total using the universal goniometer or the application for the ROM measurement of hip abduction and popliteal angle, total time spent on measurement of ankle dorsiflexion was higher when using the application.
TABLE 3 -
Results of Time Spent Measuring Joint Range of Motion
||Time Spent UG − APP Difference, s (95% CI)
||Time Spent UG − APP in Total Difference, s (95% CI)
||APP in Total (SD)
||−4.99 (−11.86 to 1.88)
||3.47 (−3.32 to 10.26)
||−12.19 (−19.45 to −4.93)
Abbreviations: APP, time spent taking the photograph during the range-of-motion sessions with the child; APP in total, time spent taking the photograph and the subsequent on-screen measurement; CI, confidence interval; UG, time spent performing the universal goniometer measurement with the child.
This is the first study to examine intra- and interrater reliability and agreement of a smartphone application and concurrent validity between a universal goniometer and an application in ROM measurement of children with CP. We demonstrated that a smartphone application could be used to assess ROM with good to excellent intra- and intertester reliability. However, a relatively large amount of measurement error resulted in high SDC values and systematic differences between the universal goniometer and the application were demonstrated.
Consistent with previous findings from studies examining universal goniometer ROM measurement in lower limbs in children with CP,7,10,15,17,18 an excellent intrarater reliability was found when using both the universal goniometer and the application in an intraday session. The results might be different if the sessions were separated by more days. In agreement with the present findings, other studies in elbow, knee, shoulder, and metatarsophalangeal joint goniometry found excellent reliability of the photography-based application used in this study.20–23 Two of these studies showed very high ICC values greater than 0.96.20,21 The higher ICC values in these studies could be explained by differences in the joints measured and different populations, since these were obtained in fixed knee joint positions and the elbow joint in an adult and asymptomatic population.
The interrater reliability was good to excellent when using the application and the universal goniometer, but the 95% CI ranges were wider for the universal goniometer indicating greater variability in the measurement values, with a lower limit of the 95% CI for the hip abduction and popliteal angle measurement below 0.4. Compared with the intrarater reliability findings, both methods showed lower ICC values for interrater reliability, which is common in studies of joint ROM measurement.27 This is also consistent with previous studies on lower limb joint goniometry in children with CP.15,18 Otter et al23 measured dorsiflexion of the first metatarsophalangeal joint using a photography-based goniometer application in an adult and asymptomatic population and reported a similar interrater ICC value (ICC: 0.708; 95% CI, 0.597-0.799) as in our study in children with CP in the age range of 4 to 16 years across all GMFCS levels while other studies reported higher ICC values ranging from 0.92 to 0.998 in joints as knee, elbow, and shoulders in an adult and asymptomatic population.20–22 Potential explanations could be the different population, joints, or joint positions under study.
Comparison of Smartphone Photography-Based Application and Universal Goniometer
The application displayed higher ICC values than the universal goniometer for both intrarater and interrater reliability. Upon reaching end-range joint positioning, the photograph is taken quickly with subsequent measurement of ROM compared with the universal goniometer, where it takes times to place the universal goniometer according to the landmarks, which is demanding for the child that must lie still and variation in body position may occur. Furthermore, the assistant must read the universal goniometer instantly allowing for misreading of the universal goniometer measurement, while the onscreen joint measurement when using the application leaves the possibility to subsequent adjustment of the markers.
Concurrent validity between devices demonstrated values ranging from good to excellent. For rater 1, the lower limit of the 95% CI was 0.042 for hip abduction and 0.121 for ankle dorsiflexion, while it was 0.749 and 0.438 for rater 2. A possible explanation for the differences between raters might be the fact that the concurrent validity for rater 1 was calculated on data from the last round of measurements, when the child's concentration and surplus energy might have been challenged.
This study compared an application to the universal goniometer typically used in a clinical setting. It is important to recognize that the use of the clinically applicable universal goniometer is considered as a reference standard and not a gold standard. Systematic differences between the universal goniometer and the application were present. The application underestimated ankle dorsiflexion measurement with 4° to 5° compared with the universal goniometer for both raters, while hip abduction measurements were overestimated with 5° only in rater 1. Since this difference in ankle dorsiflexion was seen in both raters, it could indicate the risk of a systematic measurement error between devices. This highlights that these methods cannot be used interchangeably.
The use of the application resulted in a reduction in time spent with the child for all 3 ROM measurements as compared with the universal goniometer. In ankle dorsiflexion, but not in hip abduction and popliteal angle, there was a significant increase in time spent in total using the application. These findings suggest that the use of the application can be an advantage when several ROM measurements are required repeatedly or to be performed as quickly as possible in consideration for the child, thereby being a viable alternative to the universal goniometer in clinical practice.
Methodological Strengths and Limitations
Despite acceptable levels of intrarater and interrater reliability and between the 2 measurement methods, random variation exists. Potential sources of measurement error can be attributed to 3 different components of the measurement system; the rater, the participant, and the measuring instrument.29 A protocol was developed and pilot testing was performed prior to this study to minimize measurement error. However, differences in rater performance may have affected reliability since stretching time as well as strength of force applied by the rater prior to measurement was not standardized because of the child's active resistance and ability to relax. We expected these aspects of rater variance to influence intratester as well as intertester reliability but considered the rater's subjective assessment more reliable than standardizing the stretching time and the applied force.
Accurate measurement of ROM involves determination of end-range joint positioning. The raters tried to prevent compensatory movements, for example, pelvic rotation in hip abduction as well as hip and knee flexion of contralateral leg when measuring the biarticular hamstrings in the popliteal angle measurement. To standardize the universal goniometer and application measurement and to ensure the generalizability to clinical practice, this study did not allow for the assistant to stabilize the opposite extended leg, since this was not possible during the photograph. This is a possible explanation why the popliteal angle measurements display the highest measurement error and SDC values, intrarater 16.52° and 15.47°, and intertester 23.23° and 20.48° for the universal goniometer and application, respectively.
Another reason for difficulties in determining the true end-range position could be increased muscle tone. Kilgour et al7 examined children with and without a diagnosis of CP and found that the presence of spasticity did not lead to increased measurement error in a group of children with mild spastic diplegia. While this study intended to include all GMFCS levels thereby potentially presenting a study group with larger heterogeneity, it is important to acknowledge that 64% of the children were GMFCS I-II, which might pose as a limitation of this study, since this group probably is less prone to spasticity than children with higher GMFCS levels.32
Furthermore, the participants had different ability to relax in supine position during the in-hospital examination as can be the case in a common therapy situation in daily clinical practice. Some were able to relax while others got excited during the measurements, which affected muscle tone. These environmental factors may have influenced the ability to determine the true end range and the stability of both measures as well.
Implications for Clinical Practice
The findings of this study suggest that a photographic-based application can be used for ROM measurements with good to excellent intra- and intertester reliability. The advantage of photographic-based ROM measurements is that it requires less time spent with the child. This can be an advantage if you have to document ROM examinations of toddlers or children who are challenged in their ability to lie still during ROM measurements. Furthermore, it is possible to manage the child with both hands, which can be an advantage in the examination of children with a high degree of spasticity or involuntary movements or full-grown adolescents. One disadvantage is that the photographic-based measurements require an assistant to be used in clinical practice, while the one-hand held universal goniometer can be used without an assistant.
Secondary impairment in CP, as decreased ROM, is associated with a decrease in gross motor function capacity33 and hip pain,34 and passive ROM is considered an important factor in clinical practice to monitor joint ROM longitudinally in order to detect deterioration and potential improvements from treatment. Based on our results, it is important to account for the relatively large measurement error and SDC (Table 2) when evaluating ROM in clinical practice. Despite measurement error, a fluctuation in hip abduction ROM between 35° and 45° is of less clinical importance compared with an alteration toward measurements between 15° and 25° but still important to monitor, for example, in relation to gross motor capacity and personal ADL activities. Considering the large amount of measurement error and the relatively high SDC, passive ROM measurements cannot stand alone as an outcome measurement. To support clinical decision making regarding treatment or compensating interventions in children with CP, other measures of function and patients' goals must therefore be considered alongside ROM.
This study reveals that a photography-based application can be a reliable and valid tool both within and between raters for measuring passive ROM, provided a standardized protocol and practice. Because of measurement error, changes in joint ROM must be interpreted with caution for both measurement methods. Systematic differences between the 2 measurement methods suggest that the devices should not be used interchangeably.
The authors thank physical therapist Pia Gartner Nielsen for assisting in participant assessment and statistician Peter Enemark Lund and Regitze Kuhr Skals as well as a special thanks to the children and their families, who participated in this study.
1. Hoei-Hansen CE, Laursen B, Langhoff-Roos J, Rackauskaite G, Uldall P. Decline in severe spastic cerebral palsy
at term in Denmark 1999-2007. Eur J Paediatr Neurol. 2019;23(1):94–101. doi:10.1016/j.ejpn.2018.08.010.
2. McGuire DO, Tian LH, Yeargin-Allsopp M, Dowling NF, Christensen DL. Prevalence of cerebral palsy
, intellectual disability, hearing loss, and blindness, National Health Interview Survey, 2009-2016. Disabil Health J. 2019;12(3):443–451. doi:10.1016/j.dhjo.2019.01.005.
3. SPCE. Surveillance of cerebral palsy
in Europe: a collaboration of cerebral palsy
surveys and registers. Surveillance of Cerebral Palsy
in Europe (SCPE). Dev Med Child Neurol. 2000;42(12):816–824. doi:10.1111/j.1469-8749.2000.tb00695.x|.
4. Graham HK, Rosenbaum P, Paneth N, et al. Cerebral palsy
. Nat Rev Dis Prim. 2016;2(1):15082. doi:10.1038/nrdp.2015.82.
5. Nordmark E, Hägglund G, Lauge-Pedersen H, Wagner P, Westbom L. Development of lower limb range of motion from early childhood to adolescence in cerebral palsy
: a population-based study. BMC Med. 2009;7:65. doi:10.1186/1741-7015-7-65.
6. Mathewson MA, Lieber RL. Pathophysiology of muscle contractures in cerebral palsy
. Phys Med Rehabil Clin N Am. 2015;26(1):57–67. doi:10.1016/j.pmr.2014.09.005.
7. Kilgour G, McNair P, Stott NS. Intrarater reliability
of lower limb sagittal range-of-motion measures in children with spastic diplegia. Dev Med Child Neurol. 2003;45(6):391–399. doi:10.1111/j.1469-8749.2003.tb00418.x|.
8. Stuberg WA, Fuchs RH, Miedaner JA. Reliability
of goniometric measurements of children with cerebral palsy
. Dev Med Child Neurol. 1988;30(5):657–666. doi:10.1111/j.1469-8749.1988.tb04805.x|.
9. Darrah J, Wiart L, Gorter JW, Law M. Stability of serial range-of-motion measurements of the lower extremities in children with cerebral palsy
: can we do better? Phys Ther. 2014;94(7):987–995. doi:10.2522/ptj.20130378.
10. Kim D-H, An D-H, Yoo W-G. Validity and reliability
of ankle dorsiflexion measures in children with cerebral palsy
. J Back Musculoskelet Rehabil. 2018;31(3):465–468. doi:10.3233/BMR-170862.
11. McDowell BC, Hewitt V, Nurse A, Weston T, Baker R. The variability of goniometric measurements in ambulatory children with spastic cerebral palsy
. Gait Posture. 2000;12(2):114–121. doi:10.1016/S0966-6362(00)00068-0.
12. Allington NJ, Leroy N, Doneux C. Ankle joint range of motion measurements in spastic cerebral palsy
children: intraobserver and interobserver reliability
and reproducibility of goniometry and visual estimation. J Pediatr Orthop B. 2002;11(3):236–239.
13. Fosang AL, Galea MP, McCoy AT, Reddihough DS, Story I. Measures of muscle and joint performance in the lower limb of children with cerebral palsy
. Dev Med Child Neurol. 2003;45(10):664–670.
14. McWhirk LB, Glanzman AM. Within-session inter-rater reliability
of goniometric measures in patients with spastic cerebral palsy
. Pediatr Phys Ther. 2006;18(4):262–265. doi:10.1097/01.pep.0000234960.88761.97.
15. Ten Berge SR, Halbertsma JPK, Maathuis PGM, Verheij NP, Dijkstra PU, Maathuis KGB. Reliability
of popliteal angle measurement: a study in cerebral palsy
patients and healthy controls. J Pediatr Orthop. 2007;27(6):648–652. doi:10.1097/BPO.0b013e3180dca15d.
16. Mutlu A, Livanelioglu A, Gunel MK. Reliability
of goniometric measurements in children with spastic cerebral palsy
. Med Sci Monit. 2007;13(7):CR323–CR329.
17. Glanzman AM, Swenson AE, Kim H. Intrarater range of motion reliability
in cerebral palsy
: a comparison of assessment methods. Pediatr Phys Ther. 2008;(20):369–372. doi:10.1097/PEP.0b013e31818b7994.
18. Herrero P, Carrera P, García E, Gómez-Trullén EM, Oliván-Blázquez B. Reliability
of goniometric measurements in children with cerebral palsy
: a comparative analysis of universal goniometer and electronic inclinometer. A pilot study. BMC Musculoskelet Disord. 2011;12:155. doi:10.1186/1471-2474-12-155.
19. Terry M. Medical apps for smartphones. Telemed J E Health. 16(1):17–22. doi:10.1089/tmj.2010.9999.
20. Ferriero G, Sartorio F, Foti C, Primavera D, Brigatti E, Vercelli S. Reliability
of a new application for smartphones (DrGoniometer) for elbow angle measurement. PM R. 2011;3(12):1153–1154. doi:10.1016/j.pmrj.2011.05.014.
21. Ferriero G, Vercelli S, Sartorio F, et al. Reliability
of a smartphone-based goniometer for knee joint goniometry. Int J Rehabil Res. 2013;36(2):146–151. doi:10.1097/MRR.0b013e32835b8269.
22. Mitchell K, Gutierrez SB, Sutton S, Morton S, Morgenthaler A. Reliability
and validity of goniometric iPhone applications for the assessment of active shoulder external rotation. Physiother Theory Pract. 2014;30(7):521–525. doi:10.3109/09593985.2014.900593.
23. Otter SJ, Agalliu B, Baer N, et al. The reliability
of a smartphone goniometer application compared with a traditional goniometer for measuring first metatarsophalangeal joint dorsiflexion. J Foot Ankle Res. 2015;8:30. doi:10.1186/s13047-015-0088-3.
24. Jenny J-Y, Bureggah A, Diesinger Y. Measurement of the knee flexion angle with smartphone applications: which technology is better? Knee Surg Sports Traumatol Arthrosc. 2016;24(9):2874–2877. doi:10.1007/s00167-015-3537-4.
25. Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting Reliability
and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96–106. doi:10.1016/j.jclinepi.2010.03.002.
26. Palisano RJ, Cameron D, Rosenbaum PL, Walter SD, Russell D. Stability of the Gross Motor Function Classification System. Dev Med Child Neurol. 2007;48(6):424–428. doi:10.1111/j.1469-8749.2006.tb01290.x.
27. Norkin CC, White DJ. Measurement of Joint Motion: A Guide to Goniometry. 4th ed. Philadelphia, PA: FA Davis; 2009.
28. de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine. 1st ed. Cambridge: Cambridge University Press; 2011.
29. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. Vol 3. Upper Saddle River, NJ: Pearson/Prentice Hall; 2009.
30. de Vet HCW, Terwee CB, Knol DL, Bouter LM. When to use agreement versus reliability
measures. J Clin Epidemiol. 2006;59(10):1033–1039. doi:10.1016/j.jclinepi.2005.10.015.
31. Weir JP. Quantifying test-retest reliability
using the intraclass correlation coefficient and the SEM. Natl Strength Cond Assoc. 2015;19(1):231–240. doi:10.1519/15184.1.
32. Lindén O, Hägglund G, Rodby-Bousquet E, Wagner P. The development of spasticity with age in 4,162 children with cerebral palsy
: a register-based prospective cohort study. Acta Orthop. 2019;90(3):286–291. doi:10.1080/17453674.2019.1590769.
33. Størvold GV, Jahnsen RB, Evensen KAI, Romild UK, Bratberg GH. Factors associated with enhanced gross motor progress in children with cerebral palsy
: a register-based study. Phys Occup Ther Pediatr. 2018;2638(May):1–14. doi:10.1080/01942638.2018.1462288.
34. Marcström A, Hägglund G, Alriksson-Schmidt AI. Hip pain in children with cerebral palsy
: a population-based registry study of risk factors. BMC Musculoskelet Disord. 2019;20(1):62. doi:10.1186/s12891-019-2449-8.