Congenital muscular torticollis (CMT) is a musculoskeletal deformity observed at birth or in infancy characterized by unilateral contracture of the sternocleidomastoid (SCM) muscle.1,2
It is the third most common congenital musculoskeletal disorder after congenital hip dislocation and clubfoot.2 Clinical presentation consists of persistent head tilt toward the involved side, with the chin rotated toward the contralateral shoulder.2,3 Other structures often shortened in addition to the involved SCM muscle include surrounding musculature, neurovascular bundle, and cervicothoracic fascia.3,4 In severe cases, asymmetry of facial structures and cranial bones may occur.5,6 Since introduction of the Back to Sleep Program by the American Academy of Pediatrics in 1992, the incidence of CMT in infants and young children has risen and is reported as high as 16%.7,8
There are 3 categories of CMT clinical presentation: infants with SCM tumor (SMT), muscular torticollis (MT), and postural torticollis (POST).3,9 Infants in the SMT group have palpable intramuscular fibrotic masses within the affected SCM muscle and cervical passive and active range of motion (ROM) limitations.3 Muscular torticollis is defined as the presence of limitations in passive cervical ROM due to the tightness in the SCM without a mass.3 Infants with POST have a clinical presentation of CMT without the presence of a mass or passive ROM limitations due to cervical muscle strength imbalance.3,9
The majority of infants and children with CMT achieve good to excellent outcomes with early conservative physical therapy.1,9–22 As infants get older, the treatment duration increases.10–12,15 Infants referred to physical therapy before 1 month of age typically have the shortest treatment durations (<1.5 months).10 For infants 1 to 3 months of age, treatment duration almost quadruples to 6 months, and infants who start treatment after 6 months of age may require as many as 10 months of physical therapy.10
The prognosis for full resolution of CMT and the length of intervention vary depending on the age at onset of intervention, severity of ROM restrictions, and presence of a mass in the SCM muscle.1,13–22 In addition, prognosis is related to the severity of fibrotic changes in the affected SCM muscle, with longer treatment duration correlated with severe fibrosis.22–26 Infants beginning therapeutic intervention prior to 3 months of age have excellent outcomes, with almost no infant requiring surgery.1,10–12 Newer evidence indicates that infants with a thicker SCM and a history of breech presentation, oligohydramnios, or lower birth weight have a longer treatment duration regardless of age of referral.19,27 Surgical intervention is necessary in a small percentages of infants who typically initiated therapy after 3 months of age and have high severity of CMT.1,9,12,14,22,24
Historically, infants with CMT are classified based on several taxonomies. The classification included age at the time of evaluation, severity of cervical ROM limitations, type of CMT (POST, MT, and SMT), presence or absence of plagiocephaly, and severity of fibrosis within the SCM muscle on ultrasonographic and sonoelastographic examination.1,3,4,7–9,12–21 Clinical outcomes are typically analyzed according to the age of presentation of CMT, difference in passive cervical ROM, and/or type of CMT.
In a 2013 clinical practice guideline28 (CPG), a CMT severity classification system (CMT-SCS) was proposed based on the child's age at the time of referral, cervical rotation passive ROM restrictions, and the presence or absence of an SCM mass. The CMT-SCS is first dichotomized as “early” or “late” based on age at the time of referral, with severity ranges within each category. Seven grades of severity are operationally defined (Figure). The guideline development group recommended using the CMT-SCS to classify infants with CMT into these 7 severity grades in future intervention studies.28
The CMT-SCS has undergone content validation through repeated expert review. Rater reliability has not been reported. Establishing interrater and intrarater reliability of the CMT-SCS is important to ensure that the clinicians and researchers are using the scale consistently. After the reliability is established, the CMT-SCS can be used as the standard of care by professionals working with infants with CMT. The purpose of this study was to establish intra- and interrater reliability for determining severity grades of CMT in infants.
A repeated-measures prospective reliability study was conducted, consisting of 4 phases. Phase 1 established reliability of the primary investigator (PI). Phases 2 and 3 tested interrater and intrarater reliability of practicing physical therapists. Phase 4 was a brief survey on reference values used by raters when calculating cervical ROM difference.
Eligible participants were pediatric physical therapists without restrictions to experience or practice setting. Clinicians volunteered to be contacted through an open recruitment in the Academy of Pediatric Physical Therapy (APPT) newsletter and an e-mail blast. Participants provided informed consent by agreeing to a statement prefacing the online survey. This study was approved by the Institutional Review Boards of the Hospital for Special Surgery, New York, and Rocky Mountain University of Health Professions, Utah.
All patient cases were randomly chosen from the CMT database of clinical records of infants with CMT managed at the PI's institution in 2010. There were 24 cases; 3 grade 1, 7 grade 2, 6 grade 3, 2 each for grades 4, 5, and 6, 1 for grade 7, and 1 atypical case that did not fit any of the 7 grades because the child was older than 12 months and had a cervical ROM limitation of less than 30°. The atypical case was a 16-month-old with right head tilt, no side-to-side difference in cervical rotation passive range of motion (PROM), and a 20-° difference between left and right passive cervical lateral flexion with no SCM mass present. It was included intentionally to illustrate the limitations of the scale and test clarity of the grade definitions.
Phase 1: Reliability of the PI
The reliability of the PI was established. Ten deidentified records, including the patient's age, presence or absence of mass in the SCM muscle, and passive cervical ROM data, were randomly chosen from the CMT database. Based on data obtained from the initial assessments, these 10 records were graded using the CMT-SCS by the PI and a second pediatric physical therapist with 28 years of pediatric experience. Answers from both raters were compared for level of agreement.
Phase 2: Interrater Reliability
SurveyMonkey, an online survey program, was used to record severity ratings from volunteer clinicians. Clinicians were provided a decision tree from the CPG and a table describing each grade of CMT with each of the 24 infant case descriptions (Figure). The deidentified clinical information included age at assessment, side of head tilt, cervical passive rotation and lateral flexion ROM, and presence or absence of an SCM mass. Patient cases were randomly chosen from the CMT database generated from clinical records of infants with CMT managed at the PI's institution in 2010.
In addition, raters completed a short survey about demographic data, level of experience, and caseload volume of infants with CMT. Raters also answered 2 questions about reading and using the APPT CPG28 in daily clinical practice.
Phase 3: Intrarater Reliability
For intrarater reliability, raters in phase 2 consented to repeat the classification ratings at least 2 weeks after completing the first ratings. The cases were randomly reordered, and raters repeated the classification ratings through a second SurveyMonkey link. Again, raters were provided the table describing each grade of CMT and the decision tree (Figure).
Phase 4: Postanalysis Survey
During the analysis of phase 2 data, it was suspected that participants were using different reference values for calculating cervical rotation ROM limitations when scoring cases with bilateral limitations of cervical passive rotation. In these cases, the grading among raters was almost equally divided between 2 neighboring severity grades within “early” or “late” categories. It appeared that the difference between sides was determined by subtracting cervical rotation ROM on the involved side from either 90 or 110° as normative values.
To clarify how participants were calculating the ROM deficit, phase 4 was added with a brief survey. Raters were asked to answer a question about the reference value used when scoring cases. The survey was sent to the 131 therapists who shared their e-mail addresses with the PI.
Distribution percentages of rater demographics were calculated for years of experience as a physical therapist, years of experience treating pediatric patients, years treating infants with CMT, percentage of children in caseload, and number of infants with CMT treated weekly. The classification levels are represented as 7 categories with a specific order or as a bounded ordinal scale.29 A distance metric was employed to enable the use of intraclass correlation coefficients (ICCs) as described by Shrout and Fleiss.30 We assumed a continuous true value that underlies the scale to adapt the ICC model and employ a 2-way random effect model to determine the ICC with absolute agreement. Specifically, ICC estimates and 95% confidence intervals (CIs) were calculated using SPSS statistical package version 19 (SPSS Inc, Chicago, Illinois) based on a single rater (κ = 1), absolute agreement, 2-way mixed-effects model (ICC 2,1).
Phase 1: Reliability of PI
The author and second rater initially achieved a 90% interrater agreement yielding a κ = 0.872 (95% CI, 0.64-1.10; P < .001). The one case of nonagreement was an 11 month-old infant presenting with 2° difference of passive cervical rotation between the involved and uninvolved sides and no mass in the SCM. One of the therapists assigned a grade 6 (late severe), missing the fact that the infant presented without a mass, with the correct grade being 5 (late moderate). After further exploration and clarification on a case yielding different scores, another 10 records were graded by the same raters to repeat this analysis; 100% agreement resulted in a κ = 1.0 (P < .001) indicating perfect agreement.
Study Sample. A total of 218 raters responded to the phase 2 e-mail blast and newsletter posting, with 145 completing the rating of the 24 cases, yielding a 66.5% participation rate. Of 145 participants, 141 completed the demographic survey questions. Raters were diverse in practice settings, with therapists practicing in early intervention, hospital, outpatient, school, and home settings (Table 1, A). Raters ranged from new graduates (0.7%) to more than 20 years of experience (32%); however, the raters' experience levels were skewed toward high experience, with 75% of raters having 7 or more years of clinical experience and 83% of raters treating only pediatric patients. Forty-five percent of raters specifically treated infants with CMT and had between 1 to 6 babies with CMT on their caseloads, while 31% of raters reported no current infant cases at the time of the survey (Table 1, B).
Raters were asked about their familiarity with the APPT CPG and whether they used the CMT-SCS in their daily practice. Although 85.4% of raters were familiar with the APPT CPG, only 24.5% reported assigning severity grades in their practice.
Phase 2: Interrater Reliability
The atypical case was excluded from the ICC analysis, as it was intended to be a “trick question” for raters, and analyzed descriptively. More than half of the respondents (54.5%) answered correctly by not assigning a specific grade, while 35.9% of raters marked it as grade 7, 6.9% of raters as grade 5, and singular raters assigned grades 3, 4, or 6.
Interrater reliability was calculated for the 23 cases for phase 2 (ICC (2,1) = 0.82, with 95% CI, 0.73-0.90) and phase 3 (ICC (2,1) = 0.84, with 95% CI, 0.76-0.91). Both ICCs were averaged for an overall interrater reliability ICC (2,1) = 0.83 (95% CI, 0.74-0.91) (Table 2). Koo's definitions of reliability based on the 95% CI of the ICC estimate were used to interpret the CMT-SCS's reliability: values between 0.5 and 0.75 indicate moderate reliability; values between 0.75 and 0.9 indicate good reliability, and values above 0.9 indicate excellent reliability.31 The interrater reliability interpretation, based on the 95% CI, is moderate to excellent.31 It is important to note that the lower confidence limit is only 0.01 below the cutoff point for good reliability, as is the upper confidence level at only 0.01 above cutoff point for excellent reliability. It can be conservatively concluded that the overall interrater reliability, based on the ICC and CI values, is good.
Phase 3: Intrarater Reliability
After completing phase 2 rating, 131 raters shared e-mail consent with the PI for phase 3 participation. In phase 3, a link to the SurveyMonkey program with the same 24 reordered cases was e-mailed to 131 raters; 77 raters completed grading the 24 cases in both phases to support the intrarater reliability analysis.
The intrarater reliability between phases 2 and 3, based on the 23 valid cases, resulted in an ICC (3,1) = 0.81 (95% CI, 0.66-0.91), which indicates moderate to excellent reliability31 (Table 2). The ICC value for interrater reliability was higher than intrarater reliability, but the difference was small (0.02), with similar CIs: the lower confidence limit for interrater reliability was 0.08 larger than the intrarater level, and upper confidence limits were identical in both types of reliability.
Phase 4: Postanalysis Survey
The survey was sent to the 131 therapists who shared their e-mail addresses with the PI. Of the 76 responders who answered the question about the reference values, 52.6% of raters subtracted ROM measurements on the involved side from an expected 90° of cervical rotation, 39.4% of raters subtracted from the measurements on the contralateral side (the method described by CPG), and 5.2% subtracted from 110° of cervical rotation.
Exploratory Analysis. Secondary analyses were used to determine whether the severity grade assessment accuracy was affected when stratifying raters into groups based on the raters' years of pediatric physical therapy experience or caseload size of infants with CMT. The ICC values were not substantially different across groups with different years of experience (ranging from 0.77 to 0.86), suggesting that raters' experience is not likely to be a factor in grading severity of CMT (Table 3, A). The ICC values had a more substantial variation across groups based on raters' caseload size (ranging from 0.68 to 0.85). Specifically, those physical therapists with less than 1 CMT case per week were less reliable in assigning CMT severity grades (Table 3, B).
Regression analyses were performed to evaluate the potential for a relationship between raters' grading accuracy (% correct classification) and raters' years of pediatric physical therapy experience, or raters' grading accuracy and the caseload size of infants with CMT. Correlations between the rater's grading accuracy and experience was R = −0.031 and between grading accuracy and caseload size R= 0.112 (Table 3, C).
To test the hypothesis that CMT severity grading was affected by caseload size of infants with CMT or by years of pediatric physical therapy experience, 1-way factorial analyses of variance (ANOVA) were performed for each factor on the percentage of correct answers per clinician. Size of CMT caseload was divided into 3 groups: no infants with CMT per week, 1 to 3 infants per week, and more than 4 infants per week; there were no significant differences between the groups (P = .091) (Table 3, C). In the ANOVA for pediatric experience, 4 groups were created: 0 to 3, 4 to 10, 11 to 20, and 21 years and more. There was a significant difference between the groups based on raters' years of pediatric physical therapy experience (P = .049) (Table 3, C); however, after post hoc analysis with adjustment for multiple comparisons, there was no significant difference (α = 0.005). The hypothesis that CMT assessment accuracy is affected by physical therapy experience or caseload was rejected.
A standardized classification system for categorizing levels of severity for infants and children with CMT was provided in the APPT CMT-CPG.28 The CMT-SCS relies on 3 clinical factors routinely obtained by therapists during an initial assessment: age at which therapy is initiated, difference in passive cervical rotation ROM measurements, and presence or absence of an SCM mass. These 3 clinical factors have been strongly correlated with outcomes and duration of treatment.28
This is the first report on interrater and intrarater reliability of the CMT-SCS. The results of the study support that the reference value used for calculating the difference in passive cervical rotation ROM difference was insufficiently emphasized in the APPT CPG. Raters used different methods to arrive at the amount of cervical muscle tightness: some applied the difference between involved and contralateral sides; some subtracted cervical ROM on the involved side from 90° and others from 110° as normal reference values for cervical passive ROM in infants. Instructions to determine severity level of CMT were included in the CPG Action Statement 7 describing the technique to measure cervical PROM, but the method was not reiterated under Action Statement 8, which defined severity grades in detail. Therapists familiar with measurements of an infant's cervical ROM may easily not read Action Statement 7 when reviewing the CPG. Physical therapists are accustomed to using normal reference values to determine limitations in PROM; however, this scale requires therapists to consider the difference between the left and right measures of cervical ROM. As 1 of 3 components required to assign the severity grade, detailed instructions for the calculation of cervical muscle tightness should be placed in future revisions of the APPT CPG in the section describing the grading system.
Some raters were not attentive to all 3 factors influencing the grade. Clinicians should pay close attention to all 3 factors, as missing 1 will likely result in an inaccurate grade. For example, the presence of an SCM mass directly classifies an infant as severe in the younger group and as extreme in the older age group. Omitting information about the presence of an SCM mass inaccurately places a child in a less severe grade category (Figure, A, for reasoning pathways). A therapist's ability to palpate the SCM muscle and accurately identify the presence or absence of a fibrotic mass may affect rating accuracy.
Using the electronic medical record to calculate of the difference in cervical ROM will eliminate mathematical errors. When subtracting cervical rotation PROM measurements to assign severity grades, human error may make a difference between assigning a correct versus incorrect grade. Using electronic medical record systems' capability to calculate cervical ROM difference to assign a severity grade and more clearly defining the technique to calculate cervical rotation ROM tightness in the CPG will further improve reliability of the severity scale.
Additional development of the CMT-SCS is recommended to add a severity grade for children older than 12 months. Despite efforts to educate the medical community and parents on the importance of early identification and referral, children are still referred for their initial CMT physical therapy evaluation after 12 months of age. The current CMT-SCS has no option to assign a severity grade to children older than 12 months with cervical rotation ROM limitations less than 30°.
Until now, infants with CMT have been classified by 5 taxonomies,28 making it difficult to communicate among clinicians and researchers and to compare outcomes of various interventions due to a lack of common terminology. Assigning a severity grade using CMT-SCS will aid clinicians in determining a child's prognosis and designing an evidence-based treatment plan, including frequency of therapy, frequency of daily stretches prescribed in the home program, and estimated duration of overall treatment. In addition, clinicians will be able to compare their patients with CMT to other clinicians' patients, to patients reported in case studies, and to patients described in research articles. Using the CMT-SCS in the standard of care will provide common terms for medical professionals, including physicians, when discussing infants with CMT, similar to the Gross Motor Function Classification System that is used widely by the medical community when describing children with cerebral palsy.32 With the current development of a national CMT outcome registry by the American Physical Therapy Association, standardized classification levels used by all clinicians contributing to the registry will aid in collecting standardized clinical information and future analysis of data. Ultimately, wide use of the CMT-SCS may assist physical therapists to educate and empower families of children with CMT by using an evidence- based pathway to recovery according to severity grade.
The first survey's results illustrated that, although 85.4% of respondents were familiar with the APPT CPG, only 24.5% assign severity grades in their regular practice. Because the recruitment was conducted through the APPT newsletter and an e-mail blast, it can be assumed that most participating therapists were engaged members of the APPT. They, therefore, may not be representative of the general population of therapists treating infants with CMT, and the number of therapists familiar with CPG and assigning severity grades may be lower nationwide. To achieve the goals of language standardization by all clinicians treating infants with CMT, the CMT-SCS scale should be considered as a practice standard. Clinicians assessing and treating children with CMT globally should assign severity levels during the initial assessment. Because numerous research articles on CMT are conducted in Europe and Asia, global education and outreach to promote the CMT-SCS are needed.
One potential limitation of the study was that raters may have recorded the assigned severity grades after the first rating and saved them for use during the second rating to improve their performance. To minimize this possibility, cases were randomly reordered.
A second limitation was that the survey was directed toward pediatric physical therapists only and may not be generalizable to the overall population of physical therapists. In practice, infants with CMT may be seen by orthopedic physical therapists in private practice settings where access to pediatric therapists is limited.
The ability of the raters to assign correct severity grades based on the independent assessment in a clinical setting was not tested because they were provided with clinical case information necessary to assign the severity grade. Therapists' skills to palpate and identify fibrotic masses within SCM muscles and to obtain reliable cervical PROM measurements were not addressed.
Development of reliable measurement techniques for passive cervical lateral flexion in infants and children is needed. Raters in this study repeatedly commented that the lack of inclusion of cervical lateral flexion PROM was one of the limitations of the scale and identified cervical lateral flexion PROM as a high priority area to include in the CMT-SCS. Future research on reliable measurement techniques for cervical lateral flexion PROM in infants and children and investigation of effects of cervical lateral flexion limitations on treatment duration are needed.
Research is also needed on the relationship between various severity grades of CMT and effects of different frequency models of interventions. Longitudinal studies on long-term outcomes for infants with CMT grouped according to severity grades could identify valuable prognostic indicators.
The CMT-SCS has good reliability for infants up to 12 months of age, permitting clinicians to use the scale during initial assessment of infants with suspected CMT. Use of the CMT-SCS provides medical professionals, researchers, and caregivers with a specific, standardized language of severity when discussing patients, and should assist in conversations about clinical presentation, treatment planning, treatment duration, and research design. Continued efforts are needed to educate and to promote the widespread use of the CMT-SCS as a documentation standard for classification.
1. Emery C. The determinants of treatment duration for congenital muscular torticollis
. Phys Ther. 1994;74:921–929
2. Karmel-Ross K. Congenital muscular torticollis
. In: Campbell SK, ed. Physical Therapy for Children. 4th ed. St Louis, MO: Elsevier; 2012:359–380.
3. Kaplan S, Sargent B, Coulter C. Congenital muscular torticollis
. In: Palisano RJ, ed. Campbell's Physical Therapy for Children. 5th ed. St Louis, MO: Elsevier; 2017:184–206.
4. Karmel-Ross K, Lepp M. Assessment and intervention of children with congenital muscular torticollis
. Phys Occup Ther 1997;17:21–68.
5. Hollier L, Kim J, Grayson BH, McCarthy JG. Congenital muscular torticollis
and the associated craniofacial changes. Plast Reconstr Surg. 2000;105(3):827–835.
6. van Vlimmeren LA, van der Graaf Y, Boere-Boonekamp MM, et al Effect of pediatric physical therapy on deformational plagiocephaly in children with positional preference: a randomized controlled trial. Arch Pediatr Adolesc Med. 2008;162:712–718.
7. Stellwagen LM, Hubbard E, Chambers C, Lyons KL. Torticollis
, facial asymmetry and plagiocephaly in normal newborns. Arch Dis Child. 2008;93:827–831.
8. de Chalain TM, Park S. Torticollis
associated with positional plagiocephaly: a growing epidemic. J Craniofac Surg. 2005;16:411–418.
9. Cheng JC, Wong MW, Tang SP, et al Clinical determinants of the outcome of manual stretching in the treatment of congenital muscular torticollis
. A prospective study of eight hundred and twenty-one cases. J Bone Joint Surg Am. 2001;83-A:679–687.
10. Petronic I, Brdar R, Cirovic D, et al Congenital muscular torticollis
in children: distribution, intervention duration and outcome. Eur J Phys Rehabil Med. 2009;45(1):6.
11. Cameron BH, Langer JC, Cameron GS. Success of nonoperative treatment for congenital muscular torticollis
is dependent on early therapy. J Pediatr Surg. 1994;9:391–393.
12. Cheng JC, Tang SP, Chen TM, Wong MW, Wong EM. The clinical presentation and outcome of treatment of congenital muscular torticollis
—a study of 1,086 cases. J Pediatr Surg. 2000;35:1091–1096.
13. Cheng JC, Tang SP, Chen TM. Sternocleidomastoid pseudotumor and congenital muscular torticollis
: a prospective study of 510 cases. J Pediatr. 1999;134:712–716.
14. Demirbilek S, Atayurt HF. Congenital muscular torticollis
and sternomastoid tumor: results of nonoperative treatment. J Pediatr Surg. 1999;34(4):549–551.
15. Tatli B, Aydinli N, Caliskan M, Ozmen M, Bilir F, Acar G. Congenital muscular torticollis
: evaluation and classification. Pediatr Neurol. 2006;34:41–44.
16. Taylor JL, Stamos Norton E. Developmental muscular torticollis
: outcomes in young children treated by physical therapy. Pediatr Phys Ther. 1997;9:173–178.
17. Carenzio G, Carlisi E, Morani I, et al Early rehabilitation treatment in newborns with congenital muscular torticollis
. Eur J Phys Rehabil Med. 2015;51:539–545.
18. Lee JY, Koh SE, Lee IS, et al The cervical range of motion as a factor affecting outcome in patients with congenital muscular torticollis
. Ann Rehabil Med. 2013;37(2):183–190.
19. Jung AY, Kang EY, Lee SH, et al Factors that affect the rehabilitation duration in patients with CMT. Ann Rehabil Med. 2015;39(1):18–24.
20. Ryu JH, Kim DW, Kim SH, et al Factors correlating outcome in young infants
with congenital muscular torticollis
. Can Assoc Radiol J. 2016;67:82–87.
21. Lee K, Chung E, Koh S, Lee BH. Outcomes of asymmetry in infants
with CMT. J Phys Ther Sci. 2015;27(2):461–464.
22. Lee YT, Yoon K, Kim YB, et al Clinical features and outcome of physiotherapy in early presenting congenital muscular torticollis
with severe fibrosis on ultrasonography: a prospective study. J Pediatr Surg. 2011;46:1526–1531.
23. Wang L, Zhang L, Tang Y, Qui L. The value of high-frequency and color Doppler ultrasonography in diagnosing congenital muscular torticollis
. BMC Musculoskelet Disord. 2012,13:209.
24. Cheng JC, Metreweli C, Chen TM, Tang S. Correlation of ultrasonographic imaging of congenital muscular torticollis
with clinical assessment in infants
. Ultrasound Med Biol. 2000;26:1237–1241.
25. Kwon DR, Park GY. Diagnostic value of real-time sonoelastography in congenital muscular torticollis
. J Ultrasound Med. 2012;31:721–727
26. Hu CF, Fu TC, Chen CY, et al Longitudinal follow-up of muscle echotexture in infants
with congenital muscular torticollis
. Medicine. 2017;96(6):1–6.
27. Lee YT, Cho SK, Yoon K, et al Risk factors for intrauterine constraint are associated with ultrasonographically detected severe fibrosis in early congenital muscular torticollis
. J Pediatr Surg. 2011;46:514–519.
28. Kaplan SL, Coulter C, Fetters L. Physical therapy management of congenital muscular torticollis
: an evidence-based clinical practice guideline. Pediatr Phys Ther. 2013;25:348–394.
29. de Mast J, van Wieringer W. Measurement system analysis for bounded ordinal data. Qual Reliab Engng Int. 2004;20:383–395.
30. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
31. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–163.
32. Palisano R, Rosenbaum P, Walter S, et al Development and reliability of a system to classify gross motor function in children with cerebral palsy. Dev Med Child Neurol. 1997;39:214–223.