Nonequilibrium (limb) coordination testing is a standard part of the neurologic examination1,2 and serves to assess movement capabilities that encompass coordinated movement. These capabilities include rapid alternating movement, accuracy in moving to a target, movement efficiency, and position holding.3 The most commonly utilized limb coordination tests appear to be tapping of the upper4–7 or lower5,7,8 extremities, finger-to-nose,8–10 and a test involving hip and knee motion guiding the foot to a target.3,11,12 Movement can be assessed for quality,13 speed,8,10,12 or a combination of both in a semiquantitative measure.9,11 While clinicians may prefer to use certain tests more than others,1,13 test selection should be based largely on their validity, reliability, and responsiveness in the tested population.
Although coordination testing is a basic component of a neurologic examination, there appear to be no clear guidelines as to which tests should be used in the acute neurologic setting. When the diagnosis is known, an examiner may choose tests or test batteries validated for use in persons with the given diagnosis or movement disorder. For example, the finger-to-nose test has been used to identify coordination deficits in persons with multiple sclerosis,14 essential tremor,13 Parkinson disease,15 and cerebellar dysfunction.11 More commonly, tests are combined into assessment scales and validated for use in patients with specific disorders. Four such scales with their respective limb coordination tests and validated diagnoses are shown in Table 1. In the acute neurologic setting, however, patients may be evaluated before having a clear diagnosis. In addition, they may have coordination deficits due to pathologies for which no tests have been validated. Therefore, choosing tests based on diagnosis alone can be difficult in the acute care setting.
Acute care presents an additional challenge as patients are followed typically by health care practitioners with diverse backgrounds and varying levels of experience. Tests chosen for coordination testing in this setting must be reliably administered by these practitioners to avoid the appearance of fluctuating patient performance and medical status. Of the test batteries noted in Table 1, the limb coordination section of the National Institutes of Health Stroke Scale and the Fahn-Tolosa-Marin tremor rating scale demonstrated poor26 and moderate27 reliability, respectively, when scores were compared between different members of the health care team. The interrater reliability of the various individual tests available to the clinician has never been assessed between different health care practitioners in the general acute neurologic setting.
The purpose of this study was to investigate the interrater reliability and known-group validity of limb coordination tests in participants with acute central nervous system (CNS) pathology when performance was assessed by raters of different training backgrounds. To our knowledge, the tests used have not been individually assessed, in this setting, for these properties. Results may identify those appropriate for use in individuals with acute neurologic pathology.
Potential subjects were inpatients at a tertiary hospital (1200+ beds), diagnosed with acute CNS pathology, under the care of a neurologist or neurosurgeon, and between 18 and 79 years of age. They could follow multistep commands and had adequate limb motion and strength to complete test procedures. At the time of data collection, patient disease course had to be stable and coordination status unchanged from admission. Patients with pain or limb amputations were ineligible. Because of their increased variability in motor performance, persons older than 80 years were excluded to prevent typical age-related coordination decline from confounding results due to deficits of neurologic origin.28 This study was approved by the Mayo Clinic institutional review board.
Sample size was determined with an a priori power analysis based on assumptions that 3 raters would score test performance, that the significance level (α) for minimizing type I error was 0.05, and that the desired power of statistical testing was 0.80. Determining whether an intraclass correlation coefficient (ICC) of 0.80 or higher was significantly higher than a minimum accepted reliability coefficient of 0.50 required a minimum sample size of 15 subjects. To conservatively exceed a statistical power of 0.80, 25 subjects were recruited.
The medical records of potential subjects consecutively admitted to the neurology service were screened for eligibility criteria by coauthors who did not serve as raters. Eligible subjects who agreed to participate provided informed consent. They were categorized by nonrater coauthors as either having or not having coordination deficits according to the initial examination of the attending physician (neurologist or neurosurgeon). Patients with deficits were categorized as such if coordination performance during the examination was reported as “abnormal,” or if descriptors of abnormal coordination were used (ataxic, slow, tremulous). Patients without coordination deficits were those deemed as having “normal” or “intact” motor and/or coordination on the initial examination.
Participants completed the following 20 nonequilibrium coordination tests, which were administered (in the listed order) at bedside by the first author:
Fixation/position holding (upper extremity [UE])
Pointing and past-pointing
Rebound phenomenon (UE)
Precision finger tap
Draw a circle (hand)
Fixation/position holding (lower extremity [LE])
Rebound phenomenon (LE)
Draw a circle (foot)
The same test order was utilized for each participant to simplify the testing procedure (for test descriptions, method of assessment, and associated references, see the Supplemental Digital Content, http://links.lww.com/JNPT/A28). Tests were presented to address proximal-to-distal extremity movement in the upper extremities, followed by the lower extremities, allowing participants to focus on one area at a time to minimize physical and cognitive fatigue. Each test was performed under 2 conditions: eyes-opened and eyes-closed. While the eyes-opened condition can demonstrate impaired movement from deficits in multiple neurologic areas, such as the upper motor neuron (UMN), cerebellum, or basal ganglia,1 the eyes-closed condition specifically targets somatosensory and vestibular deficits.29 The tests chosen for the current study have been cited in the literature3,6,11,19,22,24,30–34 either individually or as part of a test battery but have not been assessed for reliability and validity in a general acute neurologic population. They assess movement capabilities that comprise normal coordination and detect abnormalities such as dysmetria, dyskinesia, or tremor.
Testing was completed in the supine position to standardize the testing procedure in a way that allowed subjects who could not tolerate sitting or mobilize safely out of bed to participate. The head of the bed was elevated approximately 45° so that participants were comfortable and had a full view of their limbs. Each test was administered on the dominant and nondominant sides with eyes opened and eyes closed. While the side assessed first was randomized, the eyes-opened condition always preceded eyes-closed to ensure that participants understood the test procedure. Standardized verbal instruction and demonstration of each test were provided. Subjects were told to complete each test as quickly and accurately as possible. Timed tests were practiced for 5 consecutive repetitions as a “practice trial” before the actual “test trial” was measured, since performance becomes faster during a second trial but does not improve further on a third trial.10 Subjects rested between tests as needed. Total test time varied among participants from 17 to 45 minutes.
Coordination examinations were videotaped and transferred to DVD format. Participant performance was independently viewed and scored by 3 raters with different levels of experience and expertise: a neurologist with 12 years of experience, a physical therapist with 38 years of experience, and a second-year physical therapy student. Raters were not involved in the admission or hospital care of participants and were blinded to participant diagnosis and group designation (with or without deficits). They were provided with coordination test descriptions and a 5-point ordinal rating scale (Table 2). They were instructed to independently watch the DVDs, beginning with participant 1 and ending with participant 25, scoring participant performance on each of the 20 coordination tests using the ordinal scale. Raters received no formal training on its use, similar to a previous report in which 3 raters scored 22 participant video examinations, using an ataxia rating scale.35 With a stopwatch, the raters timed 9 of the 20 tests by measuring 5 successive repetitions.
Data were analyzed using SPSS 15.0 for Windows software (SPSS Inc, Chicago, Illinois). For all tests, interrater reliability was assessed across the 3 raters and by pairwise comparison to determine whether one pair of raters had higher reliability than another. Interrater reliability for timed test performance was estimated with ICC (model 2). For tests scored with the ordinal scale, the free-marginal kappa coefficient was used to determine interrater reliability among the 3 raters,36,37 while quadratic-weighed kappa coefficients were used for paired comparisons. Known-group validity, in which performance on the tests was compared between subjects categorized as either having or not having coordination deficits according to the attending physician's initial examination, was assessed using the nonparametric Mann-Whitney U test for ordinal test results. Independent t tests were used to compare group differences on timed measures of performance (α ≤ 0.05). The scores assigned by the rater who was a neurologist were selected by chance to conduct the known-group validity analysis.
We used Kolmogorov-Smirnov tests to examine whether the timed measures of coordination performance were normally distributed. Only 1 of the 72 different distributions tested for normality differed significantly from normal (left foot tapping, eyes closed, in the impaired coordination group, Kolmogorov-Smirnov Z = 1.404, P = 0.039). Therefore, we treated all of them as being normally distributed, justifying the use of parametric statistics to analyze interrater reliability and group differences for timed performance.
One examiner did not provide ratings on 4 of the tests for a single subject. To handle the missing data, scores were imputed as the mathematical mean score of the other 2 examiners. In the finger-to-finger test, data from 1 subject were missing from 2 examiners. In that case, we did not mathematically impute scores but instead excluded the subject from the interrater reliability analysis.
Of 250 potential subjects, 65 met the eligibility criteria. Of those, 26 were discharged before they could be consented, 4 were too sick (nausea/vomiting) to participate, 3 were in isolation and could not participate, and 7 refused. The remaining 25 subjects participated in the study. Twelve were categorized as having coordination deficits (mean age 58 ± 7 years) and 13 were identified as not having deficits (mean age 53 ± 17 years).
Participant characteristics for gender, age, hand dominance, hemibody side of deficits (if applicable), lesion location, and diagnosis are given in Table 3. Nineteen participants were diagnosed with a vascular condition: 10 stroke, 1 posterior reversible encephalopathy syndrome from hypertension, 1 arterial dissection, 1 subdural hematoma, 2 subarachnoid hemorrhage (1 of those with a subsequent hematoma), 1 artero-venous malformation with intracranial hemorrhage, 2 with intracranial hemorrhage only, and 1 transient ischemic attack. The remaining 6 participants had intracranial tumors. In those with clearly defined intraparenchymal pathology, the cerebral cortex was involved in 14 cases, the cerebellum in 5, brainstem in 2, and spinal cord in 1. In the deficit group, 5 participants demonstrated bilateral involvement, 6 were left-side involved, and 1 right-side involved.
Pairwise comparisons demonstrated that no pair of raters had higher reliability than another. The mean reliability coefficients across the 3 raters for each of the 20 coordination tests are listed in Table 4. With the exception of finger opposition and draw a circle (foot), tests scored ordinally had mean kappa coefficients that suggested at least substantial agreement (>0.60)38 among raters. Scores for finger-to-nose, finger-to-tester's-finger, and fixation/position holding UE demonstrated excellent interrater reliability (mean kappa > 0.80).38 Mean reliability coefficients for the timed tests, with the exception of tapping (hand), were greater than 0.75, suggesting “good” reliability.39 The coefficients for 5 timed tests (alternate finger-to-nose, pronation/supination, mass grasp, alternate heel-to-knee/ankle, and heel-on-shin) were greater than the 0.90 level suggested for clinical measures.39
Mean test scores and levels of significance for differences in scores between participants in the deficit versus nondeficit groups are given in Table 5 for timed measures and Table 6 for ordinal measures. Tests in which performance significantly differed between groups (P ≤ 0.05) for at least 1 condition (right or left side, eyes opened, or eyes closed) included 5 of the 9 timed measures and 8 of the 20 ordinal measures. Of these 13 tests, 6 revealed differences in performance with both eyes opened and eyes closed, 5 only with the eyes-opened condition, and 2 only with eyes-closed. Significant differences were usually found when comparing left-side performance.
We investigated the interrater reliability and known-group validity of multiple nonequilibrium coordination tests in participants with acute CNS pathology. Overall, timed test scores had higher levels of agreement than ordinal scores. Performance on 5 of the 9 timed tests met the highest level of agreement (ICC > 0.90)39 compared with only 3 of the 20 ordinal measures (kappa > 0.80).38 Our findings support a study by Swaine and Sullivan,40 who examined physical therapist interrater reliability when assessing patients with brain injury using videotape analysis. In their study, timed alternate finger-to-nose ICC values exceeded 0.91 whereas mean kappa values ranged from 0.18 to 0.54.40 In our experience, ordinal scales such as the one in our study are used typically without formal instruction, a limitation in qualitative assessment. Education about a rating scale has been shown to improve its reliability among health care practitioners whose experience and backgrounds vary.41 Thus, formal training of an ordinal scoring system should be implemented to improve interrater agreement.
Beyond higher levels of agreement, data from timed tests were also more likely than data from ordinal measures to discriminate between normal and impaired performance. Known-group validity was demonstrated from performance on 55% of the timed measures compared with 40% of the ordinal measures. Previous studies have shown that timed motor performance can distinguish between healthy individuals and age-matched individuals diagnosed with Alzheimer disease,42 Parkinson disease,4 essential tremor,43 or multiple sclerosis.44 Participants in the current study did not have these conditions.
With one exception, all participants with deficits had impairment of the left side, explaining why left-side performance typically differentiated subject groups. If performance differed between groups for a test, the eyes-opened condition more often than eyes-closed condition revealed that difference, suggesting that vision could not compensate for the impaired motor performance of our group with deficits. Our sample lacked participants with somatosensory involvement or vestibular disorders, 2 patient groups known to be more likely to demonstrate impairments with eyes closed but not with eyes opened.29 Despite this, performance on 2 tests did significantly differ between groups only in the eyes-closed condition: finger-to-tester's-finger and position holding of the LE. We speculate that the finger-to-tester's-finger, eyes-closed test may have revealed mild deficits not detected by other tests due to its difficulty, since participants had only 1 chance to “practice” the motor pattern needed to accurately reach the target during the eyes-opened test condition, which was performed first. Poorer performance during the eyes-closed condition of “position holding of the LE” test may have been attributable to mild weakness. The eyes-opened condition was performed first, requiring participants to lift and hold the position for the required 5 seconds. Subsequently, performance was repeated with eyes closed. Poorer performance during what was essentially the second repetition may have reflected insufficient strength to hold the straight leg raise position. This reasoning is similar to the methodology behind the pronator drift test of the arm or Mingazzini's maneuver for the leg to reveal mild deficits from an UMN impairment.45
Eleven tests demonstrated known-group validity, and their performance was measured with adequate reliability. This occurred for one test, alternate heel-to-knee/ankle, when it was assessed in either an ordinal or a timed manner. Tests that yielded measurements that had excellent agreement among raters (mean kappa > 0.8038 or ICC > 0.9039) and differentiated performance between persons with and without deficits included finger-to-tester's-finger, alternate finger-to-nose (timed), alternate heel-to-knee/ankle (timed), and heel-on-shin (timed). Eight tests yielded measurements that had substantial (mean kappa > 0.60)38 or good (mean ICC > 0.75)39 levels of agreement, while discriminating between groups: rebound (UE), pronation/supination, precision finger tap, draw a circle (hand), finger opposition (timed), alternate heel-to-knee/ankle, toe-to-tester's-finger (timed), and fixation/position holding (LE). These tests appear to be valid and reliable measures of coordination assessment in an acute population that includes CNS pathology from a vascular or tumor event.
A recent survey of geriatric and neurologic clinical specialists yielded 19 acute care respondents, 18 of whom reported frequent use of various limb coordination tests when assessing individuals poststroke.46 Specified tests included pronation/supination, finger opposition, finger-to-nose, alternating heel to knee/toe, and timed rapid alternating movements. Based on our results, pronation/supination performance would be reliably scored between examiners and differentiate impaired from unimpaired performance. The same is true for finger opposition and finger-to-nose when scored as the time to complete 5 repetitions. If a test that incorporates position-holding is needed, the finger-to-tester's-finger test would be better than finger-to-nose test due to its sound psychometric performance in this study. The LE test identified by the specialists,46 alternating heel-to-knee/toe, was not used in the current study. Instead, we examined alternating heel-to-knee/ankle, with the ankle as the distal target as opposed to the toe. In our experience, using the toe as a target when participants are in a supine position is not ideal, since excessive plantar flexion due to dorsiflexor weakness makes the toe target unreachable for the heel of the tested LE. Consequently, we used the ankle as the distal target. Our findings support the use of alternate heel-to-knee/ankle as a reliable measure that differentiates coordination deficits from normal performance.
A recent study involving semiquantitative assessment of the foot-tapping test found better reliability and validity compared with our results. In the study by Miller and Johnston,47 participants quickly tapped 1 foot at a time into a rater's hand. Raters scored the speed as either normal or slow. A kappa value of 0.73 indicated substantial agreement between raters, and raters were able to identify the presence of UMN deficits with 85% accuracy based on this test alone. In our study, average interrater agreement was 0.61 with qualitative assessment, 0.81 when timed, and foot tapping did not identify persons with coordination deficits. Different methodology likely contributed to the conflicting results between the 2 studies. First, the Miller and Johnston47 study had only 2 control subjects compared with 8 subjects with UMN weakness, which may have decreased the chances that raters would make a false-positive error. Second, the raters in their study directly assessed foot tapping as opposed to ours who assessed performance on the basis of video observation. Third, our raters used a 5-point scale for assessment as opposed to a 2-point scale in Miller and Johnston.47 While an option of 2 possible responses may be beneficial as a diagnostic tool (identifying either normal or abnormal performance), it cannot distinguish incremental progress over time or discriminate subtle differences in performance limiting its clinical usefulness. This lack of sensitivity of dichotomous response tests was identified in Roozenbeek et al,48 when comparing the 5-point Glasgow outcome scale to a 2-point outcome system of favorable versus unfavorable results. The ability to demonstrate treatment effects over a 6-month period of time in patients with brain injury was reduced when the 2-point scale was used.48 The objective of the current study was to assess tests used for evaluation; since these tests may also be used to assess treatment outcome, a scale useful for identifying abnormality and measuring change over time would be valuable.
There are limitations to both reliability and validity assumptions made by this study. Our analysis of interrater reliability did not include variability that would be present if the 3 raters were individually completing the examination. If the raters were carrying out the examinations and scoring the results, reliability would likely decrease because of differences in examination techniques despite standardization, and perhaps also because of variations in subject performance at different examination times.
Threats to internal validity included the lack of blinding of the person who administered the coordination examinations, which may have inserted bias into the examination procedure. Lack of patient effort is known to contaminate timed values, since speed is negatively affected by decreased motivation,49 but this was not controlled for in the current study. As there is no gold standard to identify coordination deficits in the general population, our groups were categorized as having or not having deficits based on the clinical judgment of the admitting physicians. Subjects could have been identified as having or not having deficits incorrectly, decreasing the ability to detect differences between groups. Small sample size may have also limited the ability to distinguish differences. Threats to external validity of the study include the lack of generalizability as some neurologic populations were not represented in our study sample.
Coordination tests considered in this study should be examined in patient populations not included in our sample. Future studies should evaluate properties such as specificity and sensitivity to determine how accurate the proposed tests are at detecting or ruling out, respectively, coordination impairments. Factor analysis of the tests with optimal psychometrics in a large cohort would identify the variables that comprise a comprehensive coordination examination and group tests according to those variables. An ideal assessment would contain a test representing each variable, similar to Schmahmann and colleagues'32 development of the Brief Ataxia Rating Scale.
Timed performance is very reliable, but timed outcomes are less useful if normative data are unavailable for comparison. Except for the timed finger-to-nose test, normative data for the other timed tests we examined are unknown. Collecting normative data should occur for both genders and dominant and nondominant extremities, as researchers have found timed performance to vary according to these factors.50
Finally, the full value of coordination measures as predictive of function and outcome is only beginning to be explored. Lower extremity coordination correlates moderately with gait speed in patients with Friedrich's ataxia,16 and with predicted outcome on a participation measure in persons 4 years after stroke.51 Further studies on the correlation of limb coordination tests with functional measures and their predictive validity of functional outcomes would benefit clinical practice, as coordination tests are easily assessed in patients in whom functional assessment may not be practical. Patients restricted to bed rest, tethered by multiple lines, or limited by mobility impairments, as well as clinicians limited in equipment availability and space illustrate situations in which limb coordination tests may be more useful than tests of function.
Limb coordination test performance assessed in a timed fashion was more likely to be reliably measured among raters and to identify impairments, compared to when scored with a 5-point ordinal scale. Eleven tests were scored reliably between raters (κfree > 0.60 or ICC > 0.75) and were performed differently by those with coordination deficits compared with those without deficits (P ≤ 0.05). These tests are recommended for use when examining persons with acute neurologic pathology from a vascular lesion or tumor. They are simple to perform, can be assessed at bedside, and require only a stopwatch for equipment.
1. Blumenfeld H. Neuroanatomy Through Clinical Cases. Sunderland, England: Sinauer Associates Inc; 2002.
2. Seidel HM, Ball JW, Dains JE, Benedict GW. Mosby's Guide to Physical Examination. 6th ed. St Louis, MO: Mosby Elsevier; 2006.
3. Schmitz TJ. Examination of coordination
. In: O'Sullivan SB, Schmitz TJ, eds. Physical Rehabilitation. 5th ed. Philadelphia, PA: FA Davis Company; 2007:193–225.
4. Haaxma CA, Bloem BR, Overeem S, Borm GF, Horstink M. Timed motor tests can detect subtle motor dysfunction in early Parkinson's disease. Mov Disord. 2010;25(9):1150–1156.
5. Louis ED, Marder K, Cote L, et al. Differences in the prevalence of essential tremor among elderly African Americans, whites, and Hispanics in Northern Manhattan, NY. Arch Neurol. 1995;52:1201–1205.
6. Hurvitz EA, Conti GE, Brown SH. Changes in movement characteristics of the spastic upper extremity after botulinum toxin injection. Arch Phys Med Rehabil. 2003;84:444–454.
7. Largo RH, Caflisch JA, Hug F, et al. Neuromotor development from 5 to 18 years. Part 1: timed performance. Dev Med Child Neurol. 2001;43:436–443.
8. Butcher SJ, Meshke JM, Sheppard MS. Reductions in functional balance, coordination
, and mobility measures among patients with stable chronic obstructive pulmonary disease. J Cardiopulm Rehabil. 2004;24(4):274–280.
9. Ferrarin M, Gironi M, Mendozzi L, Nemni R, Mazzoleni P, Rabuffetti M. Procedure for the quantitative evaluation of motor disturbances in cerebellar ataxic patients. Med Biol Eng Comput. 2005;42:349–356.
10. Swaine BR, Desrosiers J, Bourbonnais D, Larochelle JL. Norms for 15- to 34-year olds for different versions of the finger-to-nose test. Arch Phys Med Rehabil. 2005;86:1665–1669.
11. Trouillas P, Takayanagi T, Hallett M, et al. International Cooperative Ataxia Rating Scale for pharmacological assessment of the cerebellar syndrome. J Neurol Sci. 1997;145:205–211.
12. Desrosiers J, Rochette A, Corriveau H. Validation of a new lower-extremity motor coordination
test. Arch Phys Med Rehabil. 2005;86:993–998.
13. Louis ED, Ford B, Wendt KJ, Lee H, Andrews H. A comparison of different bedside tests for essential tremor. Mov Disord. 1999;14(3):462–467.
14. Feys PG, Davies-Smith A, Jones R, et al. Intention tremor rated according to different finger-to-nose test protocols: a survey. Arch Phys Med Rehabil. 2003;84:79–82.
15. Kimber TE, Brophy BP, Thompson PD. Ataxic arm movements alter thalamotomy for parkinsonian tremor. J Neurol Neurosurg Psychiatry. 2003;74:258–259.
16. Burk K, Malzig U, Wolf S, et al. Comparison of three clinical ratings scales in Friedreich ataxia (FRDA). Mov Disord. 2009;24(12):1779–1784.
17. Schoch B, Regel JP, Frings M, et al. Reliability and validity of ICARS in focal cerebellar lesions. Mov Disord. 2007;22(15):2162–2169.
18. Tison F, Yekhlef F, Balestre E, et al. Application of the International Cooperative Ataxia Scale rating in multiple system atrophy. Mov Disord. 2002;17(6):1248–1254.
20. Goetz CG, Poewe W, Rascol O, et al. The Unified Parkinson's Disease Rating Scale (UPDRS): status and recommendations. Mov Disord. 2003;18(7):738–750.
21. Ballard C, McKeith I, Burn D, et al. The UPDRS scale as a means of identifying extrapiramidal signs in patients suffering from dementia with Lewy bodies. Acta Neurol Scand. 1997;96:366–371
23. Brott T, Adams HP Jr, Olinger CP, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989;20:864–870.
25. Fahn S, Tolosa E, Concepcion M. Clinical Rating Scale for tremor. In: Jankovic J, Tolosa E, eds. Parkinson's Disease and Movement Disorders. 2nd ed. Baltimore, MD: Williams & Wilkins; 1993:271–280.
26. Dewey HM, Donnan GA, Freeman EJ, et al. Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovas Dis. 1999;9(6):323–327.
27. Stacy MA, Elble RJ, Ondo WG, Wu SC, Hulihan J. Assessment of interrater and intrarater reliability of the Fahn-Tolosa-Marin Tremor Rating Scale in essential tremor. Mov Disord. 2007;22(6):833–838.
28. Desrosiers J, Hebert R, Bravo G, Dutil E. Upper-extremity motor co-ordination of healthy elderly people. Age Ageing. 1995;24:108–112.
29. Orient JM. Sapira's Art and Science of Bedside Diagnosis. Philadelphia, MD: Lippincott Williams & Wilkins; 2005
30. Bain PG, Findley LJ, Atchinson P, et al. Assessing tremor severity. J Neurol Neurosurg Psychiatry. 1993;56:868–873.
31. Friesner I. Lesions of the auditory and vestibular apparatus in multiple sclerosis. Arch Neurol Psych. 1922;7(5):584–588.
32. Schmahmann JD, Gardner R, MacMore J, Vangel MG. Development of a Brief Ataxia Rating Scale (BARS) based on a modified form of the ICARS. Mov Disord. 2009;24(12):1820–1828.
33. Arceneaux JM, Kirkendall DJ, Hill SK, Dean RS, Anderson JL. Validity and reliability of rapidly alternating movement tests. Int J Neurosci. 1997;89:281–286.
34. Holmes G. The symptoms of acute cerebellar injuries due to gun-shot injuries. Brain. 1917;40:461–535.
35. Storey E, Tuck K, Hester R, Hughes A, Churchyard A. Inter-rater reliability of the International Cooperative Ataxia Rating Scale (ICARS). Mov Disord. 2004;19(2):190–212.
37. Brennan RL, Prediger DJ. Coefficient kappa: some uses, misuses, and alternatives. Educ Psychol Meas. 1981;41:687–699.
38. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.
39. Portney LG, Watkins PM. Foundations of Clinical Research. Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice Hall Health; 2000.
40. Swaine BR, Sullivan SJ. Reliability of the scores for the finger-to-nose test in adults with traumatic brain injury. Phys Ther. 1993;73(2):71–78.
41. Louis ED, Barnes L, Wendt KJ, et al. A teaching videotape for the assessment of essential tremor. Mov Disord. 2001;16(1):89–93.
42. Ott BR, Ellias SA, Lannon MC. Quantitative assessment of movement in Alzheimer's disease. J Geriatr Psychiatry Neurol. 1995;8:71–75.
43. Louis ED, Ford B, Wendt KJ, Lee H, Andrews H. A comparison of different bedside tests for essential tremor. Mov Disord. 1999;14(3):462–467.
44. Erasmus LP, Sarno S, Albrecht H, Schwecht M, Pollmann W, Konig N. Measurement of ataxic symptoms with a graphic tablet: standard values in controls and validity in multiple sclerosis patients. J Neurosci Methods. 2001;108(1):25–37.
45. Tietelbaum JS, Eliasziw M, Garner M. Tests of motor function in patients suspected of having mild unilateral cerebral lesions. Can J Neurol Sci 2002;29:337–344
46. Andrews AW, Folger SE, Norbet SE, Swift LC. Tests and measures used by specialist physical therapists when examining patients with stroke. J Neurol Phys Ther. 2008;32:122–128.
47. Miller TM, Johnston SC. Should the Babinski sign be part of the routine neurologic examination
? Neurology. 2005;65:1165–1168.
48. Roozenbeek B, Lingsma HF, Perel P, et al. The added value of ordinal analysis in clinical trials: an example in traumatic brain injury. Crit Care 2011;15(3):R127.
49. Arnold G, Boone KB, Lu P, Dean A, Wen J. Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. Clin Neuropsychol. 2005;19:105–120.
50. Ruff RM, Parker SB. Gender- and age-specific changes in motor speed and eye-hand coordination
in adults: normative values for the finger tapping and grooved pegboard tests. Percept Motor Skills. 1993;76:1219–1230.
51. Desrosiers J, Noreau L, Rochette A, Bourbonnais D, Bravo G, Bourget A. Predictors of long-term participation after stroke. Disabil Rehabil. 2006;28(4):221–230.