Limitations of Validation Study of Helmet-Based Impact Measurement System in Hockey

Wilcox, Bethany J.; Beckwith, Jonathan G. MS; Greenwald, Richard M. PhD; Crisco, Joseph J. PhD

Medicine & Science in Sports & Exercise:
doi: 10.1249/MSS.0000000000000202
SPECIAL COMMUNICATIONS: Letters to the Editor-in-Chief
Author Information

Brown University Providence, RI

Simbex Lebanon, NH

Brown University Providence, RI

Article Outline

Dear Editor-in-Chief:

Our team of researchers has extensively used Head Impact Telemetry (HIT) technology to investigate the relationship between head impact biomechanics and concussions. Members of our team invented and developed this technology specifically for these investigations, so we read with interest the recent article published by Allison et al. (1), describing a laboratory test that compared a hockey HIT System and a Hybrid III headform. Strong correlations were found between acceleration measurements; however, differences in peak accelerations were larger than those of other research-specific variants of the HIT System (2,5,7). In addition, a subset of trials was classified as invalid by HIT System data qualification algorithms. We believe the test protocol used in this study is inconsistent with on-ice use and led the authors to draw incorrect conclusions.

The authors suggest that “…validation of the HIT System for ice hockey is extremely limited…” and a “comprehensive validation” is necessary due to “differences in accelerometer orientation, processing algorithm, and helmet shape.” To clarify, HIT System hockey helmets use the same accelerometer orientation, embedded electronics, and published (not “proprietary”) processing algorithms as previously validated boxing, soccer, and football research systems (2,4,5,7). In addition, on-ice performance has been corroborated through video review, and results have been consistent across multiple independent sites (3,6,8). The authors’ tests produced lower coefficients of determination and a more skewed relationship between acceleration measures than previous evaluations of similarly configured systems (2,5,7). We believe these discrepancies are primarily due to the substantial protocol limitations described by the authors.

The hockey HIT System was designed for research and is not distributed commercially. To evaluate performance in the laboratory, helmet fit should be carefully controlled, as described in the literature (6), and test conditions (i.e., location, severity, and contact surface) should be representative of on-ice conditions and undergo verification for biofidelity (5,7). The authors described their helmet fit as a “worst-case scenario,” but it was actually unrealistic because there was no chin strap attachment and the facemask chin pad was disengaged from the headform. In addition, while 80% of on-ice impacts are attributed to contact with another player, including helmet-to-helmet, or flat surfaces (e.g., boards or ice) (8), this study delivered impacts using a noncompliant, spherical impacting ram, which is not a reasonable model of either condition. Interestingly, 19% of these tests were classified as uncharacteristic of on-ice impacts by the HIT System, demonstrating limited test fidelity and suggesting that a far more conservative interpretation of these results is warranted.

Despite these substantial study limitations, the authors recommend applying their laboratory-derived “calibration factors” to on-ice data without first considering plausibility. If previously published, on-ice data from male hockey players (3) were adjusted by the proposed calibration factor, 5% of all impacts (15–20 impacts per player) would exceed 95g—a level typically associated with concussion. In addition, distributions of peak acceleration would be 58% (3) higher than in football, an unlikely outcome considering similar injury rates exist between sports.

We agree that laboratory evaluations play an important role in assessing on-field measurement systems. Validation, however, should be a multiphase process that includes on-field/ice corroboration. The conclusions made in this study could lead to misapprehensions on hockey HIT System data, as well as misinterpretations of studies that have used this tool to advance the biomechanical understanding of concussions.

Bethany J. Wilcox

Brown University

Providence, RI

Jonathan G. Beckwith, MS

Richard M. Greenwald, PhD


Lebanon, NH

Joseph J. Crisco, PhD

Brown University

Providence, RI

Back to Top | Article Outline


1. Allison MA, Kang YS, Maltese MR, Bolte JH 4th, Arbogast KB. Validation of a helmet-based system to measure head impact biomechanics in ice hockey. Med Sci Sports Exerc. 2014; 46 (1): 115–23.
2. Beckwith JG, Chu JJ, Greenwald RM. Validation of a noninvasive system for measuring head acceleration for use during boxing competition. J Appl Biomech. 2007; 23 (3): 238–44.
3. Brainard LL, Beckwith JG, Chu JJ, et al. Gender differences in head impacts sustained by collegiate ice hockey players. Med Sci Sports Exerc. 2012; 44 (2): 297–304.
4. Chu JJ, Beckwith JG, Crisco JJ, Greenwald RM. A novel algorithm to measure linear and rotational head acceleration using single-axis accelerometers. J Biomech. 2006; 39: S534.
5. Hanlon E, Bir C. Validation of a wireless head acceleration measurement system for use in soccer play. J Appl Biomech. 2010; 26 (4): 424–31.
6. Mihalik JP, Greenwald RM, Blackburn JT, Cantu RC, Marshall SW, Guskiewicz KM. Effect of infraction type on head impact severity in youth ice hockey. Med Sci Sports Exerc. 2010; 42 (8): 1431–8.
7. Rowson S, Beckwith JG, Chu JJ, Leonard DS, Greenwald RM, Duma SM. A six degree of freedom head acceleration measurement device for use in football. J Appl Biomech. 2011; 27 (1): 8–14.
8. Wilcox BJ, Machan JT, Beckwith JG, Greenwald RM, Burmeister E, Crisco JJ. Head impact mechanisms in men’s and women’s collegiate ice hockey. J Athl Train. In press.
© 2014 American College of Sports Medicine