INTRODUCTION
Concussions, or mild traumatic brain injuries, pose short-term and long-term health risks to athletes at all levels of competition. Emerging evidence has revealed a correlation between repetitive concussions and neurodegenerative disease, most notably through the identification of chronic traumatic encephalopathy in athletes.1–5 In addition, second impact syndrome has claimed the lives or severely impaired healthy middle school and high school athletes.6,7 Of the approximately 44 million student-athletes in the United States who annually compete in organized sports, the Centers for Disease Control and Prevention (CDC) estimates that between 1.6 and 3.8 million concussions occur in sports and recreational activities each year, although recent studies suggest the incidence may be significantly higher.8–12 However, most students lack basic awareness of recognizing and appropriately responding to these injuries, underscoring the importance of concussion education for student-athletes.13–15 An increased appreciation and knowledge of concussions among student-athletes would provide a significant public health benefit by promoting a more safety-conscious sports culture in the United States.
Recent efforts conducted at the national, state, and local levels have sought to provide education about concussions to coaches, parents, and student-athletes.16 The need for validated concussion education programs is anticipated to rise in the coming years as an increasing number of states pass legislation requiring education for each of these groups. More than 40 states have either laws or pending bills addressing concussions in youth sports, with at least 31 states now requiring education for coaches, parents, student-athletes, or some combination of the groups.16 However, the nature of this education, and in particular, what constitutes an effective program for student-athletes, remains to be determined. Currently, the field lacks validated teaching approaches for concussion education directed at student-athletes. Identification and adoption of such approaches would produce a widespread impact and direct health benefit to the millions of student-athletes who compete across the United States.
Here, we evaluate the efficacy of a novel concussion education program called Sports Legacy Institute Community Educators (SLICE) by measuring learning trends of participating students. The SLICE program features trained medical and health-related students who deliver age-specific interactive presentations that include discussion, videos, audience demonstrations, and case studies of professional, collegiate, and high school athletes. Employing these interactive approaches, SLICE presentations emphasize how student-athletes can recognize concussion symptoms and respond to concussions when they occur to themselves or teammates. In the current study, we compared student responses on quizzes administered before and after the SLICE presentation, and we observed significant improvements in concussion knowledge across all age and gender subgroups. Our results suggest that lessons from the SLICE program may be broadly applicable to concussion education for all student-athletes.
METHODS
The Institutional Review Board of Children's Hospital Boston approved the review of the de-identified standard of care prepresentation and postpresentation quizzes to evaluate the effectiveness of the SLICE Program. Schools contacted Sports Legacy Institute (SLI) for the SLICE program to be presented at their respective schools.
Experimental Subjects
Subjects consisted of 636 students ranging from 9 to 18 years in elementary, middle, and high school. Subjects were excluded if they did not complete their pre- or postpresentation quiz. The majority of subjects participated in contact, limited contact, and noncontact sports. It is not known if subjects had received any previous education of concussions.
SLICE Program
The SLICE presentations were delivered by groups of 2 to 3 medical-related or health-related student volunteers. Before presenting, all volunteers received a standard training that included a review of peer-reviewed journal articles, CDC concussion guidelines, relevant concussion case studies, a previous SLICE presentation film, and delivering a mock presentation to SLICE officers. The SLICE officers monitored training and adherence to the SLICE program. Presenters were permitted to deviate from the standard presentation script if it was conducive to student learning and included the required teaching points within each section.
Presentations lasted between 40 to 60 minutes. Format included PowerPoint slides, video segments, demonstrations with audience members, case studies of professional and high school athletes, personal testimonies from collegiate athletes, interactive discussions with audience members, and a question-and-answer period. Content included signs and symptoms of concussions, the potential short-term and long-term consequences of concussions, and strategies for responding to concussions. Presentation content was compiled under the guidance of the SLI medical advisory board, with age-appropriate modifications made for specific audiences.
Quiz Design, Distribution, and Scoring
Quizzes were designed to assess students' knowledge of concussions, with an emphasis on recognition of signs and symptoms and appropriate responses after a concussion. Questions consisted of free-response, true-or-false, and multiple-choice items. Demographic information including age, gender, ethnicity, and sports played was obtained; identifiable information such as name, e-mail address, or date of birth was not collected. The quiz content on prepresentation and postpresentation quizzes was identical. All questions were consistent across age groups and presentations.
Quiz packets were distributed to students as they entered the auditorium or gymnasium. Stapled packets consisted of a prepresentation quiz and a postpresentation quiz. Each sheet in the packet contained a matching number in the upper corner to allow comparison of anonymous individual responses before and after the presentation. Students were asked to complete the prepresentation quiz, detach the completed quiz from their packets, and set aside the remaining postpresentation quiz to complete after the presentation. Quizzes were collected by SLICE volunteers, teachers, and school administrators before the presentation.
All quizzes were scored by 2 individuals using a standardized rubric and answer key to minimize variations in scoring. Question 1, a free-response question, asked students to list signs and symptoms of a concussion. Twenty individual spaces were provided for student responses. Students were awarded 0 points for 0 correct responses, 2 points for 1 to 2 correct responses, 4 points for 3 to 4 correct responses, 6 points for 5 to 6 correct responses, 8 points for 7 to 8 correct responses, and 10 points for >8 correct responses. Question 2, a free-response question, asked students to identify appropriate responses to take after a concussion to themselves or teammates. Students received 2 points for each correct response up to a maximum of 10 points. Each correct response on the 5 true-or-false or multiple-choice questions was awarded 1 point. The overall quiz was scored out of 25 points with a passing level set at 50%.
Statistical Analysis
Changes in mean quiz scores between prepresentation and postpresentation expressed as a percentage out of 100% were evaluated using paired t tests and comparisons between male and female students, and age groups in the amount of improvement were assessed by analysis of variance. Using a passing percentage of 50%, change in the proportion who passed between prepresentation and postpresentation was assessed by McNemar test for correlated binary proportions, whereas differences in the percentage passing between age groups (9-12, 13-15, and 16-18 years) and between male and female students was evaluated using Fisher exact test. Multivariable logistic regression was applied to identify whether age, gender, and race were significantly associated with postpresentation passing or failing using the likelihood ratio test to assess significance. Statistical analysis was performed using SPSS version 19.0 (SPSS Inc/IBM, Chicago, Illinois). Error bars in the figures denote standard deviations. Statistical significance was set at two-tailed P < 0.05.
RESULTS
Quizzes were administered to male and female students ranging from 9 to 18 years (Table 1). Prepresentation and post- presentation mean scores on the quiz for all 599 students were 43% ± 16% and 65% ± 20%, respectively (paired t = 25.71, P < 0.0001). On average, female students improved by 25% ± 18% (P < 0.0001) and male students by 20% ± 20% (P < 0.0001) (Figure 1). Students aged 9 to 12 years improved on average by 23% ± 18% (P < 0.0001), 13- to 15-year-olds by 23% ± 19% (P < 0.0001), and 16- to 18-year-olds by 21% ± 22% (P < 0.0001) (Figure 2).
;)
TABLE 1: Demographics
FIGURE 1: Mean prepresentation and postpresentation concussion quiz scores for all 599 participants and stratified by gender. Asterisks denote significant improvement by paired t tests (P < 0.0001). Error bars are standard deviations. Scores overall increased by an average of 22%.
FIGURE 2: Mean prepresentation and postpresentation concussion quiz scores stratified according to age group. Asterisks denote significant improvement by paired t tests (P < 0.0001). Error bars are standard deviations.
The proportion of students who passed using 50% as the criterion was 34% prepresentation (202 of 599) and 80% postpresentation (481 of 599) (P < 0.0001, McNemar test). The proportion of female students who passed increased significantly between prepresentation (113 of 290; 39%) and postpresentation (266 of 290; 92%) (P < 0.0001) and the proportion of male students who passed increased significantly between prepresentation (89 of 309; 29%) and post- presentation (215 of 309; 70%) (P < 0.0001). Although both female and male students improved significantly, the percentage who passed postpresentation was significantly higher among female students (266 of 290; 92%) compared with male students (215 of 309; 70%) (P < 0.0001, Fisher exact test) (Figure 3).
FIGURE 3: Prepresentation and postpresentation passing rates based on a criterion of 50% performance for all 599 participants and stratified by gender and age groups. Asterisks denote significant improvement by McNemar test for paired binary proportions (P < 0.0001). Overall, the percentage passing increased by 34% postpresentation. Although female and male students showed significant gains in knowledge, the postpresentation passing rate was higher for female than male students (asterisk, 92% vs 70%) (P < 0.0001, Fisher exact test).
The proportion of young students aged 9 to 12 years who passed increased significantly between prepresentation (18 of 104; 17%) and postpresentation (76 of 104; 73%) (P < 0.0001), proportion of students aged 13 to 15 years who passed increased significantly between prepresentation (105 of 310; 34%) and postpresentation (260 of 310; 84%) (P < 0.0001), and proportion of students aged 16 to 18 years who passed increased significantly between prepresentation (68 of 148; 46%) and postpresentation (125 of 148; 85%) (P < 0.0001). Although all age groups demonstrated significant improvement, the percentage who passed postpresentation was significantly higher among those students aged 13 years and older compared with those less than 13 years (P < 0.0001) (Figure 3).
Multivariable logistic regression indicated that age group (P < 0.0001) and gender (P < 0.0001) were both significant factors associated with passing the quiz postpresentation: female students fared better than male students within each age group, and students 9 to 12 years of age had lower passing rates compared with older students among male but not among female students. Among female students, in all 3 age groups, the percentage who passed postpresentation was 94%. Male students aged 9 to 12 years showed the lowest passing rate. Also, among male students, the percentage who passed postpresentation was 53% among 9- to 12-year- olds, 73% among 13- to 15-year-olds, and 78% for 16- to 18-year-olds. Passing the quiz postpresentation was not significantly different between those who play football compared with other sports (P = 0.77).
DISCUSSION
As the medical and scientific communities continue to discover the multifarious short-term and long-term risks of concussions, it is essential that student-athletes learn about concussions at an early stage in their athletic careers. Such educational interventions have been proven to be an effective mode of injury prevention.17 To improve students' knowledge of concussions and therefore decrease morbidity, there is a growing need for educational programs tailored specifically for student-athletes. The SLICE program is one whose novelty is based on the use of trained medical and health-related students as the presenters, as well as the incorporation of audience demonstrations, case studies of professional and high school athletes, and interactive games into the presentation. Here, we report significant learning improvements among participating student-athletes based on quizzes administered before and after the SLICE presentations. To our knowledge, to date, our findings represent the largest systematic and quantitative evaluation of a concussion education program focused on a student-athlete.8,18
Significant improvements in absolute quiz scores were observed across all age groups in our study. However, the significantly higher postpresentation passing rate among students 13 years of age or older indicates the effectiveness of presenting age-specific educational content to student-athletes. Importantly, although we observed improvements in absolute quiz score in both genders, female students within each age group performed significantly better than male students on both prepresentation and postpresentation quizzes. This is notable because girls are believed to have higher concussion rates compared with boys who play related sports.9,19 Regarding the observed gender difference on our assessment, interactions between gender, question format, and overall testing performance in this demographic have been studied; in general, open-ended questions, which comprise the majority of the grade on our quiz, have shown less gender differences than multiple-choice format.20,21 Therefore, we conclude that the interactive presentation style of the SLICE program promotes effective learning among both male and female students in elementary school, middle school, and high school.
Our grading criteria are based on our belief that recognition of concussions and appropriate response to concussions are of equal importance to the health and well-being of student-athletes. This notion is supported by the CDC “Heads Up” program, which provides student-athletes with the common signs and symptoms as well as actions to take after a suspected concussion.22 Indeed, the SLICE curriculum devotes significant time to these complementary concepts of recognition and response.
There are several limitations in the current study. Although students were instructed to work on their quizzes independently, the testing environment may have facilitated collaboration between students and providing responses during the presentation. However, we believe these potential confounding factors were minor, and if present, such outcomes would nevertheless contribute to student learning. Additionally, while our sample size in this study is appreciable, a larger population drawn from geographic regions across the United States would provide further evidence on the effectiveness of SLICE. Because cultural and social norms vary widely between different geographic regions, it will be important to understand which messaging is most effective for students in each of these regions as our program expands. Finally, although we identified significant short-term improvements in knowledge, it remains to be determined whether receiving concussion education through SLICE translates into long-term behavioral changes among student-athletes. The magnitudes of observed effects in this study, however, indicate significant value of early educational intervention in pediatric student-athlete populations.
In addition to the benefit that SLICE provides to athletes, student volunteers also benefit greatly from serving as educators. Beyond developing their teaching, leadership, and mentoring skills, SLICE student volunteers increase their knowledge about concussion recognition and management at an early stage in their professional training and are more likely to retain this knowledge as practicing healthcare professionals.23 Despite the high prevalence of concussive injuries in the United States, preclinical medical training in this country surprisingly devotes very little, if any, time to concussion pathophysiology and management.24 By training students to deliver presentations, SLICE helps the next generation of healthcare professionals become more aware and knowledgeable about concussions. Although our findings do not reflect the magnitude of this secondary benefit, we believe that training future physicians, athletic trainers, nurses, and other healthcare professionals through the SLICE program will lead to significant improvements in overall concussion management.
Studies are currently underway to investigate concussion incidence and proper management among student-athletes who received training through the SLICE program. Objective parameters including number of concussions reported by student-athletes during their seasons and frequency of aggressive or dangerous penalties would provide added insights into the efficacy of our approach.13 Through follow-up with school administrators and distributing quizzes and surveys at schools in the months after a SLICE presentation, we seek to measure the long-term impact of SLICE in a variety of ways. Particularly, we will assess whether students who received SLICE training reported more concussions during their seasons; the frequency of aggressive penalties that could lead to concussion; and the degree to which students have modified their behavior, such as wearing helmets, reporting concussions to adults, or returning to play. It is also of great interest to examine the educational impact of SLICE on medical and other health-related students who deliver the presentations by comparing their general knowledge of concussions to other students at comparable stages of training.
Given the high incidence and potential severity of concussions suffered by youth athletes, the widespread lack of concussion knowledge within this demographic is alarming. Educational approaches that teach students the fundamentals of concussion recognition and response represent the most important intervention the medical community can provide to mitigate the negative consequences of concussions in pediatric populations.9 The SLICE program provides a highly effective model for educating student-athletes about concussions and is particularly novel in its use of medical students and other health-related students as the educators. The SLICE program, which has the potential to be scaled to a national level, aims to become a driving force to provide vital concussion education to the millions of student-athletes across the United States.
ACKNOWLEDGMENTS
The authors acknowledge that the results of this study do not constitute endorsement by the American College of Sports Medicine. The authors acknowledge Valerie L. Ugrinow, Research Manager in the Division of Sports Medicine at Children's Hospital Boston, for her contributions to this study; and Robert Stern, PhD, Co-Director of the Center for the Study of Traumatic Encephalopathy at Boston University School of Medicine, for his helpful suggestions on the manuscript.
REFERENCES
1. Cantu RC. Chronic traumatic encephalopathy in the National Football League. Neurosurgery. 2007;61:223–225.
2. Costanza A, Weber K, Gandy S, et al.. Review: contact sport-related chronic traumatic encephalopathy in the elderly: clinical expression and structural substrates. Neuropathol Appl Neurobiol. 2011;37:570–584.
3. Gavett BE, Stern RA, McKee AC. Chronic traumatic encephalopathy: a potential late effect of sport-related concussive and subconcussive head trauma. Clin Sports Med. 2011;30:179–188, xi.
4. McCrory P, Zazryn T, Cameron P. The evidence for chronic traumatic encephalopathy in boxing. Sports Med. 2007;37:467–476.
5. McKee AC, Cantu RC, Nowinski CJ, et al.. Chronic traumatic encephalopathy in athletes: progressive tauopathy after repetitive head injury. J Neuropathol Exp Neurol. 2009;68:709–735.
6. Cantu RC. Second-impact syndrome. Clin Sports Med. 1998;17:37–44.
7. Thomas M, Haas TS, Doerer JJ, et al.. Epidemiology of sudden death in young, competitive athletes due to blunt trauma. Pediatrics. 2011;128:e1–e8.
8. Echlin PS, Tator CH, Cusimano MD, et al.. A prospective study of physician-observed concussions during junior ice hockey: implications for incidence rates. Neurosurg Focus. 2010;29:E4.
9. Halstead ME, Walter KD. American Academy of Pediatrics. Clinical report—sport-related concussion in children and adolescents. Pediatrics. 2010;126:597–615.
10. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of traumatic brain injury: a brief overview. J Head Trauma Rehabil. 2006;21:375–378.
11. Lincoln AE, Caswell SV, Almquist JL, et al.. Trends in concussion incidence in high school sports: a prospective 11-year study. Am J Sports Med. 2011;39:958–963.
12. National Council of Youth Sports. NCYS Report on Trends and Participation in Organized Youth Sports. National Council of Youth Sports. NCYS: Stuart, FL.
http://www.ncys.org/publications/2008-sports-participation-study.php. Accessed July 13, 2011.
13. Cook DJ, Cusimano MD, Tator CH, et al.. Evaluation of the ThinkFirst Canada, Smart Hockey, brain and spinal cord injury prevention video. Inj Prev. 2003;9:361–366.
14. Delaney JS, Abuzeyad F, Correa JA, et al.. Recognition and characteristics of concussions in the emergency department population. J Emerg Med. 2005;29:189–197.
15. Delaney JS, Lacroix VJ, Leclerc S, et al.. Concussions during the 1997 Canadian Football League season. Clin J Sport Med. 2000;10:9–14.
16. Schatz P, Moser RS, Covassin T, et al.. Early indicators of enduring symptoms in high school athletes with multiple previous concussions. Neurosurgery. 2011;68:1562–1567; discussion 1567.
17. Safety in youth ice hockey: the effects of body checking. American Academy of Pediatrics. Committee on Sports Medicine and Fitness. Pediatrics. 2000;105:657–658.
18. Provvidenza CF, Johnston KM. Knowledge transfer principles as applied to sport concussion education. Br J Sports Med. 2009;43(suppl 1):i68–i75.
19. Gessel LM, Fields SK, Collins CL, et al.. Concussions among United States high school and collegiate athletes. J Athl Train. 2007;42:495–503.
20. Kimball MA. New perspective on women's math achievement. Psychol Bull. 1989;105:198–214.
21. Stumpf H, Stanley JC. Stability and change in gender-related differences on the College Board Advanced Placement and Achievement Tests. Curr Dir Psychol Sci. 1998;7:192–196.
22. Centers for Disease Control. CDC Fact Sheet for Athletes. Atlanta: Centers for Disease Control and Prevention.
http://www.cdc.gov/concussion/pdf/athletes_Eng.pdf. Accessed July 15, 2011.
23. Dale E. The cone of experience. In: Audio-Visual Methods in Teaching. Vol 1. New York, NY: Dryden Press; 1946.
24. Demorest RA, Bernhardt DT, Best TM, et al.. Pediatric residency education: is sports medicine getting its fair share? Pediatrics. 2005;115:28–33.
