Echemendia, Ruben J. PhD; Putukian, Margot MD; Mackin, R. Scott MS; Julian, Laura MS; Shoss, Naomi
Of the roughly 500,000 hospital admissions due to traumatic brain injury in 1984, sports are listed as the cause in 3–10%. 1 Within the realm of sport, mild traumatic brain injuries (mTBI) are important, and account for 5.5% of all injuries at the high school level, 2 and 1.6–6.4% of all injuries at the college level. 3 The incidence of head injury at the National Collegiate Athletic Association (NCAA) level is 0.06–0.55 per 1,000 athlete-exposures (game and practice). At the high school level, football is the sport with the highest incidence of mTBI, with a 15–20% risk for head injury per season, with 200,000 of these injuries occurring annually, and an average of 8 deaths per year. 4,5
The brain injuries that can occur in sport are varied in severity, from mild injuries with minor sequelae to more severe and life-threatening injuries. Cervical spine injuries and significant vascular injuries including subdural and epidural hematomas, hemorrhage, and contusions must be considered in the head-injured athlete. “Concussion,” or mTBI was defined as early as 1966 as “a clinical syndrome characterized by immediate and transient impairment of neurologic function secondary to mechanical forces.”6 mTBI represent common and significant concerns for health care professionals taking care of the athlete.
Many mTBI go undetected. Athletes will often continue playing despite symptoms due to a fear that they will be taken out of the game if they admit to symptoms, or an expectation that mild head injury is “part of the game.” McLatchie 7 found that 56% of 544 rugby athletes sustained at least one head injury associated with amnesia afterwards. Eleven percent of athletes had posttraumatic amnesia for more than 1 hour, and yet 35% of these athletes were not brought to a hospital for treatment. The consequences of these unreported injuries remain unclear, but several concerns about this issue have been explored.
There has been concern that repetitive concussions or return to play prior to complete resolution of a brain injury can have potentially detrimental effects on long-term cognitive function. The concern for “second impact syndrome,”8 where an athlete returns to play and sustains a second injury prior to complete resolution of a first insult with resultant serious sequelae including death, is of significant concern. There has also been concern that repetitive brain injury can lead to deficits in cognitive function or the “punch drunk syndrome,” which has been described in boxers. 9–11 These long-term sequelae of serious or repetitive brain injury can have impact on the function of the athlete outside of their sports participation.
There are several classification systems and return-to-play guidelines in the literature, 12–17 which health care professionals have used in taking care of the brain-injured athlete. However, very little prospective data exist, and many of the classification systems are of questionable utility since they are based on popular belief or practice rather than empirical evidence. 18 Several classification systems use loss of consciousness (LOC) as a predictor of severity, although recent data would seemingly question the assumption that LOC signals a more serious injury. 19
A variety of studies have been used to assess the brain-injured athlete, including plain radiographs, computerized tomography (CT), magnetic resonance imaging (MRI), electroencephalography (EEG), and, more recently, near-infrared spectroscopy, 20,21 single-photon emission computed tomography (SPECT), 22,23 magnetic resonance angiography, 24 and diffusion weighted magnetic resonance imaging. 25 The utilization of these tests often depends on the result of clinical examination. Although useful in detecting structural abnormalities and vascular events, these studies have not been very useful in the identification of mTBI. For example, it has been documented that an athlete can have significant cognitive compromise and still have normal imaging. 26 The difficulty in imaging mTBI stems from the microscopic axonal damage 27 and subsequent cellular dysfunction 28 that underlies the pathology.
The use of neuropsychological techniques in the assessment of the brain-injured athlete is not new, although there has been a renewed interest in its use within the realm of sports medicine. Neuropsychological testing provides an assessment and quantification of brain function by examining brain–behavior relationships. Neuropsychological testing may be more sensitive in assessing cognitive function in the brain-injured athlete than the more traditional diagnostic studies listed above. Neuropsychological testing has been useful in the assessment of the acute and recovery phases of mTBI. 29–31 These measurements assess memory recall, attention and concentration, problem-solving abilities, visual tracking, reaction time, and speed of information processing as well as other measures of cognitive function. Neuropsychological testing has been used specifically in the assessment of the head-injured athlete, 32–36 in cross-sectional studies of athletes 30,37,38 as well as in assessing the effects of heading in soccer players. 39 Much of the literature using neuropsychological testing has been in American football, with sparse data in other sports and on female athletes.
The goal for the sports medicine team taking care of the brain-injured athlete is to ensure appropriate acute care as well as safely returning the athlete to play. The health and well-being of the athlete is the primary concern, with significant danger being associated with premature return to play. The athletes should be returned to competition as soon as possible without putting them at risk for further injury.
The current study was designed to assess the usefulness of neuropsychological testing in the assessment of concussion in college athletes. It used a prospective design in a competitive Division I university setting where close observation and follow-up is available, and included multiple sports as well as both male and female athletes. The assessments included classic medical testing (including a complete neurologic examination) and a battery of neuropsychological tests. Preinjury baseline testing was obtained on all athletes, with subsequent repeat testing if an athlete sustained an mTBI. This article represents the results of the first 29 athletes who sustained an mTBI as part of the Penn State Cerebral Concussion Program (PSCCP).
The PSCCP was established in 1995 as a prospective neuropsychological evaluation of athletes who were considered to be at risk for mTBI. The methodology of the program was based on the pioneering work of Dr. Jeff Barth and his colleagues 47 at the University of Virginia, who first introduced the concept of “baseline” testing in sport neuropsychology. Originally, the PSCCP included athletes from football, men's and women's soccer, men's ice hockey, and men's and women's basketball. Baseball and swimming (men's and women's) participated as control sports. Currently, the program has expanded to include wrestling, women's lacrosse, and the ice hockey teams from Princeton University and Acadia University. To date, a total of 996 baseline assessments have been conducted. Data from the first 29 athletes who sustained an mTBI will be reported along with the data of 20 same-sport, noninjured controls.
Participants were 49 (45 male, 4 female) college-level athletes participating in the PSCCP. The sample was 82% Caucasian, 12% African American, 1.6% Latino, and 4.4% participants with undeclared ethnicity. The participants included 29 individuals who suffered an mTBI during athletic competition, and 20 individuals who served as control subjects for those who experienced an mTBI. Control subjects were matched on the basis of age, gender, sport played, and ethnicity. There were fewer control subjects than injured athletes because, in some cases, one control participant was matched with more than one injured athlete. This multiple matching was necessary because there were a limited number of players who fit the matching criteria and who could serve as controls within some sports. This methodology has been used in other studies of this type. 18,47 Control subjects were examined at the same time intervals as their injured counterparts; however, due to logistical concerns they were not necessarily examined on the same day. For example, an injured player and his or her control may not have played in the same game on the day of injury. However, if a player was injured during a practice, their control was also tested after a practice. Similarly, if a player was injured during a game, the control was tested following a game. The time intervals of testing for the injured player were then matched for their corresponding control.
A total of 113 evaluation sessions were conducted for those who experienced an mTBI (injured), and 88 test sessions for those individuals who served as control subjects. Of the injuries, 12 (43%) occurred in football, 3 (10%) occurred in men's soccer, 10 (33%) occurred in men's ice hockey, 1 (4%) occurred in women's basketball, and 3 (10%) occurred in men's basketball.
Table 1 presents the tests used in the assessment battery along with the cognitive functions that are assessed by each test. As can be seen, the assessment battery consisted of 11 neuropsychological tests chosen to assess various aspects of memory, learning, attention, concentration, and information processing. The tests included The Post Concussion Symptom Checklist, The Hopkins Verbal Learning Test (HVLT), 40 Symbol Digit Modalities Test, 41 Symbol Digit Modalities—Memory, Digit Span Test, 42 The Penn State Cancellation Test, Trail Making Test, 43 Controlled Oral Word Association Task (COWAT), 44 The Stroop Test, 45 Vigil Continuous Performance Test, 46 and a five-word list-learning task.
Several factors were taken into account in test selection. Since there was a limited amount of time available for testing athletes, the neurocognitive domains that were assessed targeted those cognitive systems that are known to be at risk following mTBI. The test instruments chosen needed to be short in duration, easily administered without extensive neuropsychological training, have sound psychometric properties (see references above for each test), have a history of use with athletes, and produce minimal levels of frustration. Each of the measures described above has been used extensively with athletes and has demonstrated adequate levels of reliability and validity. These measures were also found to be useful and recommended for the assessment of sports-related mTBI by the Sport Neuropsychology Panel. 48
Two measures of verbal learning and memory were analyzed for this study, the HVLT and a five-word list-learning task. As with any long-term clinical protocol, tests within a battery may be substituted, as data suggest that alternative measures may prove to be more useful. The five-word list-learning task was initially included in the battery because it was often used clinically with athletes, was easy to administer, and data had suggested that it would be appropriate for this population. 44 However, after the first 2 years of the program, it was determined that the list-learning task generated a ceiling effect that limited its usefulness in assessing some athletes. In other words, the task was too easy. The list-learning task was replaced with the more difficult 12-word format of the HVLT. As a result, some of the athletes early in the program were tested with the list-learning task, and others who were assessed later in the program were given the HVLT.
Each participant in this study completed a baseline neuropsychological evaluation prior to the start of their athletic career at Penn State. During the first year of the program, all athletes were evaluated prior to the start of contact practice. The initial assessment, or baseline testing, required approximately 40 minutes to administer. In addition to evaluating each new player to the team, 25% of the team was reevaluated each year to control for developmental changes in baseline data.
If an athlete participating in the study suffered an mTBI during the course of the year, he or she was retested 2 hours following injury, 48 hours postinjury, 1 week postinjury, and 1 month postinjury. With the exception of the 2-hour assessment, all follow-up assessments used the same battery of tests as the baseline testing battery. Different forms of the Hopkins Verbal Learning Test, The Penn State Cancellation Test, and Controlled Oral Word Association Test were used to reduce practice effects. The 2-hour assessment was an abbreviated battery, which included the Concussion Symptoms Checklist, Digit Span Test, Hopkins Verbal Learning Test (or List Learning Test), and the Stroop Test. An abbreviated battery was used at the 2-hour mark for several reasons. Since the assessment was relatively close to the time of injury, it was necessary to minimize the effects of fatigue and frustration. Ease of administration was a critical concern because the tests were generally given in a setting where traditional tables and chairs were unavailable (e.g., hallway, bus). The tests were also administered by team athletic trainers and physicians in addition to the neuropsychology project team. Given the number of people involved and the need for consistent, reliable test administration, the tests needed to have concise, uncomplicated instructions for administration. Because of these concerns, those measures that were thought to be the most sensitive, efficient, and easily administered were chosen for the abbreviated battery.
Performance on the 11 neuropsychological test instruments was assessed using 23 summary scores. A multivariate analysis of covariance (MANCOVA) was conducted to determine if there were significant differences in performance on the 23 summary variables between the mTBI group and the control group on the baseline evaluation (prior to injury). The covariate used in this analysis was the Scholastic Achievement Test (SAT)–Total Score. SAT was used as a covariate to control for the effects of general cognitive ability on neuropsychological test scores. The MANCOVA (Pillai's Trace) did not reveal a significant main effect for active/control group membership at baseline testing, F(10,23) = 1.59, p = 0.227, η2 = 0.787. SAT scores were not found to be a significant covariate for neuropsychological performance with this assessment battery, F(24,1) = 1.64, p = 0.558, and for this reason, SAT was not used in subsequent analyses. Multivariate analysis of variance (MANOVA) procedures were used to determine if statistical differences between mTBI athletes and control subjects existed on neuropsychological measures at the four test intervals following baseline testing (2 hours postinjury, 48 hours postinjury, 1 week postinjury, and 1 month postinjury).
At 2 hours postinjury, a MANOVA indicated that control subjects performed significantly better on neuropsychological tests in the abbreviated assessment battery when compared with the performance of concussed athletes at the same testing interval, F(12,10) = 8.70, p < 0.000, η2 = 0.913. Controls performed better than injured athletes on all 12 neuropsychological indices in the abbreviated battery. As can be seen in Table 2, 6 of the 12 univariate F-tests showed statistically significant differences between mTBI athletes as compared with control subjects. Figure 1 presents the group data normalized to a mean of 100 and standard deviation (SD) of 15. The findings suggest that mTBI athletes performed significantly worse on measures of working memory, attention and concentration, and verbal learning when compared with noninjured controls.
At 48 hours postinjury, significant differences between mTBI athletes and control subjects were found using MANOVA, F(24,15) = 6.18, p < 0.000, η2 = 0.908. Control subjects performed better on 20 of 23 measures of neuropsychological functioning (full battery) when compared with mTBI athletes at the same time intervals. Presented in Table 3 are the data from the five measures that reached statistically significant differences between injured and noninjured participants, with control athletes scoring higher on all five tests. Figure 2 presents the data using normalized scores (mean = 100, SD = 15). As can be seen, significant differences were obtained on measures of working memory, verbal learning, verbal memory, divided attention, and speed of information processing, with injured athletes scoring significantly below noninjured controls.
Of interest is the greater discrepancy between injured and noninjured athletes at 48 hours on the Hopkins Learning Index (HLI) measure, when compared with their data at the 2-hour postinjury evaluation. Presented in Figure 4 are the HLI data from each testing interval. The groups are indistinguishable at baseline. Beginning with the 2-hour evaluation and continuing through the 48-hour evaluation, the control athletes appeared to be making use of prior exposure to the task and demonstrated the expected “practice effect” on this measure. In other words, the control athletes were learning how to make their performance more efficient. In contrast, the injured athletes' performance declined at the 48-hour mark. At 1 week postinjury, the injured athletes began to recover, with the groups reaching equivalent scores at 1 month postinjury.
At 1 week postinjury the controls continued to score better than their concussed counterparts on 20 of the 23 neuropsychological indices. However, when subjected to a MANOVA procedure, the differences between injured athletes and controls were not significant, F(24,7) = 0.906, p = 0.608, η2 = 0.756. Given the exploratory nature of this study, the univariate ANOVAs were examined. Table 4 presents the results of four univariate F-tests that were significant, with control subjects scoring higher on each of these tests. This finding suggests that some participants may have continued to experience problems with visual–motor speed, working memory, sustained attention and concentration, and reaction time up to 1 week postinjury. However, given that the overall MANOVA did not reach significance, these results were only considered suggestive.
A MANOVA procedure could not be used at 1 month postinjury because missing data for some of the participants led to an unacceptably low number of cases for inclusion in the analysis. Consequently, a series of ANOVAs (with Bonferroni correction) were conducted using the neuropsychological tests as dependent measures and injured versus control status as factors. Of the 23 analyses, only one reached statistical significance. Injured athletes performed better than their controls on the Symbol Digit Modalities Test, total correct; F(1,38) = 5.867, p = 0.020. Injured athletes achieved a group mean of 71.53 and control athletes scored 70.88. Although statistically significant, the difference between the means is clinically insignificant.
Lastly, an examination of Tables 2 through 4 and Figures 1 through 4 reveals interesting differences between the variability in scores for those athletes with mTBI and the controls. The standard deviations depicted in the graphs reflect a pattern of markedly greater variability among the concussed athletes when compared with the controls. This finding strongly underscores the individual differences that exist among athletes with mTBI and, as will be discussed later, reinforces the notion that return-to-play decisions should be based on idiographic, not group comparisons.
This study examined whether neuropsychological assessment techniques could differentiate between college athletes who sustained an mTBI and same-sport noninjured controls. The results indicated that the injured athletes did not significantly differ from the control group on demographic and neuropsychological test data at baseline. It was also found that SAT scores were not a significant covariate of the neuropsychological test data for either group. This finding was surprising since significant relationships between general cognitive abilities and neuropsychological test scores have been found in the literature. 44 It is possible that a relationship did not emerge because the neuropsychological tests that were used in this study were selected for their ability to detect changes that were specific to the sequelae of mTBI. Consequently, the tests may not have assessed enough of the general intellectual factor that is thought to be measured by the SAT. Alternatively, it may be argued that since the SAT was designed to predict college performance, it is not a robust measure of general cognitive abilities.
The results of the study indicate that neuropsychological tests were able to significantly differentiate between injured athletes and their controls at 2 hours postinjury. Injured athletes scored significantly lower on tests assessing attention and concentration, verbal learning, and verbal memory. The magnitude of the difference between the controls and injured subjects was not only statistically robust, it was clinically significant with three of the tests showing a difference greater than 1 standard deviation between injured athletes and controls. To our knowledge, this is the first report of a limited neuropsychological test battery being used within hours of the concussive event. The finding strongly suggests that neuropsychological instruments have utility in identifying cognitive changes following mTBI within 2 hours of injury. Usually, neuropsychological tests have been used to monitor recovery of function within the context of sports. These data suggest that the tests may also be useful in the diagnosis of mTBI.
At 48 hours postinjury, neuropsychological tests continued to significantly differentiate between injured athletes and controls. Again, robust differences were found in the overall MANOVA between injured and noninjured athletes. Five measures demonstrated statistically significant differences on follow-up univariate analyses with control athletes performing significantly better on tests assessing working memory, attention and concentration, verbal learning, verbal memory, and divided attention. Although five measures reached statistically significant differences, controls outperformed injured athletes on 20 of the 23 summary scores. This finding suggests that cerebral concussions may disrupt a wide range of cognitive functions 48 hours postinjury. As the sample size continues to grow over time in this project, it is likely that a greater number of tests will reach statistical as well as clinical significance.
Also of interest is that the average scores of the injured athletes decreased from the 2-hour assessment to the 48-hour assessment on the HLI, which was one of the three tests that was found to significantly differentiate between the two groups. While the injured athletes' scores decreased, the control athletes' scores improved over the same time period, thereby widening the gap between injured and noninjured athletes. In other words, the control subjects demonstrated a practice effect whereas the injured athletes' performance deteriorated. This finding may be understood in the context of the biochemical cascade models that have been developed on animals. Hovda and colleagues 28 have described a neurochemical and metabolic cascade that occurs following mTBI. Within the first hour of injury and subsequently up to several days postinjury, the brain is hypothesized to be in a vulnerable state due to an acute increase in glucose metabolism and diminished regional cerebral blood flow. The reduction in blood flow is thought to be produced by the effects of Ca++ on hemodynamics. 28 The imbalance between glucose needs and available supply begins within minutes of the injury and then begins to recover over time (up to 10 days postinjury). An interesting parallel can be seen in our HLI data where the injured athletes experienced difficulty in verbal learning almost immediately postinjury. This difficulty increased by 48 hours postinjury and then gradually began to “catch up” to the noninjured data by 1 week. At 1 month postinjury, the two groups were indistinguishable. Unfortunately, we do not have data on intermediate time intervals such as 1 day postinjury, 5 days postinjury, 10 days postinjury, etc., to more fully understand the rates of cognitive deterioration and recovery after mTBI. Additional studies will be needed that use different timelines. Data from the various studies can then be combined to generate more continuous descriptions of this phenomenon.
This failure of the injured athletes to demonstrate a practice effect has implications for return-to-play decision making. As can be seen in Figure 3, at 1 month postinjury the control and injured groups do not differ from each other. However, both groups now have scores that are greater than those obtained at baseline. Given these data, it may be argued that a return to preseason baseline scores is not a sufficient indicator of “normal” functioning. In fact, the player may need to produce scores that exceed baseline scores on measures with known practice effects.
The deterioration in performance within 48 hours postinjury also has implications for the sequence of assessment. Although tests administered within minutes or hours of injury are useful, they may not be enough. It is quite possible that an athlete would score within the expected range at 2 hours but show a significant deterioration at 24 or 48 hours postinjury. This possibility has implications for the use of sideline assessments without follow-up serial testing.
The Post-Concussion Symptoms Scale revealed significantly more symptoms being reported by injured athletes when compared with controls at the 2-hour assessment. Injured athletes reported a greater frequency of headache, dizziness, nausea, balance problems, drowsiness, sensitivity to light and noise, feeling “slowed down” and “foggy”, memory problems, and problems concentrating. By the time athletes reached the 48-hour mark, there was no significant difference between injured athletes and controls in symptom reporting. This finding is important in light of the significant differences found in the neuropsychological tests at the same time period. Within this sample of athletes, only the neuropsychological test data differentiated the groups at 48 hours. These data indicate that without neuropsychological testing, it is possible to return a player to sport who reports feeling well, but continues to experience cognitive sequelae from the concussion. This premature return to play may have serious deleterious consequences for the athlete, particularly in light of the vulnerability model described earlier.
No multivariate group differences were found between the injured athletes and controls at 1 week postinjury. This result is consistent with those of Collins et al., 18 indicating that most players have returned to the same level of functioning as uninjured athletes by 1 week after injury. However, univariate tests conducted at 1 week postinjury revealed significant differences between injured athletes and controls on measures of working memory, reaction time, and sustained attention. Although no definitive conclusions can be drawn in the absence of a significant MANOVA, these findings suggest that some athletes may continue to experience cognitive disturbances 1 week (or more) after injury.
At 1 month postinjury, only one neuropsychological measure reached significance using ANOVA, with injured athletes performing better than their noninjured counterparts. Although the difference in means is clinically insignificant, the finding raises an interesting question. First, given that a MANOVA could not be conducted due to missing player data, it is quite possible that the finding is simply a Type I error and due solely to chance. Yet our clinical experiences with the athletes in the study have led us to observe that injured athletes are highly motivated to perform well on these tests; failure to perform well could keep them or take them out of play. Other than their intrinsic competitive drive (which is often challenged by these tests), the control athletes have little motivation to perform well. The 1-month testing is their fifth time being tested and they may be “tired of it.” As such, they did not perform poorly, they just did not perform as well as their more motivated counterparts. The issue of motivation in neuropsychological testing is important and will be addressed in another paper.
Taken together, the results of this study underscore the utility of neuropsychological testing prior to and following a sports-related mTBI. The data from these tests provide the sports medicine team with an objective index of cognitive functioning that can signal the return to preinjury levels of functioning. The neuropsychological data differentiated injured from control athletes better than using a report of symptoms alone. However, there are several limitations to this study. Although sizable relative to other prospective studies, the sample size in this study was limited. Additional data are being collected that in the years to come will provide a better understanding of the utility of neuropsychological tests. The data in this study were only analyzed at the group level. Although this approach is important in assessing the utility of neuropsychological tests, it obscures the significant individual variability that exists among the injured athletes. This variability was discussed earlier in light of the marked differences in standard deviations on most of the measures between the athletes with mTBI and their controls. Clinically, these indices of variability imply that there are injured individuals functioning normally at the 2-hour assessment and some functioning in the impaired range at the 1-month assessment. Group data are useful benchmarks, but return-to-play decisions must be made at the individual level. It is for this reason that baseline testing is essential in the neuropsychological assessment of injured athletes.
Group data also obscure the variability that exists between neuropsychological tests. As can be seen from the present data, different tests may be useful at different time periods. Clinical examination of individual player data indicates that cerebral concussions affect players in different ways. Some have memory disturbances, some have problems in learning, others have slow reaction times, some have physiologic symptoms and no cognitive decline, and some report no symptoms and have frank cognitive declines. These differences in impairment may reflect differences in the type (e.g., linear, rotational), location, and severity of the blow that caused the mTBI. To identify these differences, a battery of tests must be used since no single test has demonstrated the sensitivity or the specificity that is necessary for clinically relevant and empirically supported return-to-play decisions.
The authors would like to thank the following persons for their valuable assistance in data collection and management: David Sabsevitz, Cathy Elsinger, Kevin Kachin, Jim Chok, Tracey Shissler, Dimitrios Donavos, and John Furtado.
1. Kraus JF. Epidemiology of head injury. In: Cooper PR, ed. Head Injury
Baltimore: Williams & Wilkins, 1987:1–19.
2. Powell JW. Traumatic brain injury in high school athletes. JAMA 1999; 282:958–963.
3. Dick RW. A summary of head and neck injuries in collegiate athletics using the NCAA Injury Surveillance System. In: Hoerner EF, ed. Head Neck Injuries in Sports. Philadelphia: American Society for Testing and Materials, 1997.
4. Buckley WE. Concussions in college football. Am J Sports Med 1988; 16:51–56.
5. Wilberger JE. Minor head injuries in American football. Sports Med 1993; 15:338–343.
6. Report of the Ad Hoc Committee to Study Head Injury Nomenclature. Proceedings of the Congress of neurological Surgeons in 1964. Clin Neurosurg
7. McLatchie G, Jennett B. ABC of sports med: head injury in sport. Brit Med J 1994; 308:1620–1624.
8. Saunders RL, Harbaugh, RE. The second impact in catastrophic contact-sports head trauma. JAMA 1984; 252:538–539.
9. Spear J. Are footballers at risk for developing dementia? Intl J Geriatr Psychiatry 1995; 10:1011–1014.
10. Mayeux R, Ottman R, Tang MX, et al. Genetic susceptibility and head injury as risk factors for Alzheimer's disease among community-dwelling elderly persons and their first degree relatives. Ann Neurol 1993; 33:494–501.
11. Mortimer JA, French LR, Hutton JT, et al. Head injury as a risk for Alzheimer's disease. Neurology 1985; 35:264–266.
12. Quality Standards Subcommittee, American Academy of Neurology, Practice Parameter: The management of concussion in sports (summary statement). Neurology
13. Cantu RC. Return to play guidelines after a head injury. Clinics Sports Med 1998; 17:45–60.
14. Colorado Medical Society. Guidelines for the Management of Concussion in Sports
. Colorado Medical Society, Sports Medicine Committee, May 1990 (revised May 1991).
15. Nelson WE, Jane JA, Gieck JH. Minor head injury in sports: a new classification and management. Phys Sportsmed 1984; 12:103–107.
16. Torg JS. Athletic Injuries to the Head, Neck, and Face
. St. Louis: Mosby-Year, 1991.
17. Kelly JP, Rosenburg JH. Diagnosis and management of concussion in sport. Neurology 1997; 48:575–580.
18. Collins MW, Grindel, SH, Lovell MR, et al. Relationship between concussion and neuropsychologicalal performance in college football players. JAMA 1999; 282:964–970.
19. Lovell MR, Iverson, GL, Collins MW, et al. Does loss of consciousness predict neuropsychologicalal decrements after concussion? Clin J Sport Med 1999; 9:193–198.
20. Kirkpatrick PJ. Use of near-infrared spectroscopy in the adult. Philosophical Transaction of the Royal Society of London-Series B: Biological Sciences
21. Robertson CS, Chance B. A new application for near-infrared spectroscopy: Detection of delayed intracranial hematomas after head injury. J Neurotrauma 1995; 12:591–600.
22. Masdeu JC, Abdel-Dayem H, Van Heertum RL. Head trauma: use of SPECT. J Neuroimag 1995; 5(Suppl.1):S53–57.
23. Lewis DH. Functional brain imaging with cerebral perfusion SPECT in cerebrovascular disease, epilepsy, and trauma. Neurosurg Clinics NA 1997; 8:337–344.
24. James CA. Magnetic resonance angiography in trauma. Clin Neurosci 1997; 4:137–145.
25. Ono J, Harada K, Takahashi J, et al. Differentiation between dysmyelination and demyelination using magnetic resonance diffusional anisotropy. Brain Res 1995; 671:141–148.
26. Putukian M, Echemendia RJ. Managing successive minor head injuries: Which tests guide return to play? Phys Sportsmed 1996; 24:25–38.
27. Gennarelli TA. Mechanisms and pathophysiology of cerebral concussion. J Neurotrauma 1986; 1:23–29.
28. Hovda DA, Prins M, Becker DP, et al. Neurobiology of concussion. In: Bailes JE, Lovell MR, Maroon JC, eds. Sports-Related Concussion. St. Louis: Quality Medical Publishing, Inc. 1999:12–51.
29. Abreau F, Templer DI, Schuyler BA, et al. Neuropsychological assessment of soccer players. Neuropsychol 1990; 4:175–181.
30. Rimel RW, Giordanni B, Barth JT. Disability caused by minor head injury. Neurosurg 1981; 9:221–228.
31. Rimel RW, Giordani B, Barth JT, et al. Moderate head injury: completing the clinical spectrum of brain trauma. Neurosurg 1982; 11:344–351.
32. Levin HS, Amparo E, Eisenberg HM, et al. Magnetic resonance imaging and computerized tomography in relation to the neurobehavioral sequelae of mild and moderate head injuries. J Neurosurg 1987; 66:706–713.
33. Alves WM, Rimel RW, Nelson WE. University of Virginia prospective study of football-induced minor head injury: status report. Clin Sports Med 1987; 6:208–211.
34. Tysvaer AT, Lochen EA. Soccer injuries to the brain: a neuropsychological study of former soccer players. Am J Sports Med 1991; 19:56–60.
35. McLatchie G, Brooks N, Galbraith S, et al. Clinical neurological examination, neuropsychology, electroencephalography and computed tomographic head scanning in active amateur boxers. J Neurol Neurosurg Psychiatry 1987; 50:96–99.
36. Porter MD, Fricker PA. Controlled prospective neuropsychologicalal assessment of active experienced amateur boxers. Clin J Sport Med 1996; 6:90–96.
37. Haglund Y, Eriksson E. Does amateur boxing lead to chronic brain damage? A review of some recent investigations. Am J Sports Med 1993; 21:97–109.
38. Matser EJT, Kessels AG, Lezak MD, et al. Neuropsychological impairment in amateur soccer players. JAMA 1999; 282:971–973.
39. Putukian M, Echemendia RJ, Mackin RS. The acute neuropsychological effects of heading in soccer: a pilot study. Clin J Sport Med 2000; 10:104–109.
40. Brandt J. The Hopkins verbal learning test: Development of a new verbal memory test with six equivalent forms. Clin Neuropsychol 1991; 5:125–142.
41. Smith A. The Symbol Digit Modalities Test (SDMT) Manual
. Los Angeles: Western Psychological Services, 1982.
42. Williams M. Cognitive Behavior Rating Scale
. Odessa, FL: Psychological Assessment Resources, 1991.
43. Partington JE, Leiter RG. Partington's Pathway Test
. New York: The Psychological Service Center Bulletin, 1949.
44. Lezak M. Neuropsychological Assessment.
3rd. Ed. New York: Oxford University Press, 1995.
45. Stroop JR. Studies of interference in serial verbal reactions. J Exper Psychol 1935; 18:643–662.
46. The Psychological Corporation. Vigil/W Continuous Performance Test
. San Antonio: The Psychological Corp., 1994.
47. Barth JT, Alves WM, Ryan TV, et al. Mild head injury in sports: Neuropsychological sequelae and recovery of function. In: Levin HS, Eisenberg HM, Benton AL, eds. Mild Head Injury. New York: Oxford, 1989:257–275.
48. Lovell MR, Collins MW. Neuropsychological assessment of the college football player. J Head Trauma Rehabil 1998: 13;9–26.
© 2001 Lippincott Williams & Wilkins, Inc.