Effect of Education and Language on Baseline Concussion Screening Tests in Professional Baseball Players

Abstract

Objective: 

The purpose of the present study was to investigate the possible effects of sociocultural influences, specifically pertaining to language and education, on baseline neuropsychological concussion testing as obtained via immediate postconcussion assessment and cognitive testing (ImPACT) of players from a professional baseball team.

Design: 

A retrospective chart review.

Setting: 

Baseline testing of a professional baseball organization.

Participants: 

Four hundred five professional baseball players.

Independent Variables: 

Age, languages spoken, hometown country location (United States/Canada vs overseas), and years of education.

Main Outcome Measures: 

The 5 ImPACT composite scores (verbal memory, visual memory, visual motor speed, reaction time, impulse control) and ImPACT total symptom score from the initial baseline testing.

Results: 

The result of t tests revealed significant differences (P < 0.05) when comparing native English to native Spanish speakers in many scores. Even when corrected for education, the significant differences (P < 0.05) remained in some scores.

Conclusions: 

Sociocultural differences may result in differences in computer-based neuropsychological testing scores.

INTRODUCTION

Concussion and its related sequelae have continued to gain attention in recent years, both in the public and medical arenas. This interest has been driven in part by high profile cases seen in professional sports and in the media. The Centers for Disease Control and Prevention has estimated that between 1.6 and 3.8 million mild traumatic brain injuries occur on a yearly basis in the United States, with the majority attributed to concussions.¹ Although the recognition of a concussion is important, return-to-play decisions following a concussion may be even more vital. Once a concussion occurs, the likelihood of sustaining another concussion dramatically increases, especially if the first concussion has not completely resolved.² Therefore, it becomes crucial to prevent early return to play. However, when one considers the importance of sports in the lives of some individuals, it is a disservice to the athlete to withhold him or her longer than necessary based on a cookie-cutter or an outdated approach to concussion management. It is important to remember that concussed athletes may have such subtle signs and symptoms that may be overlooked by coaches, sports medicine personnel, clinicians, or even the athlete. Therefore, to make these crucial decisions, physicians, athletic trainers, and other health care professionals often rely on a battery of tools, including sports concussion computerized neuropsychological testing.

In the past 2 decades, there have been many advancements in our understanding of concussions, yet much remains unknown or poorly understood, not only in terms of the pathophysiology but also in terms of concussion evaluation. This is evidenced in the fact that over the years, at least 14 different return-to-play scales and 25 published injury-grading systems were created, most of which were founded on subjective clinical experience rather than evidence-based research.^3–5

The first International Symposium on Concussion in Sport was held in 2001 in Vienna to discuss the concerns of concussion diagnosis and management and listed neuropsychological testing as an important component of concussion management, although there was concern with the reliability and sensitivity of such testing batteries.⁶ The second and third International Symposiums on Concussion in Sport, that took place in Prague in 2004 and Vienna in 2008, respectively, upheld the endorsement of neuropsychological testing as a key component in concussion management.^7,8 Most recently, the fourth International Symposium on Concussion in Sport held in Zurich in 2012 reaffirmed the clinical value of neuropsychological testing because it contributes significant information on concussion evaluation.⁹ The 2012 Zurich Symposium recommended clinical neurological assessment for all athletes, which could include computerized neuropsychological testing and, in select athletes, formal neuropsychological testing performed and interpreted by a neuropsychologist.⁹

With such endorsement, several computer-based test programs for the assessment and management of sports-related concussion, most of which are commercially available, were developed. They provide multiple advantages including improved ease of administration, automated scoring, decreased practice effects, increased test-retest reliability, and a cost-effective alternative to the former paper-and-pencil testing.^10–13

Among the most widely used computer-based tests is the immediate postconcussion assessment and cognitive testing (ImPACT) test battery, which was developed for the assessment of sports-related concussion in high school, collegiate, and professional athletes.¹⁴ It consists of 3 main components: demographic data, neuropsychological tests, and the postconcussion symptom scale.⁵ The 6 neuropsychological tests are designed to evaluate aspects of cognitive functioning including attention, memory, processing speed, and reaction time. From the 6 tests, 4 composite scores are generated: verbal memory, visual memory, visuomotor speed, and reaction time. The validity of ImPACT has been studied, as well as the sensitivity and specificity for concussion in athletes.^5,15,16 Although normative data have been gathered and available for use, ImPACT is used ideally in the setting where a baseline test has been completed because the individual-specific data allow for tailored return-to-play decisions and work a means of assessing improvement in the postconcussion setting.

Immediate postconcussion assessment and cognitive testing was developed and normed in the United States on the basis of multiple university-based research studies.^17,18 Although it has been translated into 13 different languages, the fact that it was developed and tested in the United States could make it susceptible to social-cultural influences. Because of the ease of use of computerized neuropsychological testing, it can become dangerously easy to treat all athletes the same, allowing the computer to make the interpretation and therefore not take into account the social-cultural differences inherent to all persons. Prior studies have recognized the need for further investigation into possible cultural differences in ImPACT symptom scale and studied a need for culture-specific norms, especially when looking at symptom reporting.^19,20 Aside from the obvious language differences, values inherent to a specific culture can affect cognitive testing.¹⁹ Therefore, no test, including computerized neuropsychological testing, is exempt from possible social-cultural influences.

The purpose of the present study was to investigate the possible effects of sociocultural influences, specifically pertaining to language and education, on baseline neuropsychological concussion testing as obtained via ImPACT of players from a professional baseball team. There is limited research in terms of cultural equivalence in regards to computerized neuropsychological concussion testing.

METHODS

This study was reviewed and approved by the Institutional Research Board of the Medical College of Wisconsin. A retrospective chart review study design was used to compare baseline concussion test scores of professional baseball players belonging to a professional organization between January 2008 and September 2010. The ImPACT examination, a neuropsychological computer-based test, was used to obtain baseline scores. The ImPACT test was performed as a screening test as part of their standard preparticipation examinations, as recommended by the Major League Baseball labor rules. The subject included 405 professional baseball players. English was the first language for 304 players, and Spanish was the first language for 101 players. Thirty-five of the players had less than 9 years of formal education, 126 had between 9 and 12 years, and 240 had at least 1 year of formal education beyond high school (see Table for additional demographic details).

TABLE Demographics

Outcome Measures

The 5 ImPACT composite scores (verbal memory, visual memory, visual motor speed, reaction time, impulse control) and ImPACT total symptom score from the initial baseline testing were used for outcome measures. The ImPACT total symptom score is generated from the ImPACT symptom scale, which consists of a 22-item scale in which athletes subjectively report their present symptoms. In addition to the ImPACT scores, other variables that were analyzed included age, languages spoken, hometown country location (United States/Canada vs overseas), and years of education.

Data Analysis

Independent sample t tests were used to test for differences in ImPACT neurocognitive composite scores and the total ImPACT symptom scores between native English-speaking and native Spanish-speaking professional baseball players, while also taking into account their education and whether they could speak a second language.

RESULTS

The result of t tests revealed significant differences (P < 0.05) when comparing native English to native Spanish speakers, without accounting for education or a second language, in verbal memory, visual memory, visual motor speed, reaction time, and total symptom scores but not in impulse control (Figure 1). When corrected for education, the differences became insignificant in verbal memory, impulse control, and total symptom composite scores. However, significant differences (P < 0.05) remained in visual motor speed, reaction time, and visual memory composite scores in those with high school education without the ability to speak a second language (Figure 2). In the college-educated, regardless of the ability to speak a second language, the only statistically significant difference (P < 0.05) was found in visual motor speed composite score (Figure 2). Of note, all of the native Spanish college-educated subjects spoke English as second language, so we were not able to control for second language in the college age group. Although not statistically significant, a trend was observed in which with increasing education, composite scores improved (Figure 3).

FIGURE 1: Without taking education into account, significant differences exist in all composite scores except impulse control. Lang, language; Rxn, reaction. *Significant difference of P < 0.05.

FIGURE 2: Even with stratifying for education, differences remained in reaction time in the high school educated and in visual motor speed in both college-educated and high school–educated athletes. Coll, college; Edu, educated; Eng, English; HS, high school; Rxn, reaction; Sp, Spanish. *Significant difference of P < 0.05.

FIGURE 3: Trend can be observed that education does affect the composite scores. Rxn, reaction, yrs, years.

LIMITATIONS

The test-taking effort of the subjects was unknown. However, ImPACT test guidelines provide norms based on education to help identify individuals who are potentially trying to generate a low score. In the subject population, some of the composite scores were low, but only 1 test taker was found to have an impulse control above 30 seconds, which is the number above which ImPACT believes the test should not be considered valid.

Test takers' emotional and musculoskeletal injury status was also unknown. Fatigue and anxiety may negatively affect neuropsychological testing, as well as the presence of a musculoskeletal injury.^21,22

The subjects mainly spoke 2 languages: Spanish and English. Our observations and results may have been different if we compared other languages or regions of the world. Most of the native Spanish-speaking players came from the Caribbean and Latin America, areas that tend not to be as socioeconomically developed as other areas. In these areas, the quality of education and access to educational resources, to most individuals, is much lower than those found in the United States.

CONCLUSIONS

The study shows that native Spanish-only speakers and, to a lesser extent, native Spanish speakers with a second language and college education have a lower baseline testing performance in certain components of the ImPACT examination that may be the result of sociocultural differences. Although a higher level of education seems to somewhat help minimize this difference, education alone does not explain away the statistically significant differences in composite scores.

Trying to understand or research how specific sociocultural differences may affect ImPACT and other such examinations is a difficult and complex task. Important cross-cultural variables that should be taken into account include the value placed on computer use proficiency and test-taking familiarity, especially for computer-based testing. Depending on where the individual is from, familiarity with computers may not be as commonplace or accessible as it is in American society. Even if computers are accessible, computerized testing may not be common, and there is a definite learning curve when it comes to taking computerized examinations. One possible area of future research in eliminating this learning curve and minimizing sociocultural differences would be to use other forms of neurocognitive testing, such as the SCAT3 (Sport Concussion Assessment Tool, Third Edition), a paper test administered and interpreted by a health care professional.⁹

Local attitudes of the importance of working at speed may differ; this in turn could directly affect composite scores.¹⁹ These attitudes may be inherent to a culture's education techniques and values. What is taught and how it is taught is likely very different in other countries, including how knowledge is processed and assimilated. Critical thinking skills may be different when processing through cognitive examinations such as found on ImPACT. Some test-takers may be more used to rote memorization, whereas others have been taught to critically analyze information presented and react differently to the information. They may eventually come to the same outcome or result but may be quicker or slower to arrive at such conclusions due to differences in analytical processing or information synthesis.

Another possible influence is the issue with symptom reporting. Previous studies looking at the differences in psychological responses among the genders to athletic injury showed that male athletes often feel pressured by coaches and teammates to “play through pain.”^23,24 So when they are told they cannot play due to their injury, especially one that is many times not physically apparent, they tend to report more depressive or sad symptoms.^23,25 These studies were done on American athletes, and in our study, most of our players speaking only Spanish were from Central and South America, where the culture of machismo still is very prevalent. In such a culture, aggressive independence and toughness, especially mental, is not only valued but also encouraged. In keeping with this thinking, to be held out of competition for having an injury that one cannot physically “see,” such as a concussion, could be seen as a weakness. Therefore, when taking neurocognitive examinations, which may seem on the surface to be abstract, true effort may not be given or even symptom underreporting may occur secondary to the machismo effect.

Finally, one must consider the socioeconomic implications on education. The quality of schools and education in more affluent areas in general would be advantageous to the test-taker merely for the fact that more resources are available. Shuttleworth-Edwards et al²⁶ have done prior research using the Wechsler adult intelligence scale III in the South African context and found that the educationally disadvantaged populations had lower intelligent quotients when compared with age-equivalent and grade-equivalent educationally advantaged populations. In this case, even the processing speed index was lower, which is similar to our findings of slower reaction time in the native Spanish speakers. These very subjects had lower levels of education. It would seem that with increasing education, baseline test scores increase in certain areas, which makes sense as cognition improves as well. Register-Mihalik²⁷ found this to be true in college athletes who performed better on ImPACT processing speed composite score than high school athletes. So when considering normative data baseline testing, care must be taken to consider the individual's educational level. An individual with poor education could get lower scores than better-educated athletes with impairment or could be classified as “impaired” when comparing to normative baseline data.²⁸ One could postulate that with less education, concussion resolution may seem to resolve more slowly on neurocognitive testing. Although having a baseline test would be ideal, as it provides an individual-specific ImPACT score for future comparison, it may not be enough to overcome the educational disadvantage. That baseline test may then be obscured by the test-taker practice effect, which in theory would have improved performance on subsequent tests, therefore possibly obscuring continued concussive symptoms, resulting in premature return to play.¹³ Despite the apparent sociocultural differences, it is clinically advantageous to have baseline testing, especially for native Spanish-only speakers with less education, as the individual-specific data would be of greater benefit than the more generic normative data in the postconcussion setting.

In conclusion, no 2 concussions are alike; thus, medical personnel need to be cognizant that there are multiple factors at play with each athlete, which are evaluated. Computer-based neuropsychological testing is a tool that may help better individualize that evaluation, but it too is not above the same sociocultural influences that so subtly, but powerfully, stretch our limits of understanding of concussion management. Thus, as others have concluded, at the present time concussion management is still an art, and there is no magic test that can make the decision to return our athletes safely to play with 100% accuracy.