Recent research has explored the long-term neurodegenerative effects associated with recurrent concussion in athletes (4,5,9,13). Findings from a cohort of former professional football players found that the number of concussions sustained during their professional careers was associated with the rate of diagnosed clinical depression (8) and late-life cognitive impairment (7). A limitation of these studies was the use of a self-reported history of concussion. The reliability of athletes to recall and self-report a concussion history has never been quantified.
Imperfect recall can generate bias in epidemiologic studies when the proportion of events recalled is associated with the health end points of interest (such as neurodementia or depression). Associations observed in these studies may be spurious if athletes differ in their knowledge and recognition of concussion symptomatology in a manner that is associated with the health outcome of interest. Taking depression as an example end point of interest, if depressed athletes tend to overreport their concussions (or if nondepressed athletes tend to underreport their concussions), then the association between concussion and depression observed in these studies might be spurious and due to bias. Without data about the reliability of concussion self-report, there will always be a concern that the observed association between concussion and neurodementia/depression could be spurious and due to differential recall bias.
To address this issue, we readministered a concussion history questionnaire to this same cohort of former professional football players, at an approximately 10-yr interval after the original administration of the instrument. The numbers of self-reported concussions at the two administrations of the instrument were compared. The aims of this study were to 1) examine the reliability of the self-report history of concussion measure used in the former players cohort and 2) explore determinants of recall in the number of self-reported concussions, particularly with respect to self-reported changes in physical/mental health, during the past decade.
MATERIALS AND METHODS
The former players cohort comprised a diverse group of former professional football players, ranging from those who played before World War II to those who played into the late 1990s and early 2000s. The instruments used to obtain concussion and other health data in this cohort are detailed below.
Baseline general health survey.
The original 2001 general health survey (2001 GHS) of former National Football League (NFL) players was first sent to all living members of the NFL Retired Players Association (n = 3729) through the Center for the Study of Retired Athletes at the University of North Carolina at Chapel Hill. The GHS was initially mailed in May 2001, followed by remailings to nonrespondents in August 2001 and February 2002. We then completed the questionnaire with additional nonrespondents through telephone follow-up. A total of 2536 (68.7%) eventually completed the GHS. Informed consent was obtained from all subjects.
The 2001 GHS asked about musculoskeletal, cardiovascular, and neurological conditions that the former player experienced during and after his football career. It included questions about the number of concussions sustained during his professional football career (self-reported concussion history) and the prevalence of diagnosed medical conditions such as depression, Parkinson disease, Alzheimer’s disease (AD), and schizophrenia.
Assessment of concussion recall during professional career—2001 GHS.
The 2001 GHS defined concussion as “an injury resulting from a blow to the head followed by a variety of symptoms that may include any of the following: headache, dizziness, loss of balance, blurred vision, seeing stars, feeling in a fog or slowed down, memory problems, poor concentration, nausea, or throwing up.” Participants were reminded that getting “knocked out” or being unconscious did not always occur with a concussion. They were asked to report the total number of recalled concussions, the number of times they had been evaluated by a physician or athletic trainer for concussion, the number of times they lost consciousness, the number of times they had memory loss, and the number of times they had prolonged symptoms (defined as >4 wk), the number of times they were returned to competition on the same day after concussion, the number of times they were returned to competition on the same day after losing consciousness, the number of times they were returned to competition on the same day while still feeling dizzy or disoriented, the number of times they never reported a concussion to medical staff, and the total games missed because of any concussions. These items were asked separately for the high school, collegiate, and professional playing period. We also asked about playing careers in Armed Forces football.
Assessment of health and well-being—GHS 2001.
The 2001 GHS also included the Short Form 36 (SF-36) Measurement Model for Functional Assessment of Health and Well-Being, which addressed how well the former athlete functioned with activities of daily living (15). From the SF-36, we calculated a physical health composite score (PCS), which included scores of physical functioning, role physical, bodily pain, and general health, as well as a mental health composite score (MCS), which included scores of vitality, social functioning, role emotional, and mental health. A higher PCS and MCS score indicated higher physical and mental health, respectively.
Follow-up general health survey.
In August 2010, we sent a follow-up GHS (2010 GHS) to 2102 former players who had completed the 2001 GHS and with whom contact was not lost or who were not deceased (n = 434). The 2010 GHS featured the same measures as the 2001 GHS. We followed up with nonrespondents in October and December 2010. A total of 1291 former players (61.4%) completed and returned the 2010 GHS.
Statistical analyses.
We focused the analysis on the self-reported concussions during the professional playing career because the number of prior concussions has been the focus of scientific interest. We also assumed that injuries during the professional playing career would be recalled with higher accuracy than injuries during collegiate or high school play. We categorized self-reported concussion history data from 2001 and 2010 into three categories: 0, 1–2, and 3+ concussions. To ascertain reliability between the two data collection periods, we computed the weighted version of the Cohen κ coefficient (2,3).
Former players were also categorized on the basis of the change in the number of self-reported concussions. We used the two-sample paired t-test to compare the SF-36 mean scores and the changes from 2001 to 2010 in the SF-36 scores between the “greater number reported” and “same number reported” groups. Former players who reported no history of concussion in 2001 and later reported 1–2 or 3+ concussions in 2010 or self-reported 1–2 concussions in 2001 and later reported 3+ in 2010 were included in the greater-number-reported group. The same-number-reported group consisted of former athletes who self-reported the same number of concussions in both 2001 and 2010. Former athletes who reported 1–2 concussions in 2001 and later reported no concussions in 2010 or self-reported 1–2 or 3+ concussions in 2001 and later reported no concussions in 2010 were included in the “lower number reported” group. The institutional review board at the University of North Carolina at Chapel Hill approved this study.
RESULTS
Demographics.
Of the 1291 former players who returned GHSs, 392 were excluded because of incomplete self-report concussion history data from 2001 and/or 2010, leaving 899 patients (69.6%) for analysis. Demographic characteristics were similar among former players with complete self-report concussion history data and those without complete concussion history data (Table 1). However, former players with complete self-report concussion history data were younger than those without complete concussion history data. Also, the distributions of the most recent decade played, marital status, and work status differed between former players with and without complete concussion history data.
;)
TABLE 1: Demographics of 2010 GHS follow-up respondents with and without self-report concussion history data.
Reliability of self-report concussion history measure.
The comparison of self-report concussion history in 2001 and 2010 yielded a weighted κ coefficient with moderate reliability (Cohen κ = 0.485, 95% confidence interval (CI) = 0.440–0.523, P < 0.001). The unweighted κ coefficient was 0.428 (95% CI = 0.383–0.474, P < 0.001). Most of the discrepancy in the self-reports was due to an increase in the number of self-reported concussions. Of the 899 former players who had complete self-report concussion history data on both the 2001 and 2010 GHS, 6.6% (n = 59) reported fewer concussions during their professional careers in the 2010 GHS than in the 2001 GHS; 62.1% (n = 558) reported the same number of concussions during their professional careers in the 2010 and 2001 GHS; 31.4% (n = 282) reported more concussions during their professional careers in the 2010 GHS than in the 2001 GHS (Table 2).
TABLE 2: Comparison of self-report concussion history of former professional football players in 2001 and 2010 GHS.
SF-36 scores and change in SF-36 scores from 2001 to 2010.
PCS and MCS SF-36 measures from the 2001 GHS and the 2010 GHS, as well as the change in PCS and MCS scores from the 2001 and 2010 GHS, were compared between the greater-number-reported and same-number-reported groups (Tables 3 and 4). The greater-number-reported and same-number-reported groups did not differ in average PCS and MCS scores from the 2001 GHS (PCS, P = 0.102; MCS, P = 0.566). However, the greater-number-reported group reported lower average PCS and MCS scores in the 2010 GHS than the same-number-reported group (PCS: t = −2.1, P = 0.011; MCS: t = −2.0, P = 0.045). Furthermore, the greater-number-reported group had a greater average drop in MCS score from the 2001 GHS to the 2010 GHS, relative to the same-number-reported group (t = −2.7, P = 0.008). Curiously, the greater-number-reported group tended to have a greater average drop in PCS from the 2001 GHS to the 2010 GHS than the same-number-reported group (t = −1.1, P = 0.277).
TABLE 3: Comparison of 2001 and 2010 GHS physical health composite scores (PCS) by change in self-report concussion history of former professional football players.
TABLE 4: Comparison of 2001 and 2010 GHS mental health composite scores (MCS) by change in self-report concussion history of former professional football players.
DISCUSSION
This study, the first to quantify the reliability of concussion reporting in former athletes, found that the self-report history of concussion measure was moderately reliable, given the categories that were used in previous analyses. We also found that approximately one in three former professional football players recalled more playing career concussions in 2010 than in 2001. The increase in self-report is particularly interesting, given the 9-yr extension of recall and the lack of additional exposure time (due to all participants remaining retired from professional sports). Furthermore, those who recalled more concussions in 2010 than in 2001 had lower 2010 SF-36 scores and reported greater drops in SF-36 scores in 2010 from 2001, suggesting that changes in health status (e.g., onset of depression or neurodementia) may be differentially associated with recall of concussion. Additional to these individual changes are changes in the social environment due to the increased media coverage of concussions. Although we are unable to quantify this societal effect in the current study, it also has likely played an important role.
Assessing the reliability of the self-report concussion history measure.
The guidelines of Landis and Koch (10) are widely used to interpret κ statistics: according to their criteria, our κ values are in the “moderate” agreement range. Although the guidelines of Landis and Koch have no empirical basis, they are widely accepted. On the basis of categorization of concussion history into three categories (0, 1–2, 3+), 62% of players fell into the same category in 2010.
However, approximately one in three retired NFL players self-reported more concussions in 2010 than in 2001. It is difficult to understand the increase in self-reported concussion data in the retired player cohort in the absence of additional insight from the participants that might explain this systematic increase. Many factors could potentially influence the ability of athletes to accurately self-report their concussion history. These include the usual issues inherent in any self-report injury data, such as imperfect recall (resulting in a downward bias or undercount) and telescoping of injury events from before the professional playing career period (resulting in an upward bias or overcount). In addition, circumstances at the time of the event, such as the skill and knowledge of the clinician(s) treating the injured athlete, and the quality of the clinician–athlete communication could also influence the quality of the self-reported concussion history.
It is possible that news and sports media may have served as a social–environmental factor that contributed to this increase in the self-reported number of playing career concussions. Increased media reporting of concussion injury could have sensitized the former athletes to the significance of concussion and altered their recall of the injuries they sustained during their professional careers. This could create differential recall bias if it resulted in a more complete recall of concussion details in those who have worse mental health. Differential recall bias could also result from former players aging, thus beginning to dwell more on their health and consequently attributing life changes more to concussions. At the same time, the fact that SF-36 mental health scores had a higher mean decrease in those who increased their reported of concussion suggests the potential for differential recall bias due to mental health. As formers players’ mental health worsened, they tended to dwell more on the details of their playing career and recalled more concussions. However, it is difficult to estimate the magnitude of the bias in the absence of any “gold-standard” measure of concussion history.
Media’s influence on concussion history recall.
During the past few years, the presence of stories related to the safety of athletes has increased in newsprint (e.g., The New York Times) and sport-specific media (e.g., ESPN, Sports Illustrated) (12). Three competing theories could explain how the media affect and influence how retired NFL players perceive their concussion histories.
“Agenda setting” (11) argues that the media promote selected topics to prominence through heavy coverage and cause the public to thus perceive those topics as important. Media coverage of concussions made the issue of concussion injury salient. In effect, the retired NFL athletes, particularly those who are heavy viewers, are made more aware of the possibility of concussion injuries occurring during their professional careers. As a result, they may be more likely to classify past hard hits and injuries as concussions.
Other possible phenomena cannot be ruled out, however. The “social cognitive theory” (1) suggests that individuals gain knowledge through the observation of others, including through media. If the retired NFL players’ media exposure served as a means for education regarding concussion symptomatology and identification, then self-reported concussion injury recall would have naturally increased to a more accurate number. The “cultivation theory” (6) approaches media influence in manners similar to agenda setting. However, the cultivation theory focuses on influencing what viewers think, not what they themselves think about. The cultivation theory stipulates that heavy viewers of media are more likely to adopt the “media view” of events. Such a theory could apply with the retired NFL players reporting more concussions given that (a) retired NFL players who self-reported a greater number of concussions in 2010 than in 2001 were heavy media viewers and (b) media had sensationalized concussion coverage to the point that (c) the retired NFL players in question were led to believe they had more concussions than they actually sustained.
Future directions.
We investigated the reliability of self-reported concussion history data, but we are unable to ascertain the validity of the data. Future research needs to examine the validity of self-report concussion history, especially when recall involves injuries that occurred years, if not decades, prior. Researchers investigating associations between playing era concussions and health outcomes may need to develop instruments and methodologies that are robust to recall bias. One approach would be to compare self-report data to an athlete’s medical records during his/her playing career. However, this would have limitations because some concussions may have gone unreported or undocumented. It would also be helpful to understand how athletes perceive media coverage of concussions and how such coverage affects their understanding and awareness of concussions and how this knowledge affects their behaviors and decision making during sports play. It would be helpful to know if the player self-reported more concussions because of media coverage making concussion injury a salient issue through increased coverage (i.e., reporting that a player sustained a concussion but nothing regarding identifying a concussion) or media advocating concussion awareness and education in terms of identifying a concussion.
CONCLUSIONS
Because of concerns raised about the quality of self-reported concussion history (14), we considered that it was important to evaluate the reliability of our concussion history instrument. We found that self-reported concussion history had moderate reliability in former professional football players, on the basis of two administrations of the same instrument 9 yr apart, with 62% of players falling into the same data grouping on both administrations of the instrument. This suggests that the previously observed associations between concussion and neurodementia/depression may be relatively robust to recall bias. However, this cannot be guaranteed without recourse to some external source of valid data. Furthermore, increases in concussion reporting were associated with declines in SF-36 mental and physical composite scales, suggesting a possible relationship between concussion reporting and changes in health status. It would be helpful to investigate whether increased recent media coverage of concussions has influenced player attitudes and behaviors toward concussion awareness and reporting.
The authors thank the staff at the Matthew A. Gfeller Sport-Related Traumatic Brain Injury Research and the Center for the Study of Retired Athletes for their input, particularly research associates Amy Matthews and Candice Goerger.
The authors also thank the support of the NFL Players Association. The Matthew A. Gfeller Sport-Related Traumatic Brain Injury Research and the Center for the Study of Retired Athletes are funded by private donations and various grant funding agencies interested in understanding the late-life consequences and benefits of playing sports.
There have been no involvements that might raise the question of bias in the work reported in the conclusions, implications, or opinions stated. The authors report no conflicts of interest. The authors of this article appearing in this journal are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the American College of Sports Medicine.
REFERENCES
1. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs (NJ): Prentice Hall; 1986. p. 544.
2. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960; 20: 37–46.
3. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968; 70: 213–20.
4. De Beaumont L, Theoret H, Mongeon D, et al.. Brain function decline in healthy retired athletes who sustained their last sports concussion in early adulthood. Brain. 2009; 132: 695–708.
5. DeKosky ST, Ikonomovic MD, Gandy S. Traumatic brain injury—football, warfare, and long-term effects. N Engl J Med. 2010; 363: 1293–6.
6. Gerbner G., Gross L, Morgan M, Signorielli N. Living with television: the dynamics of the cultivation process. In: Bryant J, Zillman D, editors. Perspectives on Media Effects. Hillsdale (NJ): Lawrence Erlbaum Associates; 1986. p. 17–40.
7. Guskiewicz KM, Marshall SW, Bailes J, et al.. Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery. 2005; 57: 719–26.
8. Guskiewicz KM, Marshall SW, Bailes J, et al.. Recurrent concussion and risk of depression in retired professional football players. Med Sci Sports Exerc. 2007; 39 (6): 903–9.
9. Guskiewicz KM, McCrea M, Marshall SW, et al.. Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA Concussion Study. JAMA. 2003; 290: 2549–55.
10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–74.
11. McCombs M, Shaw D. The agenda-setting function of mass media. Public Opin Q. 1972; 36: 176–87.
12. McCrory P. Future advances and areas of future focus in the treatment of sport-related concussion. Clin Sports Med. 2011; 30: 201–8.
13. McCrory P. Sports concussion and the risk of chronic neurological impairment. Clin J Sport Med. 2011; 21: 6–12.
14. Valadka AB. Comment on “Association between recurrent concussion and late-life cognitive impairment in retired professional football players.” Neurosurgery. 2005; 57: 725.
15. Ware JE Jr, Gandek B, Kosinski M, et al.. The equivalence of SF-36 summary health scores estimated using standard and country-specific algorithms in 10 countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998; 51: 1167–70.
