This is the first in a series of reports that are intended to build an evidence base for a definition, diagnostic criteria, and prognostic indicators for concussion. The initial goal of Step 1 was to derive an evidence-based definition for concussion using data from the highest-quality published literature. Currently, there is no definition for concussion that is being uniformly applied in clinical and research settings.1
A general approach for deriving a definition was constructed: A systematic review of the literature would be conducted to identify the prevalence of signs, symptoms, neurologic deficits, and cognitive deficits (SSDs) after potential concussive events (PCEs) in athletic, hospital, and military populations. (For the purposes of this report, prevalence is the proportion of the sample with the particular sign, symptom, or deficit being discussed at the particular time point of measurement. In reports of data from studies used to support the conclusions, the absolute prevalence, that is, the difference between the prevalence of the sign, symptom, or deficit in the potentially concussed group compared with control subjects [either self or control group], is used.) The goal was to determine who (which patients) had what (which signs, symptoms, and deficits) when (at what specific time points). The product would be analyses of the frequencies of indicators of concussion at specific time points and of their associations. Using only high-quality studies, the product would provide a simple but concrete evidence-based foundation on which to develop a definition.
A multidisciplinary Panel of Technical Experts was assembled, and the project was initiated with a preliminary survey of the current literature about concussion. The survey revealed pervasive circularity in the relationship between case identification and outcome reporting for individual studies. That is, in most of the published literature, criteria for a definition of a case of concussion were prespecified, and then patients were selected on the basis of those criteria. Thus, data on the prevalence of the criteria as outcomes in those samples are overestimated and are not useful for deriving a definition (eg, not useful for knowing “who had what, when”). There is, however, a subset of studies that use broadly inclusive case definitions. Their samples consist of participants involved in some PCE with or without loss of consciousness (LOC), with or without posttraumatic amnesia (PTA), with at least 1 symptom such as disorientation, headache, nausea, dizziness, blurred vision, or altered mental state. Operationally, such a definition would include almost any individual involved in some form of PCE. For this report, only publications with the most inclusive case definitions were used in the analyses. (Complete details on how studies were selected are provided in the Methods section of this document.) The benefit of this approach is an increased likelihood that most of the true concussions were included in the samples; the downside is an increased likelihood that nonconcussed participants were also included in the samples.
In addition, only publications that met prespecified design characteristics and a threshold level of potential for bias were included in the analyses (see the Methods section). Therefore, the final library consists of 26 studies that provide sufficient replication of data to endorse only 4 parameters as candidate attributes of a definition for concussion. Furthermore, although current research about imaging and biomarkers shows promise, it falls short of providing evidence to support use of a physiologic marker in a definition. As a result, there are insufficient data to derive a definition for concussion at this time. Thus, the product of Concussion Guidelines Step 1 is a Systematic Review of Prevalent Indicators. The revised goals of Step 1 of the Concussion Guidelines Project were to conduct a systematic review of the literature on the occurrence of signs, symptoms, and neurologic and cognitive deficits associated with concussion; to apply rigorous inclusion/exclusion and quality assessment criteria; to include only the studies with design and quality characteristics that would allow at least moderate confidence in the findings; and to present conclusions about the most prevalent indicators of concussion.
The obvious must be stated at this point. Without knowing what concussion is, that is, without a definition, this review (“Concussion Guidelines Step 1: Systematic Review of Prevalent Indicators”) leaves the unanswered question, “Prevalent indicators of what?” What is concussion? The inclusion criteria for the literature review probably provide a corral within which concussion exists; it is likely the corral also contains nonconcussed individuals and those with more serious brain trauma; it also may exclude some versions of concussion.The corral consists of the following attributes, in which concussion:
- is a change in brain function,
- follows a force to the head,
- may be accompanied by temporary LOC,
- is identified in awake individuals, and
- includes measures of neurologic and cognitive dysfunction.
Thus, this report provides evidence-based data on indicators of a phenomenon, as yet undefined, called concussion. This report is not a guideline; it is the first step, the foundation for evidence-based guidelines. It is the mandate of this group to derive a definition, guidelines for diagnosis and prognosis, and, if possible, treatment guidelines for concussion using research for which there is at least moderate confidence in the findings. To that end, future work will include ongoing review of new studies, reanalysis of existing data sets with a focus on the prevalence of indicators and their associations with objective measures, design and validation of diagnostic and prognostic models, and generation of new prospective studies targeting gaps in information that must be filled.
The following report is a summary of a comprehensive review and analysis process. The detailed analysis, with supporting tables and graphs, can be found in the Supplemental Digital Content (available at https://links.lww.com/NEU/A633, https://links.lww.com/NEU/A634, https://links.lww.com/NEU/A635, https://links.lww.com/NEU/A636, https://links.lww.com/NEU/A637, https://links.lww.com/NEU/A638, https://links.lww.com/NEU/A639, https://links.lww.com/NEU/A640, https://links.lww.com/NEU/A641, https://links.lww.com/NEU/A642, https://links.lww.com/NEU/A643, https://links.lww.com/NEU/A644, https://links.lww.com/NEU/A645, https://links.lww.com/NEU/A646, https://links.lww.com/NEU/A647).
Structure of the Research Team
Subgroups of the research team include an Executive Committee, a Methods Group, and a Panel of Technical Experts. The Executive Committee includes Jamshid Ghajar, MD, PhD; Andy Jagoda, MD; Silvana Riggio, MD; Nancy Carney, PhD; Lisa McGuire, PhD; and Victor Coronado, MD. The Methods Group, from Oregon Health & Science University, includes Nancy Carney, PhD; Cynthia Davis-O'Reilly, BSc; Amy Huddleston, MPA; Nora Helfand; Steven Bedrick, PhD; Tracie Nettleton, MS; and Hugo du Coudray, PhD. Members of the Panel of Technical Experts are listed in Supplemental Digital Content 1, Task Force (https://links.lww.com/NEU/A633). There is no formal hierarchical relationship among the 3 subgroups.
Panel of Technical Experts
Individuals with clinical and research expertise in concussion who also represent stakeholder organizations were invited to participate as members of a Panel of Technical Experts. Also included were representatives from the Department of Defense and the Centers for Disease Control and Prevention. Six meetings were held at 3- to 4-month intervals. At the first meeting, recommendations for the parameters of the project were presented, and the panel provided feedback. The parameters were revised according to panel recommendations, and the revised scope document was circulated after the meeting for panel endorsement.
Role of the Funder
This project was funded by the US Army Contracting Command, which was represented on the Panel of Technical Experts. In addition, the Brain Trauma Foundation provided funding for 2 meetings of the Panel of Technical Experts.
Objectives and Scope
The objective was to conduct a systematic review of the literature about the SSDs associated with concussion. The goal was to identify the most prevalent indicators of concussion and their associations. To that end, the literature search was wide and inclusive, and the inclusion/exclusion criteria were highly specific.
Key questions for the review were articulated to provide a structure for acquiring data on the signs, symptoms, and deficits of concussion:
- Key Question 1. What are the most common signs, symptoms, and neurologic and cognitive deficits within 3 months after a PCE?
- Key Question 2. Does the presence of signs, symptoms, and deficits within 3 months of a PCE vary by demographics, premorbid conditions, comorbidities, mechanism of injury, case definition, or other factors independent of the PCE?
- Key Question 3. What is the association between different signs, symptoms, and deficits or between the same signs, symptoms, or deficits at different time points for the same patient after a PCE?
- Key Question 4. What is the relationship between signs, symptoms, and deficits and imaging or biomarkers after a PCE?
Literature Search and Review
A PhD-level research librarian conducted an electronic search from 1980 to September 2012 in Medline, Sports Discus, PsychINFO, and Cochrane. Rationale for the beginning date included limiting to literature generated after the introduction of computerized tomography (CT) scan technology and scoping the project to emphasize the most current data. Reference lists of review articles were compared with the list captured in the electronic search, and abstracts for those not captured were acquired and reviewed for eligibility. Publications referred by colleagues or by clinical expert participants were also reviewed and included if eligible. The publications not acquired with the electronic search were used to rewrite the original strategy and to rerun the search to acquire other studies that might have been missed (see Supplemental Digital Content 2, Search Strategy, https://links.lww.com/NEU/A634).
Abstracts were read by 2 assessors from the Methods Group. The dual coding was compared, and discrepancies were resolved through consensus or by a third reviewer. Full-text publications were acquired for all included abstracts. One assessor read each publication and specified its inclusion/exclusion status. A second assessor, not blinded to the specification, read the publication to confirm or disagree. Discrepancies were resolved through consensus or by a third reviewer.
Supplemental Digital Content 3, Data Points From Abstraction Instrument (https://links.lww.com/NEU/A635), contains the data points abstracted from each included publication. One person from the Methods Group performed the primary abstraction, and a second person checked the work. Data for the category “ascertainment of signs/symptoms/neurologic deficits/cognitive deficits” were abstracted only for the publications rated as medium potential for bias (see the Quality Assessment section).
The focus of the categorization of the quality of individual studies was the assessment of potential for bias and confound. An instrument developed by investigators of the Oregon Evidence-Based Practice Center was used for rating potential for bias and confound in observational studies (see Supplemental Digital Content 4, Instrument to Assess Potential For Bias and Confound, https://links.lww.com/NEU/A636). The instrument addresses 7 domains:
- Selection bias
- Bias resulting from missing data
- Ascertainment bias related to case definition and identification
- Ascertainment bias related to case assessment
- Ascertainment bias related to SSD description and evaluation
- Ascertainment bias related to SSD assessment
Scoring of individual domains was “yes” (yes, potential for bias was minimized), “no” (no, potential for bias was not minimized), or “unclear” (insufficient information to know if potential for bias was minimized).
Each article was assigned an overall potential for bias rating. The overall rating takes into consideration the scores on the individual domains but is not an additive process using the individual scores. In assigning the overall score, the assessor considers the existing flaws of each publication in the context of the purpose of the study and in relation to the strengths of the publication. Scoring of overall potential for bias was low (low potential for bias), medium (medium potential for bias), or high (high potential for bias).
Two members of the Methods Group, blinded to each other's work, used the instrument in Supplemental Digital Content 4 (Instrument to Assess Potential For Bias and Confound, https://links.lww.com/NEU/A636) to rate each publication. Ratings were compared and discrepancies resolved through consensus or by a third reviewer.
Principles Used to Identify Evidence for Prevalent Indicators
Throughout the rest of this report, information across studies is combined to answer the key questions and to derive conclusions. Principles were used for combining evidence and assessing the overall quality of a body of literature outlined in the 2010 publication, “Grading the Strength of a Body of Evidence When Comparing Medical Interventions: Agency for Healthcare Research and Quality and the Effective Health Care Program.”2 Specifically, individual publications were assessed for risk of bias and confound, and information from studies of the highest quality was synthesized. When the data from multiple studies about a specific measure were inconsistent across studies, the data were not used to support the conclusions. Information derived from a single study that was not replicated was not used to support the conclusions.
In sum, the following rules were used to identify data to specify prevalent indicators of concussion: (1) Findings must be from >1 independent sample (replicated), from studies of medium or low potential for bias, from studies with inclusive case definitions, and from studies with fixed time points for reporting disease and outcome measures. (2) Findings from single studies were reported but not used in the conclusions. (3) Findings from multiple studies with contradictory information were reported but were not used in the conclusions.
Detailed information about definitions, inclusion/exclusion criteria, the protocol for identifying studies with inclusive case definitions, and the algorithm for assessing the utility of data can be found in Supplemental Digital Content 5, Methods (https://links.lww.com/NEU/A637).
Final Library of Included Publications
The multiple electronic searches and hand search methods yielded 5592 abstracts of potentially relevant studies. Of those, 1362 full-text publications appeared to meet inclusion criteria and were acquired and read for eligibility. Two hundred thirty-one met the prespecified criteria, and of those, 62 publications were rated as medium potential for bias (see Evidence Tables, Supplemental Digital Contents 6 and 7, https://links.lww.com/NEU/A638, https://links.lww.com/NEU/A639). Of the 62 publications, 26 had inclusive case definitions, reported data at fixed time points relevant to 1 or more of the key questions, and were included in the analysis. Eleven independent samples from 8 publications contributed data for the conclusions. Table 1 lists the 26 studies used in the analysis for this report. Cells indicate which study contributed to which key question and to the conclusions.
Prevalent Indicators: Evidence Derived From Multiple Independent Samples
The SSDs at the time points indicated that constitute the prevalent indicators are what was found in the existing literature from studies that met the prespecified inclusion criteria and adhered to standards for minimizing potential for bias and confound. The absence of evidence for other impairments at other time points must not be interpreted as evidence of no impairment. The evidence for these conclusions should be used as a starting point from which to identify missing information as targets for future investigation.
Sample Characteristics of Studies Used for the Conclusions
Eleven independent samples (3 independent samples are reported in one publication by Broglio et al3 and 2 in one publication by Field et al10) met the criteria specified in the Methods section and contributed data to the conclusions (Table 1). All were conducted in athletic settings; 5 included adults only,3,5,6 1 included adolescents only,15 and 5 included adults and adolescents.10,13,14,28
Evidence-Based Indicators for Concussion
The list of evidence-based indicators for concussion is shown in Table 2.
A summary of the evidence derived from single studies, which may be indicators of concussion, can be found in Supplemental Digital Content 8, Evidence From Single Studies (https://links.lww.com/NEU/A640).
Key Question 1 Results
Key Question 1: What are the most common signs, symptoms, and neurologic and cognitive deficits within 3 months after a PCE? (See Evidence Table, Supplemental Digital Content 6, https://links.lww.com/NEU/A638).
Details of the analysis and results can be found in Supplemental Digital Content 9, Analysis (https://links.lww.com/NEU/A641).
Fourteen studies containing data on signs associated with a PCE met the criteria for this analysis and were included as evidence for this section (see Table, Supplemental Digital Content 10, https://links.lww.com/NEU/A642). 3,6-9,12-17,19,25,28 Thirteen were studies of athletes; 6 samples were adults, 5 were adolescents, and 3 mixed adult and adolescent subjects. The fourteenth study included adult and pediatric patients in a hospital environment.9 A total of 1007 participants were assessed. In 7 studies, 381 subjects served as their own controls with preinjury baseline tests. Across 6 studies, 381 PCE subjects were compared with 212 control subjects, and for 1 study, 20 subjects both served as their own controls and were compared with 13 control subjects.
The prevalence of LOC ranged from 1% to 14.3% (evidence from 10 studies).3,6-8,13-15,19,25,28 The prevalence of PTA ranged from 2% to 29.7% (evidence from 7 studies).7,13-16,19,25 The prevalence of retrograde amnesia ranged from 7.4% to 53.3% (evidence from 5 studies).7,13,15,19,28 The prevalence of disorientation/confusion ranged from 18% to 44.7% (evidence from 3 studies).13,15,28
One study containing data on symptoms after a PCE met the criteria for this analysis and was included as evidence for this section.17 Seven symptoms (headache, dizziness, blurred vision, nausea, double vision, noise sensitivity, and light sensitivity) were measured at 2 hours after injury in a sample of 28 adult athletes and compared with the same symptoms in 28 uninjured, matched control subjects.
Statistical or clinical significance was not reported. Proportions of PCE and control subjects with symptoms and the absolute prevalence are reported in Table 3. Absolute prevalence exceeded 50% for headache, dizziness, blurred vision, and nausea. Supplemental Digital Content 11 (see Table, https://links.lww.com/NEU/A643) lists studies included in this review and reasons why symptoms data could not be used in the analysis.
Four publications containing data on neurologic deficits after a PCE met the criteria for this analysis and were included as evidence for this section (see Table, Supplemental Digital Content 12, https://links.lww.com/NEU/A644). 3-5,20 The only neurologic function tested was balance. All samples were adult athletes, and samples ranged in size from 26 to 150 participants. A total of 266 participants were assessed. In 3 studies, 116 subjects served as their own controls with preinjury baseline testing.3-5 In the fourth study, 94 PCE subjects were compared with 56 uninjured control subjects.20
For the publications included in this analysis, measurement times ranged from immediately after the event to 7 days after injury. A total of 20 measures of function using 11 neurologic tests were performed during that time span. Ten of the tests were conducted with the NeuroCom Sensory Organization Test (SOT). Results of the SOT are reported by composite, somatosensory ratio, visual ratio, and vestibular ratio, as well as by testing condition as well as by 6 distinct balance challenge conditions. The eleventh test was the Balance Error Scoring System. (Although Cavanaugh et al5 analyzed SOT data, they did not use the SOT manufacturer's software as their metric).
Of the 20 measures of balance, 12 (60%) showed clinically significant differences between PCEs and comparators within 1 week of injury. For 5 measures, the results are equivocal (see the Definitions section of Supplemental Digital Content 5, https://links.lww.com/NEU/A637), and for 3 measures, there was no difference. Thirty-one percent of the sample tested immediately after the event showed clinically significant decrements in function. The prevalence of decrements at 1 day ranged from 23.8% to 36.5%. By day 2, significant decrements persisted in 8 of 14 tests (57%), and the 1 test taken at day 7 showed no difference.
The prevalence of balance deficits within 2 days ranged from 23.8% to 36.5% within 24 hours of injury and decreased to between 19.2% and 24% by day 2. For balance taken at fixed time points between days 1 and 7, 60% of the measures indicated decrements in function in PCE subjects compared with comparators.
Nine publications containing data on cognitive deficits after a PCE met the criteria for this analysis and were included as evidence for this section (see Table, Supplemental Digital Content 13, https://links.lww.com/NEU/A645). 3,6,9,10,14,15,18,19,28 (In the Broglio et al3 report, results from 3 independent samples were reported [n = 23, 28, and 24]. Thus, samples sizes for this study vary in this report.) One of the 9 reported on hospital patients9; the other 8 took place in athletic environments. Sample sizes ranged from 16 to 122 participants. A total of 604 participants who had sustained a PCE were assessed. There were 720 control participants; 444 served as their own controls in pretrauma baseline testing, and 276 were from “other injury” or “no injury” comparison groups. Measurement times ranged from immediately after the event to 7 days after injury. A total of 100 measures of function using 27 cognitive tests were performed during that time span.
The prevalence of deficits identified with measures of reaction time ranged from 41.7% to 71.4% within 24 hours of injury and persisted through 2 days after injury to a significant degree, although the exact prevalence is not known beyond day 1 (evidence from 3 studies, 3 measures, 6 testing time points).3,14,28
Attention/Processing Speed/Working Memory
The prevalence of deficits identified with measures of attention/processing speed/working memory, reported in the form of reliable change index (see Supplemental Digital Content 14, Reliable Change Index, https://links.lww.com/NEU/A646), ranged from 0% to 30.4% to 50% to 52.2% within 24 hours of injury, with no evidence that they persist beyond this time point (evidence from 4 studies, 6 measures, 15 testing time points).3,6,18,19
Prevalence of deficits identified with measures of memory ranged from 0% to 20.8% to 39.1% to 41.7% within 24 hours of injury, although the exact prevalence is not known beyond day 1 (evidence from 8 studies, 12 measures, 53 testing time points).3,6,10,14,15,18,19,28
Prevalence of deficits identified with measures of executive function ranged from 0 to 34.8%, to 52.2% within 24 hours of injury, with no evidence that they persist beyond this time point.
Prevalence data are not available for deficits identified with measures of motor/sensory function.
Global Cognitive Measures
Prevalence data are not available for deficits identified with global cognitive measures.
Results of Tests of Cognitive Deficits Across Time
Table 4 shows the number of tests conducted at each time point, the proportion of tests that found differences between PCE and comparators, and the number of studies that contributed data at each time point. Time points were removed for which only 1 study and a limited number of tests contributed data. Results are displayed in Figure 1. As can be seen, the proportion of tests that showed decrements in function decreased from 58% on day 1 to 8% on day 7.
Results of Tests of Cognitive Deficits Across Domains
Table 5 shows the number of tests conducted for each cognitive domain, the proportion of tests that found differences between PCE and comparators, and the number of studies that contributed data at each time point. One domain (global measures) for which only 1 study contributed data was removed. Results are displayed in Figure 2. Eighty-three percent of measures of reaction time taken between 1 and 7 days after injury indicated decrements in function for this domain in PCE subjects vs comparators, 43% of measures of memory, and 29% of measures of attention/processing speed/working memory.
Key Question 2 Results
Key Question 2: Does the presence of signs, symptoms, and deficits within 3 months of a PCE vary by demographics, premorbid conditions, comorbidities, mechanism of injury, case definition, or other factors independent of the PCE? (See Evidence Table, Supplemental Digital Content 6, https://links.lww.com/NEU/A638).
Four publications met the criteria and are included as evidence for this question.6,8,13,14 All took place in athletic environments: 2 with adults and 2 combining adults and adolescents. Two studies compared differences on outcome measures between players with and without previous concussions.6,13 Collins et al6 also assessed the influence of learning disability on outcome. One study8 assessed sex differences, and 1 study14 assessed differences in outcomes between white and black subjects.
A detailed analysis of the data from the included studies on subgroup differences can be found in Supplemental Digital Content 9, Analysis (https://links.lww.com/NEU/A641), summarized as follows:
Evidence from 1 study suggests that athletes who sustained prior concussions had lower preinjury baseline scores on 2 of 9 (22%) neuropsychological tests (Trails B and Symbol Digit) than those with no history of concussion.6 A second study suggests that the odds of sustaining a 14-point drop in the ImPACT Memory Index within 5 days of injury are 7 to 8 times greater for athletes who sustained prior concussions than those with no history of concussion; a difference was found in 1 of 3 tests (33%) for this study.13
One study suggests no influence of learning disability on preinjury baseline neuropsychological tests among male athletes.6
One study suggests that on 1 of 4 ImPACT measures (25%), Visual Memory, female subjects had significantly lower scores than male subjects when measured at a range of 1 to 3 days after injury.8
One study suggests that at 7 days after injury, black athletes are 2.4 times more likely to have a clinically significant decline on 1 of the 4 ImPACT measures than white athletes.14
Key Question 3 Results
Key Question 3: What is the association between different signs, symptoms, and deficits or between the same signs, symptoms, or deficits at different time points for the same patient after a PCE? (See Evidence Table, Supplemental Digital Content 6, https://links.lww.com/NEU/A638).
Four publications met the criteria and were included as evidence for this question.4,9,15,18 Three took place in athletic environments: 1 with adults, 1 with adolescents, and 1 combining adults and adolescents. A fourth study assessed adult and pediatric hospital patients.9 Three studies examined the relationship between signs (severity estimates) and measures of function on cognitive tests.9,15,18 One examined the association between symptoms and cognitive function.4
A detailed analysis of the data from the included studies on associations among signs, symptoms, and deficits can be found in Supplemental Digital Content 9, Analysis (https://links.lww.com/NEU/A641), summarized as follows:
Signs and Cognitive Function: Severity of Injury
Evidence from 1 study suggests that athletes who sustain a PCE and have amnesia or disorientation for >5 minutes after injury may have significant reduction in memory function up to 7 days after injury and that those with amnesia or disorientation for <5 minutes may have significant reduction in memory function up to 4 days after injury.15 A second study suggests that athletes with both LOC and PTA after a PCE may have significant reduction in function as measured by the Standardized Assessment of Concussion immediately after injury compared with those with PTA alone or those with neither LOC or PTA, and that those with PTA alone may have significant reduction in function compared with those with neither LOC or PTA.18 A third study suggests that function as measured by performance on the Digit Symbol test within 24 hours of injury may be significantly lower for hospital patients with PTA than those without PTA.9
Symptoms and Cognitive Function
One study reported a significant correlation between subjective reports of symptoms and objective measures of cognitive function and balance within 48 hours of injury.4 Significant correlations were between the following:
Feeling mentally foggy and reaction time (P = .03)
Difficulty concentrating and verbal memory (P = .01)
Difficulty remembering and verbal memory (P < .001) and reaction time (P = .03)
Balance problems and the SOT composite (P < .001), somatosensory ratio (P = .03), visual ratio (P = .04), and vestibular ratio (P < .001)
Dizziness and the SOT composite (P < .001) and vestibular ratio (P = .01)
Key Question 4 Results
Key Question 4: What is the relationship between signs, symptoms, and deficits and imaging or biomarkers after a PCE? (See Evidence Table, Supplemental Digital Content 7, https://links.lww.com/NEU/A639).
Seven publications met the criteria and were included as evidence for this question.11,21-24,26,27 Four reported on the relationship between CT scans and SSDs,21,24,26,27 and 3 reported on biomarkers and SSDs.11,22,23 All took place in hospital environments: 4 with adult samples, 2 with adult and pediatric patients, and 1 with pediatric patients only.
A detailed analysis of the data from the included studies on associations between SSDs and imaging or biomarkers can be found in Supplemental Digital Content 9, Analysis (https://links.lww.com/NEU/A641), summarized as follows:
In the 4 publications assessing CT imaging, a total of 4803 patients were scanned within 24 hours of injury. Of those, 360 patients had positive findings on CT scan (7.5%); the prevalence of positive findings across studies ranged from 4.7% to 19% (see Table, Supplemental Digital Content 15 [https://links.lww.com/NEU/A647] for confidence intervals and odds ratios).
One study showed a significant relationship between CT and LOC/amnesia, 2 for Glasgow Coma Scale (GCS) score of 13, 1 for PTA >4 hours, and 4 for vomiting.
Symptoms, Neurologic Deficits, and Cognitive Deficits
The data on the association between CT and symptoms and neurologic deficits are contradictory or equivocal (see Table, Supplemental Digital Content 15, https://links.lww.com/NEU/A647). No study meeting inclusion criteria considered the association between CT and measures of cognitive function.
Limited evidence from single studies about the associations between signs/symptoms/objective measures of neurologic and cognitive function and biomarkers shows that GCS score (13-14 vs 15) is correlated with serum levels of ubiquitin C-terminal hydrolase within 4 hours of injury22 and that GCS score (13-14 vs 15) is correlated with serum levels of glial fibrillary acidic protein breakdown products within 4 hours of injury.23
The objectives of this project were to conduct a systematic review of the highest-quality literature about concussion and to assemble evidence about the prevalence and associations of key indicators of concussion. The goal was to establish an evidence-based foundation from which to derive, in future work, a definition, diagnostic criteria, and prognostic indicators of concussion.
Summary of Relevant Findings
Replicated data from studies that met the prespecified inclusion criteria with medium potential for bias and confound suggest that prevalent and consistent indicators of concussion are the following:
- Observed and documented disorientation or confusion immediately after the event,
- Impaired balance within 1 day after injury,
- Slower reaction time within 2 days after injury, and/or
- Impaired verbal learning and memory within 2 days after injury.
Recovery Patterns in Tests of Cognitive Function
For the studies included in this review, the proportion of tests that showed decrements in cognitive function (absolute differences between PCEs and comparators) decreased from 58% on day 1 to 8% on day 7. This finding could indicate that in the majority of cases cognitive deficits resolve within 1 week.
Tests Showing Deficits in Cognitive Function
Tests of reaction time, memory, and attention/processing speed/working memory most consistently showed deficits in cognitive function within the first week of injury.
Individuals with a history of previous concussions had lower scores than those without previous concussions on tests of cognitive function from baseline to 5 days after injury.
Severity of injury, measured by duration of amnesia or disorientation or the presence or absence of LOC and PTA, was associated with deficits in cognitive function up to 7 days after injury.
Self-reported symptoms may be associated with neurologic and cognitive deficits within 48 hours of injury.
Positive findings on CT imaging were associated with indicators of having sustained a PCE, measured by LOC/amnesia, a GCS score of 13, and vomiting; however, those measures are not linked to having sustained a concussion.
Serum levels of ubiquitin C-terminal hydrolase and glial fibrillary acidic protein breakdown products may be associated with an indication of having sustained a PCE, measured by a GCS score of 13 to 14; however, this measure was not linked to having sustained a concussion.
Limitations of This Review
A major limitation of this review is that most of the studies meeting inclusion and quality criteria were from athletic environments. The limited findings can be generalized only to similar populations, leaving unanswered questions about the feasibility, applicability, and utility of measurement instruments and findings in other populations such as patients seen in healthcare facilities or deployed military.
Another limitation is the lack of a definition for concussion. This review aims to identify prevalent indicators of concussion. However, as stated earlier, without a definition for concussion, the findings of this review can be viewed only as indicators of some phenomenon that is being called concussion.
Finally, the signs, symptoms, and deficits at the time points indicated that constitute the prevalent indicators are what was found in the existing literature from studies that met the prespecified inclusion criteria and adhered to standards for minimizing potential for bias and confound. The absence of evidence for other impairments at other time points must not be interpreted as evidence of no impairment. The evidence for these conclusions should be used as a starting point from which to identify missing information as targets for future investigation.
Practice Effects and Test Reliability
The ability of neuropsychological tests to detect deficits indicative of concussion is in question. Even with conditions more severe than concussion, many cognitive tests can be confounded by factors unrelated to the injury or disease.29 Relevant to this review, scores can improve as a result of repeated test exposure over a short period of time (eg, multiple tests within 7 days of injury).
Six of the 8 studies used to identify prevalent indicators in this report used cognitive tests. Two used average scores without separate control groups as comparators (eg, preinjury baseline scores were used to control),3,14 and although a third study used a control group, those data were not reported.28 Therefore, an assessment of practice effects cannot be made with these studies.
Three studies used average scores with separate control groups as comparators.6,10,15 Although the use of average scores limits the ability to assess practice effects, there is some evidence of practice effects from days 5 to 7 in these studies.
Limitations of the Literature
Although there is an abundance of research about concussion—the search captured 1362 potentially relevant publications—many studies were excluded because they did not provide information that could be used to determine the prevalence of and associations among indicators of concussion. Primary reasons for excluding studies or not using them in the final analysis include the following:
Lack of Comparison Groups
Signs and symptoms considered potential indicators of concussion are also common in nonconcussed populations. Thus, data from samples without comparators overestimate the prevalence of SSDs in the samples.
Mixed Patient Populations
Some samples included other pathologies or included the full spectrum of severity of traumatic brain injury.
Time From Injury Not Specified
Many samples included individuals whose chronicity varied. Some ranged from days to decades in the same sample. Some studies did not report time from injury.
Measures Not Validated
Some investigators adapted standardized measures; others formulated their own measures without validation data. Some studies used self-report or caregiver report of cognitive measures.
Data Reported as Composite Scores
Many composite scores combine a number of questions or tests that collapse individual SSDs into an aggregate. This form of reporting, although useful for some analyses, masks the information needed to identify individual indicators.
High Potential for Bias or Confound
As described in the Methods section of this report, each included study was assessed for potential for bias and confound across 7 domains. For each domain, each study was rated as minimized bias, did not minimize bias, or unclear. Table 6 summarizes the results of these assessments for studies used to address Key Questions 1 and 2. Lack of clarity in reporting accounts for much of the high potential for bias and confound.
Exclusive Case Definitions
Most studies prespecified concussion, selected the individuals who met the prespecified criteria for their study samples, and measured them for targeted outcomes. This would have been a reasonable approach if concussion were defined and the true presence or absence of concussion could be identified in the sample. That was not the case. The result is that the probability of subjects having the outcomes of interest was increased by the inclusion criteria. In addition, with many of the studies conducted in hospital settings, allocation into PCE or control groups was determined on the basis of specific signs or symptoms. Although limiting inclusion criteria serves the purpose of having a clearly defined sample, it minimizes the utility of the data for the purpose of identifying prevalent indicators.
Lack of Correlational Analyses
The most striking gap appears to be the lack of simple correlational analyses among SSDs and across time. Some studies reported proportions of the samples with specific SSDs at specific time points but did not correlate them. Other studies conducted sophisticated factor analyses demonstrating clustering of SSDs but reported results in terms of hypothesized latent variables, thus masking the information needed to identify specific indicators.
Analysis of Group Means on Outcome Measures
Using group averages for samples of potentially concussed individuals has limited value. The “concussed” patient population appears to have signs, symptoms, and deficits that are subtle and difficult to detect. In addition, from the available evidence, it appears that a relatively small proportion of individuals sustaining a concussion show the signs and symptoms commonly thought to be signature indicators of concussion. That problem is amplified by averaging measures such as response time or immediate recall across the entire sample of PCE participants. Data presented in proportions of patients with specific signs, symptoms, and deficits would be more useful in identifying prevalent indicators.
Measurement Time Points
Measurement time points in the available literature were not well suited for identifying prevalent indicators of concussion. Many studies took 1 or 2 measures during the first week and then again at 1 and/or 3 months after injury. From the results of this report, it appears that most SSDs associated with concussion resolve within a week of the injury. More research is needed to confirm this finding. If true, the information does not mitigate the need to address the long-term deficits sustained by the smaller number of concussed patients who do not recover. However, what is needed is a set of studies that take measures more consistently during the first several weeks after a PCE. With that, accurate recovery curves could be plotted for the observed SSDs.
In this analysis, we found cases in which 2 similar studies showed opposite (contradictory) findings for group differences on specific measures at specific time points. When contradictory findings between 2 studies occurred, there were differences in control groups (self as controls vs control group), methods of analysis (group means vs a reliable change index), or population ages (mixed sample of high school and college students vs separate). Variations in these 3 study characteristics (and others not yet identified) could account for the observed differences in results.
Study Design Recommendations
Evidence for Concussion vs Evidence for a PCE
An important distinction that needs to be clarified and used to design future studies more precisely is the distinction between evidence for concussion and evidence for a PCE.
In the absence of a physiologic measure, what is available to use as evidence for a concussion are signs, symptoms, and objective measures of neurologic or cognitive dysfunction. In addition, brain imaging, biomarkers, helmet accelerometer studies, and other objective methods are being explored for their ability to detect concussion. However, studies have not examined the links among these measures in a way that distinguishes evidence for concussion from evidence for a PCE.
For example, a forehead laceration after a fall is evidence for a PCE but not necessarily for a concussion. Similarly, an increase in the level of a brain-specific serum protein after a PCE may confirm that an impact occurred but does not necessarily confirm that a concussion occurred. Recent studies of imaging and biomarkers provide the groundwork for investigating the relationship between these measures (evidence for a PCE) and objective measures of dysfunction (evidence for a concussion).22,23,30 Future research should clearly distinguish between indicators of a PCE and indicators of a concussion and should analyze the associations between them to find if, and under which circumstances, evidence for a PCE can reliably be used to indicate a high probability of a concussion.
Distinguish Concussion From Other Phenomena
A critical gap in research on concussion is the lack of distinction between which signs, symptoms, and deficits are specific to concussion and which are also indicators of other pathologies or of transient, nonpathological conditions such as fatigue. This gap exists because most studies include and measure only individuals who sustained a PCE. Table 7 is an outline of a study recommended to clarify the distinction between concussion and other phenomena. This is intended to be an outline only and does not specify operational definitions and other critical information required for a complete design.
If well conducted, this study would provide data on concussion at multiple levels, including (1) detected vs undetected PCE; (2) association between force of impact and observed PCEs vs no observed PCEs; (3) changes in individual presentation of signs, symptoms, deficits, and serum proteins for different levels of force of impact (including no impact) and for observed PCEs vs no observed PCEs; and (4) association between signs, symptoms, deficits, serum proteins, and imaging for different levels of force of impact (including no impact) and for observed PCEs vs no observed PCEs.
Findings would be specific to college-level male subjects in an athletic setting (direct evidence). The next step would be to consider to what extent the findings could be generalized to other populations (indirect evidence). Information from a study conducted in an athletic setting would be considered indirect evidence if applied to hospital or battlefield settings. Indirect evidence may not provide a degree of confidence as strong as direct evidence from well-designed and well-conducted studies. There are, however, protocols for using indirect evidence that are derived from evidence-based principles and used in the development of guidelines.31 These protocols could be used to consider generalizability of findings. Given the difficulties inherent in collecting data under certain circumstances (such as armed conflict), the development of reliable methods for acquiring indirect evidence should be a priority. In addition, the activity of conducting a comprehensive and rigorous prospective study in an athletic setting could lay the groundwork for what would be required to conduct a well-controlled study in more volatile environments. It could also serve to clarify which specific questions, unique only to battlefield environments, remain unanswered and need to be addressed.
Road Map From Indicators to Diagnostic Criteria
This project accomplished the assembly of evidence about the prevalent indicators of concussion. The next step, which will be undertaken by the authors of this report, is to derive diagnostic criteria that are evidence-based and clinically useful. The proposed working definition for diagnostic criteria is given below:
Diagnostic criteria are the essential attributes and the pattern or relationship among those attributes used to identify a case.
The development of diagnostic criteria takes the prevalent indicators and identifies the consistent patterns among them. The following road map from indicators to diagnostic criteria is proposed:
There are a set of publications from large cohort studies rated as medium potential for bias that appear to have accomplished extensive data collection. They did not, however, report the data in a way that could be used to provide evidence for a definition or to derive diagnostic criteria. However, the necessary data exist, and preliminary inquiries indicate that they would be available for secondary analysis. These data sets are a potential source of information that could be analyzed with the use of simple frequency and correlational methods to derive more evidence to derive a definition and diagnostic criteria.
This project, the Raw Dataset Review (RaDaR), was initiated in July 2012 (funded by the US Army Contracting Command). The objectives of RaDaR are to acquire existing sets of data from studies about concussion or mild traumatic brain injury; to organize and analyze them to identify the frequency of specific signs, symptoms, and deficits across time; to analyze them to identify the correlations among them over time; and to use the findings to modify the evidence base for the prevalent indicators.
It is expected that some of the existing indicators will be confirmed; some of the existing indicators will not be confirmed and may be dropped from the evidence base; new indicators will be identified and be added to the evidence base; and relationships among indicators will be identified, providing the framework for diagnostic criteria.
The product will be a set of evidence-based indicators and their associations to be used as candidate diagnostic criteria for concussion. Subsequent work will involve validating the candidate criteria in retrospective and prospective studies.
At this time, there are no known objective measures to identify the change in brain function called concussion. Consequently, observed signs, subjective reports, and objective measures of neurologic and cognitive function that may be indicators of the underlying change in brain function are used to identify individuals with a high likelihood of having a concussion.
The task of this project was to identify which signs, symptoms, and neurologic and cognitive deficits have the highest and most consistent prevalence in samples of individuals sustaining a PCE and to assess how they are associated. From the available evidence, slowed reaction time, impaired verbal learning and memory, impaired balance, and disorientation or confusion were found to be significantly prevalent in early samples of exposed individuals. There is insufficient evidence to assess the relationships among these measures.
At a minimum, future studies should include comparison groups; take measures at fixed and relevant time points; report distinct signs, symptoms, and deficits in terms of frequencies and correlations; and follow standards for minimizing bias and confound.
There are no conflicts of interest. The Brain Trauma Foundation Concussion Guidelines project is supported by the US Army Contracting Command, Aberdeen Proving Ground, Natick Contracting Division, under contract No. W911QY-11-C-0074. The Brain Trauma Foundation provided funding for 2 meetings of the Panel of Technical Experts. Dr Jagoda is a consultant for Banyan Biomarkers. The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article.