Secondary Logo

Journal Logo

Original Articles

Persistent Postconcussion Symptoms: An Expert Consensus-Based Definition Using the Delphi Method

Lagacé-Legendre, Corinne MD; Boucher, Valérie MSc; Robert, Sébastien MD; Tardif, Pier-Alexandre MA, MSc; Ouellet, Marie-Christine PhD; de Guise, Elaine PhD; Boulard, Geneviève PhD; Frémont, Pierre MD, PhD; Émond, Marcel MD, MSc; Moore, Lynne PhD; Le Sage, Natalie MD, PhD

Author Information
Journal of Head Trauma Rehabilitation: March/April 2021 - Volume 36 - Issue 2 - p 96-102
doi: 10.1097/HTR.0000000000000613

Abstract

MILD TRAUMATIC BRAIN INJURY (mTBI) is a common problem, and its incidence in Canada has been estimated to be between 500 and 600 cases per 100 000 in a study conducted on hospital and family physician consultations.1 Considering that up to 25% of mTBI patients do not seek medical care after their injury,2 this number is likely an underestimation. The most common postconcussion symptoms are headaches, dizziness, sleep disturbance, and psychological and cognitive symptoms.3 Although most patients who suffered an mTBI recover without complications, various studies have estimated that 11% to 64% of patients will develop significant symptoms lasting more than 3 months after their injury.3–5 The wide range of reported incidence could be explained by the lack of a standardized definition for diagnosing persistent symptoms.

Long-term post-mTBI symptoms have been referred to as “postconcussion syndrome (PCS)”6 in the International Statistical Classification of Diseases and Related Health Problems (ICD-10), as “postconcussionnal disorder (PCD)”7 in the Diagnostic and Statistical Manual of Mental Disorders (4th edition, DSM-IV), and as “persistent symptoms” according to other authors.8–10 The concept of PCD was replaced by “major or mild neurocognitive disorder” in the DSM-V11 and excludes a vast proportion of patients diagnosed with PCS based on DSM-IV or ICD-10.12,13 These various definitions and diagnostic criteria are often criticized in the literature due to their lack of empirical support,10,14 low reproducibility between the different sets of criteria,15 and their low specificity.16–18

Thus, there is no clear consensual definition of persistent symptoms.8,10,16 The Ontario Neurotrauma Foundation has emphasized the need for a new consensual definition following an extensive literature review and expert panel meeting.19,20 Likewise, the International Collaboration on Mild Traumatic Brain Injury Prognosis responsible for the development of the 2014 update of the Archives of Physical Medicine and Rehabilitation highlighted the uncertainty around the possible long-term outcomes of mTBI and the need for high-quality follow-up studies.21–23 Moreover, their systematic review shows the great variety of definitions and assessment methods used to evaluate postconcussion symptoms. Indeed, while some refer to the DSM-IV and ICD-10 definitions, others assess the symptoms with a wide variety of scales and questionnaires (ie, Rivermead Postconcussion Symptoms Questionnaire [RPQ],24 Postconcussion Syndrome Checklist [PCSC],25 or the British Columbia Postconcussion Symptom Inventory).22,26,27

The objective of this study was therefore to develop a consensual definition of mTBI persistent symptoms as a whole, which includes the actual symptoms, their frequency, and duration.

METHODS

Delphi method

A conventional Delphi28,29 design was used in this study. A Delphi is a structured process used to gather anonymous information through a series of questionnaires or “rounds.”29 This technique has been used by several authors to develop prescribing indicators,30 performance indicators,31,32 and consensus definitions.33

The first phase of the Delphi method consists of asking each participant to contribute additional information that is pertinent to the issue (exploration of the subject) in an attempt to better understand the participant's opinion on the issue (as a group).34 All gathered information is analyzed and fed back to the participants, who then rate the relevance of each items. Each following round shows the group response and the participant's response until an agreement is reached.

Delphi panel

A subset of 30 healthcare professionals involved in the care and follow-up of mTBI patients older than 16 years on a regular basis were asked to participate in this Delphi study, and 26 agreed to participate. Those individuals were randomly selected through regional trauma programs in Quebec and Ontario, where they were all working. Those trauma programs assume a leadership role in the provincial trauma systems for the development, evaluation, and continuous quality improvement of care. This multidisciplinary group of experts included physicians, nurses, psychologists, neuropsychologists, occupational therapists, physiotherapists, and sports therapists from various settings in the provinces of Quebec and Ontario (hospital, rehabilitation center, private clinic) offering services to different types of patients (sports injuries, road trauma, or other).

Delphi technique and distribution

The Web-based (FluidSurveys; Fluidware, Ottawa, Ontario, Canada) French and English questionnaires were developed (N.L.S., S.R., and C.L.L.), reviewed, and validated by 4 authors (C.L.L., N.L.S., S.R., and V.B.) before being distributed to each expert via e-mail. This e-mail contained the details of the study and the link to the survey. Participants were given 5 weeks to complete each questionnaire, with reminders sent after 2 and 4 weeks.

This study was approved by the CHU de Québec-Université Laval Research ethics committee.

Data collection procedure

After collecting sociodemographic data, the first Delphi round required participants to list the dimensions or criteria they thought should be included in a definition of persistent symptoms. Open-ended questions pertaining to the criteria used to determine whether a patient suffers from persistent postconcussion symptoms were presented to the participants.

As per the standard Delphi technique, the second round anonymously presented the list of items obtained from the first round, which were divided by 2 authors (S.R. and N.L.S.) into the following categories: description of symptoms, number of symptoms, intensity of symptoms, frequency of symptoms, temporal association of the symptoms with the mTBI, duration of symptoms, impact of the symptoms on the person's life, and assessment scale. Experts were asked to assess the relevance of each item for a definition of persistent symptoms following an mTBI using a Likert scale ranging from 1 (“irrelevant”) to 9 (“essential”). Group responses were fed back to all participants via personalized online surveys, with histograms showing the percentages of responses for each level of the Likert scale in all of the following rounds. The median response was also shown, as well as the individual response of each participant, who were asked to rate the relevance of each item in the definition of persistent symptoms again.

Items that reached consensus were included in the definition of persistent symptoms following an mTBI.

Statistical analysis and definition of consensus

Frequency distributions of the group scores on a scale of 1 to 9 and the median score were calculated for each item of each round. Achievement of consensus was evaluated on the basis of all responses. Consensus on the relevance of an item was defined in this study as a median score of 7 or more and a sufficient level of agreement, whereas consensus toward the exclusion of an item was defined as a median score of 3 or less, again with sufficient agreement. An item was said to have reached sufficient agreement and was not evaluated in the subsequent rounds when, after having removed the 2 higher scores and the 2 lower scores, the remaining responses were in a range of 3 consecutive score points. This consensus definition corresponds to the “relaxed definition” of the RAND/UCLA Appropriateness Method User's Manual.35

RESULTS

A total of 5 rounds of Delphi questionnaires were necessary to reach consensus on all criteria, and 26 experts were contacted to participate in this Delphi study. Of these, 8 were neuropsychologists (30.8%), 6 were nurses (23.1%), 5 were occupational therapists (19.2%), and other professionals included physiotherapists, psychologists, doctors, sports therapists, trauma coordinator, and researchers. The median number of years of practice was 13 years (n = 19), and the median number of mTBI patients they cared for per month was 12.5 (n = 19). Those who were practicing in academic hospitals (trauma centers) accounted for 63% of our respondents, 25% were clinicians in rehabilitation centers, and 12% from private practice or research centers.

Consensus items

Overall, 92 items were submitted by the participants and integrated in the Delphi process. A consensus was reached for 50 items: 41 for adoption and 9 for exclusion. The remaining 42 items did not reach consensus because of disagreement and were therefore not included in the final definition. Supplemental Digital Content 1 (available at: http://links.lww.com/JHTR/A364) shows the complete list of Delphi items as well as the state of consensus achieved.

After 5 rounds, consensus was reached on a set of criteria. According to the participating experts, patients could be considered as having persistent symptoms if they report suffering from any unresolved symptoms that appeared within hours of the mTBI and are not better accounted for by another mental disorder. These patients show little improvement and are unable to return to their previous state and activities even when they are compliant with their postinjury recommendations. Three months after the mTBI, the daily symptoms show little improvement and have a negative impact on patients' activities, functioning, quality of life, and/or at least one sphere of their life. Figure 1 shows the full consensual definition.

Figure 1.
Figure 1.:
Consensual definition of PPCS. mTBI indicates mild traumatic brain injury; PTSD, posttraumatic stress disorder.

Response rates

Eighty-one percent of the participants completed at least one questionnaire. Sixteen of the 26 solicited experts completed the first survey (61.5% response rate).

In the second round (46.2% response rate), 96 items were submitted to the experts. Eighty-eight were rated using the Likert scale, and 8 were multiple-choice questions. A consensus toward the adoption of 6 items was reached, and no item was deemed irrelevant by the participants.

The third round (42.3% response rate) contained 83 Likert items and 8 non-Likert items. Of these items, 24 were adopted for the final definition by the experts and 3 were excluded.

In the fourth round (42.3% response rate), 61 Likert items and 10 non-Likert items were resubmitted to the participants. A consensus was reached for 18 items, 10 toward adoption and 8 toward irrelevancy.

The fifth and final round (30.8% response rate) contained 1 item rated using the Likert scale and 3 multiple-choice items for some propositions that were submitted by participants and needed more clarification because they were not mutually exclusive. The Likert item was adopted for the final definition by the experts.

DISCUSSION

This study is, to our knowledge, the first to offer an expert consensus-based definition of persistent symptoms following an mTBI. After 5 rounds, consensus was reached on a set of criteria that can be summarized as follows: presence of any symptom that cannot be attributed to a preexisting condition and that appeared within hours of an mTBI, that is still present every day 3 months after the trauma, and that has an impact on at least one sphere of a person's life.

Some participants suggested items from the ICD-10 definition of PCS6 (see Supplemental Digital Content 2, available at: http://links.lww.com/JHTR/A365) in the first round. Several items from this definition achieved consensus in our study (items 2, 6, 10, 20, 28, and 29), whereas others did not (items 7, 8, and 21). However, the ICD-10 definition included the concept of mTBI severity, on which we obtained no consensus, and of loss of consciousness (excluded in round 3), but unlike our definition, the ICD-10 definition did not specify a cutoff time after which those symptoms should be considered as persistent.

Items from the DSM-IV definition of PCD7 (see Supplemental Digital Content 2, available at: http://links.lww.com/JHTR/A365) were also suggested by our participants in the first round, and various PCD symptoms have been included in our definition (items 2, 6, 10, 18, 20, 28, and 29). Experts also agreed that symptoms should be assessed using validated tools (items 81, 83, and 84) and that there should be a temporal relationship between the symptoms and the mTBI (items 52 and 53), that those symptoms should not be better accounted for by another mental disorder (item 39), and that the symptoms should last for at least 3 months (item 54). The impact of these symptoms on a person's life is also important for our experts, and consensus was reached on items 55 to 57, 59 to 65, and 68. Other symptoms from the DSM-IV definition did not reach consensus (items 7, 19, 21, 23, and 24) and another one was excluded (item 25). Our experts also did not agree that there should be a minimum number of symptoms (the DSM-IV suggested that 3 symptoms should be present), and this was excluded in round 4 (item 41). The severity of the mTBI (ie, if the mTBI definition should include specific symptoms such as loss of consciousness, posttraumatic amnesia, or convulsions) did not reach consensus in our group (item 38) and neither did the concept of worsening of preexisting symptoms (item 43).

The DSM-V11 replaced the concept of PCD with “major or mild neurocognitive disorder,” and no consensus was reached on this definition. However, DSM-V items that were included in our Delphi study either did not reach consensus or were excluded (items 13, 37, 38, and 43). The temporal relationship between the symptoms and the mTBI (items 52 and 53) are consistent with our definition and that of the DSM-IV.7

Multiple definitions of persistent postconcussion symptoms are provided in the literature: for some authors, at least 3 symptoms are required,13,36–40 and for others, any symptom on the RPQ is sufficient.41,42 Some studies consider persistent postconcussion symptoms only when documented with neuropsychological tests.43 Recently, Tator et al13 proposed a novel definition excluding traumatic cerebral hemorrhage. Other define persistent postconcussion symptoms with various cutoffs on the Glasgow Outcome Scale Extended.44,45 The duration of symptoms in previous definitions varied from 1 month to 5 years. But in the end, none of these definitions were reached by consensus.

Our Delphi experts agreed that screening/evaluation tests, symptoms grids, physical tests, and cognitive tests should be used in the diagnosis of persistent symptoms. Similarly, some authors recommend use of the RPQ as a standardized scale to assess the symptoms.19 Other symptom scales, such as the Postconcussion Scale Revised46 and the PCSC,25 are also available; yet, there is no consensus on how or whether these scales should be used to establish the diagnosis.25,47–49 We were not able to obtain a consensus on which test and which cutoff scores should be used; however, all of the symptoms on which the experts agreed were symptoms assessed in the currently available symptom checklists.

There are currently no clear consensus criteria or definition for persistent postconcussion symptoms,50 and this confirms the relevance of a consensus-based definition. Several authors have expressed the need for new sets of criteria aiming to improve diagnosis and patient management.8,20 The final consensus obtained by this Delphi study provides such a set of criteria.

Strengths and limitations

The Delphi approach features several advantages. It allows the inclusion of individuals from various locations and with varied and complementary expertise. It also minimizes the effect of group interaction and eliminates the possibility of an individual dominating the consensus process. Delphi processes maximize information and knowledge collection from every expert while also allowing anonymous exchange of information. Our results are strengthened by our multidisciplinary team of experts. This may have reduced the influence of participants overrating a symptom they frequently encounter in practice.35 The fact that information is fed back to the participants at each round is an inherent strength of the Delphi design. The histogram describing other experts' responses as well as each expert's personal response to every item helped polarize the overall opinion toward a consensus.51 However, the fact that most experts involved in this study are from dedicated traumatic brain injury follow-up clinics in a publicly funded healthcare system may constitute a bias. The Berlin Consensus Statement on Concussion in Sport52 recently defined a time frame for persistent symptom definition: more than 2 weeks in adults and more than 4 weeks in children. It should be remembered that this consensual definition applies in a sporting context and aims to initiate specific individual interventions at an early stage. The definition proposed by our consensus does not contradict this concept, but it does suggest a standardization that can be used by clinicians and researchers to measure the impact of interventions made earlier.

The main limitation of this Delphi consensus is the low response rate, which is a under the median response rate of other Delphi studies (87%; Q1: 80% to Q3: 100%).29 It is possible that clinicians grew less involved since our study included 5 rounds instead of the usual 2 or 3 rounds.29,53 Although we obtained an overall 46.8% response rate, the number of respondents was the lowest in the last survey. However, the responses obtained for the Likert question were highly polarized: all responses ranged between 7 and 9 on the scale, which corresponds to the “strict definition of agreement” of the RAND/UCLA Appropriateness Method User's Manual.36 Furthermore, this manual suggests that an expert panel of 7 to 15 experts allows a certain diversity of opinions while remaining manageable.35 Since we obtained 8 completed surveys during our lowest response round, our Delphi remains in the recommended range.

CONCLUSION

This Delphi study provides an expert consensus definition of persistent symptoms following an mTBI. Our results are consistent with various items contained in other proposed definitions.6–10,12,13 It is hoped that this definition will help guide healthcare professionals in the diagnosis of persistent symptoms following an mTBI. Moreover, it will allow a standardized approach for research on mTBI prognosis. Future research should evaluate the reliability and validity (criterion, predictive) of diagnoses based on this definition.

REFERENCES

1. Ryu WH, Feinstein A, Colantonio A, Streiner DL, Dawson DR. Early identification and incidence of mild TBI in Ontario. Can J Neurol Sci. 2009;36(4):429–435.
2. Cassidy JD, Carroll LJ, Peloso PM, et al. Incidence, risk factors and prevention of mild traumatic brain injury: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med. 2004;(43)(suppl):28–60.
3. Ponsford J, Willmott C, Rothwell A, et al. Factors influencing outcome following mild traumatic brain injury in adults. J Int Neuropsychol Soc. 2000;6(5):568–579.
4. Ingebrigtsen T, Waterloo K, Marup-Jensen S, Attner E, Romner B. Quantification of postconcussion symptoms 3 months after minor head injury in 100 consecutive patients. J Neurol. 1998;245(9):609–612.
5. Boake C, McCauley SR, Levin HS, et al. Diagnostic criteria for postconcussional syndrome after mild to moderate traumatic brain injury. J Neuropsychiatry Clin Neurosci. 2005;17(3):350–356.
6. World Health Organization. The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. Geneva, Switzerland: World Health Organization; 1992.
7. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV. 4th ed. Washington, DC: American Psychiatric Association; 1994.
8. Marshall S, Bayley M, McCullagh S, Velikonja D, Berrigan L. Clinical practice guidelines for mild traumatic brain injury and persistent symptoms. Can Fam Physician. 2012;58(3):257–267, e128–e140.
9. Kashluba S, Paniak C, Casey JE. Persistent symptoms associated with factors identified by the WHO Task Force on Mild Traumatic Brain Injury. Clin Neuropsychol. 2008;22(2):195–208.
10. Carroll LJ, Cassidy JD, Peloso PM, et al. Prognosis for mild traumatic brain injury: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med. 2004;(43)(suppl):84–105.
11. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). Washington, DC: American Psychiatric Association; 2013.
12. Zuckerman SL, Yengo-Kahn AM, Buckley TA, et al. Predictors of postconcussion syndrome in collegiate student-athletes. Neurosurg Focus. 2016;40(4):E13.
13. Tator CH, Davis HS, Dufort PA, et al. Postconcussion syndrome: demographics and predictors in 221 patients. J Neurosurg. 2016;125(5):1206–1216.
14. McCauley SR, Boake C, Pedroza C, et al. Correlates of persistent postconcussional disorder: DSM-IV criteria versus ICD-10. J Clin Exp Neuropsychol. 2008;30(3):360–379.
15. McCauley SR, Boake C, Pedroza C, et al. Postconcussional disorder: are the DSM-IV criteria an improvement over the ICD-10? J Nerv Ment Dis. 2005;193(8):540–550.
16. Fayol P, Carriere H, Habonimana D, Dumond JJ. Preliminary questions before studying mild traumatic brain injury outcome. Ann Phys Rehabil Med. 2009;52(6):497–509.
17. Iverson GL, Lange RT. Examination of “postconcussion-like” symptoms in a healthy sample. Appl Neuropsychol. 2003;10(3):137–144.
18. Dean PJ, O'Neill D, Sterr A. Postconcussion syndrome: prevalence after mild traumatic brain injury in comparison with a sample without head injury. Brain Inj. 2012;26(1):14–26.
19. Ontario Neurotrauma Foundation. Guidelines for Concussion/mTBI & Persistent Symptoms. 2nd ed. Toronto, ON, Canada: Ontario Neurotrauma Foundation; 2013.
20. Ontario Neurotrauma Foundation. Guidelines for Concussion/mTBI & Persistent Symptoms. 3rd ed. Toronto, ON, Canada: Ontario Neurotrauma Foundation; 2018.
21. Carroll LJ, Cassidy JD, Cancelliere C, et al. Systematic review of the prognosis after mild traumatic brain injury in adults: cognitive, psychiatric, and mortality outcomes: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. 2014;95(3)(suppl):S152–S173.
22. Kristman VL, Borg J, Godbolt AK, et al. Methodological issues and research recommendations for prognosis after mild traumatic brain injury: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. 2014;95(3)(suppl):S265–S277.
23. Hartvigsen J, Boyle E, Cassidy JD, Carroll LJ. Mild traumatic brain injury after motor vehicle collisions: what are the symptoms and who treats them? A population-based 1-year inception cohort study. Arch Phys Med Rehabil. 2014;95(3)(suppl):S286–S294.
24. Lannsjo M, Borg J, Bjorklund G, Af Geijerstam JL, Lundgren-Nilsson A. Internal construct validity of the Rivermead Post-Concussion Symptoms Questionnaire. J Rehabil Med. 2011;43(11):997–1002.
25. Gouvier WD, Cubic B, Jones G, Brantley P, Cutlip Q. Postconcussion symptoms and daily stress in normal and head-injured college populations. Arch Clin Neuropsychol. 1992;7(3):193–211.
26. Iverson G, Gaetz M. Practical considerations for interpreting change following concussion. In: Traumatic Brain Injury in Sports: An International Neuropsychological Perspective. Lisse, the Netherlands: Swets & Zeitlinger; 2004:323–356.
27. Cassidy JD, Cancelliere C, Carroll LJ, et al. Systematic review of self-reported prognosis in adults after mild traumatic brain injury: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. 2014;95(3)(suppl):S132–S151.
28. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393.
29. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476.
30. Campbell SM, Cantrill JA, Roberts D. Prescribing indicators for UK general practice: delphi consultation study. BMJ. 2000;321(7258):425–428.
31. Normand SL, McNeil BJ, Peterson LE, Palmer RH. Eliciting expert opinion using the Delphi technique: identifying performance indicators for cardiovascular disease. Int J Qual Health Care. 1998;10(3):247–260.
32. Beattie E, Mackway-Jones K. A Delphi study to identify performance indicators for emergency medicine. Emerg Med J. 2004;21(1):47–50.
33. Lakhani BK, Giannouladis K, Leighton P, King AJ. Seeking a practical definition of stable glaucoma: a Delphi consensus survey of UK glaucoma consultants. Eye (Lond). 2020;34(2):335–343.
34. Linstone HA, Turoff M, eds. The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley; 1975.
35. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. The RAND/UCLA Appropriateness Method User's Manual. DTIC Document. Santa, Monica, CA: RAND Corp; 2001.
36. Oldenburg C, Lundin A, Edman G, Nygren-de Boussard C, Bartfai A. Cognitive reserve and persistent postconcussion symptoms—a prospective mild traumatic brain injury (mTBI) cohort study. Brain Inj. 2016;30(2):146–155.
37. Lannsjo M, Backheden M, Johansson U, Af Geijerstam JL, Borg J. Does head CT scan pathology predict outcome after mild traumatic brain injury? Eur J Neurol. 2013;20(1):124–129.
38. Kraus JF, Hsu P, Schafer K, Afifi AA. Sustained outcomes following mild traumatic brain injury: results of a five-emergency department longitudinal study. Brain Inj. 2014;28(10):1248–1256.
39. Hou R, Moss-Morris R, Peveler R, et al. When a minor head injury results in enduring symptoms: a prospective investigation of risk factors for postconcussional syndrome after mild traumatic brain injury. J Neurol Neurosurg Psychiatry. 2012;83(2):217–223.
40. Franke LM, Czarnota JN, Ketchum JM, Walker WC. Factor analysis of persistent postconcussive symptoms within a military sample with blast exposure. J Head Trauma Rehabil. 2015;30(1):E34–E46.
41. Stalnacke BM, Bjornstig U, Karlsson K, Sojka P. One-year follow-up of mild traumatic brain injury: postconcussion symptoms, disabilities and life satisfaction in relation to serum levels of S-100B and neurone-specific enolase in acute phase. J Rehabil Med. 2005;37(5):300–305.
42. Savola O, Hillbom M. Early predictors of postconcussion symptoms in patients with mild head injury. Eur J Neurol. 2003;10(2):175–181.
43. Muller K, Ingebrigtsen T, Wilsgaard T, et al. Prediction of time trends in recovery of cognitive function after mild head injury. Neurosurgery. 2009;64(4):698–704.
44. Jacobs B, Beems T, Stulemeijer M, et al. Outcome prediction in mild traumatic brain injury: age and clinical variables are stronger predictors than CT abnormalities. J Neurotrauma. 2010;27(4):655–668.
45. Yang CC, Tu YK, Hua MS, Huang SJ. The association between the postconcussion symptoms and clinical outcomes for patients with mild traumatic brain injury. J Trauma. 2007;62(3):657–663.
46. Lovell MR, Collins MW. Neuropsychological assessment of the college football player. J Head Trauma Rehabil. 1998;13(2):9–26.
47. Chan RC. How severe should symptoms be before someone is said to be suffering from postconcussion syndrome? An exploratory study with self-reported checklist using Rasch analysis. Brain Inj. 2005;19(13):1117–1124.
48. Potter S, Leigh E, Wade D, Fleminger S. The Rivermead Post Concussion Symptoms Questionnaire: a confirmatory factor analysis. J Neurol. 2006;253(12):1603–1614.
49. Chen JK, Johnston KM, Collie A, McCrory P, Ptito A. A validation of the postconcussion symptom scale in the assessment of complex concussion using cognitive testing and functional MRI. J Neurol Neurosurg Psychiatry. 2007;78(11):1231–1238.
50. Dwyer B, Katz DI. Postconcussion syndrome. Handb Clin Neurol. 2018;158:163–178.
51. Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–382.
52. McCrory P, Meeuwisse W, Dvorak J, et al. Consensus statement on concussion in sport—the 5(th) international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;51(11):838–847.
53. Schmidt RC. Managing Delphi surveys using nonparametric statistical techniques. Decis Sci. 1997;28(3):763–774.
Keywords:

concussion; consensus; Delphi; mild traumatic brain injury; mTBI; persistent symptoms; postconcussion symptoms

Supplemental Digital Content

© 2020 The Authors. Published by Wolters Kluwer Health, Inc.