Secondary Logo

Journal Logo

AAPA Members can view Full text articles for FREE. Not a Member? Join today!

Evaluating the quality of online information about concussions

Berg, Gina M., PhD; Hervey, Ashley M., MEd; Atterbury, Dusty; Cook, Ryan; Mosley, Mark, MD; Grundmeyer, Raymond, MD; Acuna, David, DO

Journal of the American Academy of PAs: February 2014 - Volume 27 - Issue 2 - p 1–8
doi: 10.1097/01.JAA.0000442712.05009.b1
Original Research

Objectives: Compare and assess information available on the Internet about the definition, symptoms, treatment, and return to play recommendations after a concussion.

Methods: The top 10 websites generated by a Google search on the keyword “concussion” were evaluated by two independent researchers and three medical professionals for definition, signs, symptoms, home treatment, care-seeking advice, and return to play recommendations. The medical professionals also rated their willingness to recommend each website to patients.

Results: Each website contained a general list of signs, symptoms, and home treatment. One website advised the use of ibuprofen, four advised against ibuprofen, and five made no medication recommendations. Nine websites contained guidance on seeking physician care, and eight recommended athletes not return to play until cleared by a healthcare professional.

Conclusion: Nine of the websites contained information for each section evaluated; however, information was inconsistent. Healthcare providers should be aware of the variable quality of information available on the Internet and guide patients to more optimal resources.

Gina M. Berg is director of trauma research at Wesley Medical Center in Wichita, Kan.; Ashley M. Hervey is a research associate at the University of Kansas School of Medicine-Wichita; Dusty Atterbury and Ryan Cook are PA students at Wichita State University. Mark Mosley is medical director of emergency services at Wesley Medical Center. Raymond Grundmeyer practices at Abay Neuroscience Center in Wichita. David Acuna is a trauma surgeon and critical care intensivist at Wesley Medical Center and associate medical director of trauma services for the medical center. The authors have indicated no relationships to disclose relating to the content of this article.

A concussion, also known as a mild traumatic brain injury, results from a direct impact to the head or impact to another part of the body with an impulsive force transmitted to the head.1 Concussions can cause significant morbidity and/or devastating complications.2 The Zurich Consensus Statement on Concussion in Sport describes a concussion as a “complex pathophysiological process affecting the brain, induced by traumatic biomechanical force.”1 Concussions occur frequently in contact sports such as football, hockey, and soccer and prevalence is 10% to 15% of athletes during any given season.3,4 The CDC estimates the incidence of concussions is as high as 3.8 million per year, translating into an average rate of about 10% of athletes sustaining a concussion during any season.4 Concussions are complex and often underreported due to factors such as individual perceived significance, and lack of awareness of symptoms, treatment, and return to play recommendations.3,5,6

Seeking health information online using general purpose search engines such as Google is the third most common use of the Internet, after e-mail and researching products or services.7,8 The information found influences how consumers manage their own care, and for some patients, may be a substitute for seeking treatment by a healthcare professional.9–11 Yet the information provided online often lacks peer review.12 In the absence of health content regulation, these sites can be both misleading and dangerous for the average consumer. Some examples of online health information that have been evaluated and found to be lacking in adherence to expert recommendations and guidelines include temperature taking in children, child safety education, nutrition information, breast cancer, inflammatory bowel diseases, treatment of depression, eating disorders, childhood diarrhea, postnatal mental health, diagnosis and management of acute lower back pain, physical activity information intended for patients with spinal cord injuries, and age-related macular degeneration.13–25 Because of the lack of content regulation, consumers may become overwhelmed with misinformation and not seek the proper care.

Back to Top | Article Outline


A Google search on the keyword “concussion” results in about 18,500,000 hits. The purpose of this study was to compare and assess online information readily available to consumers about concussions. The research question was Is online information about concussion accurate, complete, and consistent in terms of accepted recognition and treatment?

Back to Top | Article Outline


Concussion websites were identified using the keyword “concussion” on the popular search engine Google and included the top 10 nonmedia concussion websites as ranked by Google.26 The top 10 websites were chosen due to the average consumer viewing only the first few websites and rarely going beyond the first page of results.27,28 The content of each website was evaluated against current literature based on recognition and treatment defined as: definition, signs and symptoms, home treatment, when to seek physician care, and return to play recommendations. The amount of advertisement within each website also was noted.

The initial evaluation of the websites was performed by two independent researchers (DA and RC) in December 2011. Immediately following the evaluation, any discrepancies related to site material were discussed and resolved. Content for each website was then evaluated for expert opinion by three medical professionals: a neurosurgeon, trauma surgeon, and ED physician. The medical professionals scored the websites based on accuracy and completeness of five dimensions: definition, signs and symptoms, home treatment, when to seek care, and return to play recommendations. Scores were on a scale of 1 (poor) to 5 (excellent). The totals from each section were added together for an overall score out of 75 possible points. The medical professionals also rated their “willingness to recommend this website to patient consumers,” on a scale of 1 (strongly disagree) to 4 (strongly agree), which was dichotomized to agree/disagree. Three-way interrater agreement was assessed by using intraclass correlation coefficient using SPSS for Windows version 20.29

Back to Top | Article Outline


The top 10 websites evaluated were chosen from the top 11 ranked Google hits. The sixth-ranked website was a CBS sports-related website. Because this was a media source related to an individual athlete, it was removed from analysis. The top 10 websites analyzed were (in Google ranked order):

  • After the Injury
  • PubMed Health
  • WebMD
  • Wikipedia
  • Mayo Clinic
  • eMedicineHealth
  • CDC
  • MedicineNet
  • NYTimes
  • TeensHealth.

For details on each site, see Table 1.



Back to Top | Article Outline


Concussion definitions, summarized in Table 2, vary in specificity. For example, eMedicineHealth stated that a concussion is a term used to describe an injury to the brain resulting from an impact to the head. PubMed Health and NYTimes stated that a concussion is a traumatic brain injury (TBI) that may result in a bad headache, altered levels of alertness, or unconsciousness. TeensHealth defined concussion as a temporary loss of normal brain function.



Back to Top | Article Outline


Signs and symptom information provided per website (Table 2) varied in completeness, ranging from 10 to 30 signs and symptoms. Wikipedia and NYTimes contained the most extensive lists; After the Injury, PubMed Health, and eMedicineHealth had the fewest symptom descriptions.

Back to Top | Article Outline


Information about the use of nonsteroidal anti-inflammatory drugs (NSAIDs) such as ibuprofen, aspirin, and naproxen for home treatment is listed in Tables 1 and 2. Four websites advised directly against NSAID use, one site advised the use of ibuprofen, and five websites made no specific medication recommendations.

Back to Top | Article Outline


Nine websites listed information (Tables 1 and 2) on when to seek a healthcare professional. WebMD, Mayo Clinic, and CDC suggested that everyone who sustains a concussion should be evaluated by a healthcare provider. TeensHealth defined the difference between a simple and a complex concussion and stated that a complex concussion needs to be evaluated by a specialist. Five of the websites listed specific symptoms that require professional care. Four websites listed loss of consciousness as the symptom requiring medical attention. MedicineNet did not contain information on when to seek medical care.

Back to Top | Article Outline


Nine websites listed information (Tables 1 and 2) about when patients could return to play after a concussion. Three websites—the Mayo Clinic, eMedicineHealth, and CDC—suggested that a child or athlete not return to play the same day after receiving an injury. The eMedicineHealth website recommended the CDC four-step plan. MedicineNet did not contain return to play recommendations for athletes or parents.

Back to Top | Article Outline


The scores for each of the websites evaluated by the physicians (ED physician, neurosurgeon, and trauma surgeon) are listed in Table 3. The correlation coefficient indicating reliability among the physician raters was 0.613 (CI -0.197, -0.898). The physicians unanimously agreed to recommend only five of the 10 websites evaluated (Table 3). MedicineNet was the only website that none of the physicians were willing to recommend.



Back to Top | Article Outline


Article recommendation and advertising content (Table 1) varied among websites. Only four of the websites recommended articles for additional reading. Four websites contained no advertisements; however, After the Injury, Wikipedia, eMedicineHealth, MedicineNet, and NYTimes had more than 10 advertisements.

Back to Top | Article Outline


This study sought to evaluate concussion information (definition, signs and symptoms, home treatment, when to seek physician care, and return to play recommendations) easily available on the top 10 Google ranked websites. Out of the 10 websites evaluated, nine contained some type of information for each section; however, information such as concussion definition was inconsistent across websites. Although all definitions were described in less technical language than the Zurich consensus, some included altered levels of consciousness as part of the definition, which may be open to misinterpretation if the reader did not observe a change in consciousness in the patient.1 Nor was there a consistent or definitive list of the signs and symptoms of a concussion. This may be concerning, specifically in head trauma, as Diaz reported that as many as 11% of respondents reported using the Internet instead of seeing or speaking with their primary care provider.11 This lack of completeness (depending on which site the reader references) may also result in error in recognizing brain injury. In fact, recognizing the injury is the most challenging aspect of managing a patient with a concussion, even for those trained in screening, such as athletic trainers.30 Concussion symptoms are diverse and onset may be sudden (seconds) or delayed (hours to days). Severe symptoms such as impaired attention, vacant stare, or inability to focus may be recognized immediately, but milder symptoms such as headache, dizziness, memory dysfunction, fatigue, and irritability can be unnoticed.31 A late or inaccurate diagnosis of a concussion can have serious consequences, including a predisposition to future concussions, postconcussion syndrome, and the second-impact syndrome.2,30,32

Home treatment options were available on all websites, but were varied and included: no specific treatment instructions (WebMD and MedicineNet) to detailed instructions including: rest, talking to physician if needed, activities to avoid, and potential medications to take that help ease symptoms (Wikipedia). Only five of the websites directly instructed rest and light activity for a home treatment option. Health literature recommends that patients with concussions avoid activities that place the athlete at risk for further head injury, elevate the heart rate, or require moderate to high levels of cognition, as these activities may exacerbate postconcussion symptoms.33 Rest is acceptable, but excessive sleep should be avoided.30

Online information about using specific medications for home treatment was very inconsistent. Literature recommends that patients avoid aspirin products and NSAIDs in the acute stage of a concussion because these medications may enhance intracranial bleeding and mask some concussion-related symptoms, such as headache.33 The Athletic Trainers Association recommends only taking acetaminophen, and only with approval by a healthcare provider.30 Of the 10 websites evaluated, only four were consistent with the above-mentioned recommendations and advised patients to avoid ibuprofen, aspirin, and naproxen. Wikipedia advised the use of NSAIDS, which is contrary to the recommendation. Finally, five websites had no specific medication recommendations for home use. The lack of recommendations against using NSAIDs can put patients at risk.

Information about when to seek a healthcare provider and return to play recommendations were also inconsistently described and only available on nine sites. Anecdotally, two concerns were noted:

  • PubMed Health instructs readers to call their healthcare provider if they note “worrisome symptoms,” but does not define those symptoms.
  • NYTimes describes the symptom of “muscle weakness on one or both sides,” which may indicate a stroke or bleed.

Literature recommends that an individual believed to have a concussion should go through an initial onsite assessment by a trained professional. Once a concussion has been accurately diagnosed, opinions differ on treatment and how soon an athlete can return to play. However, it is widely accepted that an athlete should not return to play the same day as a concussion, and most treatment plans suggest physical and cognitive rest until symptoms have completely resolved, usually within 2 to 10 days.1,2,6,34,35 Referral to a medical facility is warranted if symptoms persist or worsen, and the referral should be to a medical professional with concussion management experience.30,36 Once symptoms have resolved, at both rest and exercise, an athlete should follow a graded exertion program to return to play.1,5,30,36 In athletics, a serious concern is the possibility of second impact syndrome, in which an athlete who has sustained a head injury receives a second head injury before symptoms from the first incident have resolved.37 Inappropriate return to play decisions may place at risk players who do not seek professional treatment based on erroneous information obtained online.

Health information consumers often use information available on the Internet to determine if the condition warrants seeking professional treatment and report that information provided online is similar, if not better, than what their physician would provide.11 Further, they may not choose to discuss the information when consulting their healthcare providers. Consumers sometimes assess information credibility based on a website's design or use of scientific language.27 In this study, the two websites that the physicians would not recommend (eMedicineHealth and MedicineNet) both have medical-sounding page titles and were scored well below Wikipedia.

Further, patient satisfaction may be at risk if the patient expectation is primed by inappropriate information as a result of Internet research, and the patient does not receive an expected outcome, such as a return to play release. Evaluation of patient satisfaction with provider services after personal Internet research would inform the necessity of discussing knowledge levels and treatment expectations.

This study is not without limitations. Only the top 10 Google ranked websites were evaluated, although health information consumers often do not search beyond a few sources.27,28 Website evaluations were based on the expert opinion of hospital-based physicians, yet concussions are often seen and treated by primary or urgent care providers. Expert opinion can be subjective and is open to arbitrary results and is categorized in the lowest grade of medical evidence; however, expert opinion can be useful for informing clinicians toward the optimal practice of patient care. Differences between inpatient and outpatient physician providers should be considered.

The literature notes that online information can be misleading and dangerous in a variety of diseases and treatment options. However, to our knowledge, online information about concussions has not been evaluated. Incomplete and inconsistent information may erroneously lead the reader to conclude that the patient did not suffer a concussion or does not need to be treated by a healthcare professional. Readers also may be prompted to use inappropriate home treatment. These results support a call to regulate the quality of health information available online. Further, strategies are necessary to inform readers about the quality of information they can easily access via popular search engines.

Back to Top | Article Outline


Providers should be aware of both the quality and consumer value of the information available on the Internet. In the context of concussions, the top 10 Google ranked websites provide incomplete, inaccurate, and inconsistent information. As concussions are often hard to recognize and can result in subsequent serious injury, readers not following recommended guidelines for professional evaluation and appropriate treatment may incur further head injury. Providers should assume that patients or their families have sought information online and may be misinformed. Clinicians who sense pushback may want to inquire where the patient or family has obtained their information and guide them to a more appropriate resource.

Back to Top | Article Outline


1. McCrory P, Meeuwisse W, Johnston K, et al. Consensus statement on concussion in sport–the 3rd International Conference on concussion in sport, held in Zurich, November 2008. J Clin Neurosci. 2009;16(6):755–763.
2. Almasi SJ, Wilson JJ.An update on the diagnosis and management of concussion. WMJ. 2012;111(1):21–27.
3. McCrea M, Hammeke T, Olsen G, et al. Unreported concussion in high school football players: implications for prevention. Clin J Sport Med. 2004;14(1):13–17.
4. Fjordbak B. Protecting student athletes: growing number of states pass concussion-related legislation. ASHA Lead. 2011;16(10):1–9.
5. Meehan WP 3rd, Bachur RG. Sport-related concussion. Pediatrics. 2009;123(1):114–123.
6. Khurana VG, Kaye AH. An overview of concussion in sport. J Clin Neurosci. 2012;19(1):1–11.
7. Fox S, Fallows D. Internet Health Resources: Health Searches and Email Have Become More Commonplace, But There Is Room for Improvement in Searches and Overall Internet Access. Washington, DC: Pew Internet and American Life Project; 2003.
8. Morahan-Martin JM. How internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav. 2004;7(5):497–510.
9. Wilkins AS. Expanding Internet access for health care consumers. Health Care Manage Rev. 1999;24(3):30–41.
10. Rainie L, Fox S. The Online Health Care Revolution: How the Web Helps Americans Take Better Care of Themselves. Washington, DC: Pew Internet and American Life Project; 2000.
    11. Diaz JA, Griffith RA, Ng JJ, et al. Patients' use of the Internet for medical information. J Gen Intern Med. 2002;17(3):180–185.
    12. Cline RJ, Haynes KM. Consumer health information seeking on the Internet: the state of the art. Health Educ Res. 2001;16(6):671–692.
    13. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ. 1997;314(7098):1875–1879.
    14. Isaac D, Cusimano MD, Sherman A, Chipman M. Child safety education and the world wide web: an evaluation of the content and quality of online resources. Inj Prev. 2004;10(1):59–61.
    15. Ostry A, Young ML, Hughes M. The quality of nutritional information available on popular websites: a content analysis. Health Educ Res. 2008;23(4):648–655.
    16. Bernstam EV, Walji MF, Sagaram S, et al. Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. Cancer. 2008;112(6):1206–1213.
    17. Quinn EM, Corrigan MA, McHugh SM, et al. Breast cancer information on the internet: analysis of accessibility and accuracy. Breast. 2012;21(4):514–517.
    18. van der Marel S, Duijvestein M, Hardwick JC, et al. Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis. 2009;15(12):1891–1896.
    19. Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ. 2000;321(7275):1511–1515.
    20. Murphy R, Frost S, Webster P, Schmidt U. An evaluation of web-based information. Int J Eat Disord. 2004;35(2):145–154.
    21. McClung HJ, Murray RD, Heitlinger LA. The Internet as a source for current patient information. Pediatrics. 1998;101(6):E2.
    22. Moore D, Ayers S. A review of postnatal mental health websites: help for healthcare professionals and patients. Arch Womens Ment Health. 2011;14(6):443–452.
    23. Hendrick PA, Ahmed OH, Bankier SS, et al. Acute low back pain information online: an evaluation of quality, content accuracy and readability of related websites. Man Ther. 2012;17(4):318–324.
    24. Jetha A, Faulkner G, Gorczynski P, et al. Physical activity and individuals with spinal cord injury: accuracy and quality of information on the Internet. Disabil Health J. 2011;4(2):112–120.
    25. Schalnus R, Aulmann G, Hellenbrecht A, et al. Content quality of ophthalmic information on the internet. Ophthalmologica. 2010;224(1):30–37.
    26. Tang H, Ng JH. Googling for a diagnosis–use of Google as a diagnostic aid: internet based study. BMJ. 2006;333(7579):1143–1145.
    27. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324(7337):573–577.
    28. McTavish J, Harris R, Wathen N. Searching for health: the topography of the first page. Ethics Inf Technol. 2011;13:227–240.
    29. IBM SPSS for Windows [computer program]. Version 20.0. Armonk, NY: IBM Corporation; 2011.
    30. Guskiewicz KM, Bruce SL, Cantu RC, et al. National Athletic Trainer's Association position statement: management of sport-related concussion. J Athl Train. 2004;39(3):280–297.
    31. Fisher J, Vaca F. Sport-related concussions in the emergency department. Top Emerg Med. 2004;26(3):260–266.
    32. Cantu RC. Head injuries in sport. Br J Sports Med. 1996;30(4):289–296.
    33. Martineau C, Kingma JJ, Bank L, McLeod TC. Guidelines for treatment of sport-related concussions. JAAPA. 2007;20(5):22–27.
    34. Crawford M. Concussions and chiropractic. JACA. 2011;48(5):8–11.
    35. Meehan WP 3rd. Medical therapies for concussion. Clin Sports Med. 2011;30(1):115–124.
    36. Casa DJ, Guskiewicz KM, Anderson SA, et al. National Athletic Trainer's Association position statement: preventing sudden death in sports. J Athl Train. 2012;47(1):96–118.
    37. Cantu RC. Second impact syndrome: immediate management. Phys Sportsmed. 1992; 20(9):55–66.

    Internet; review; concussion; return to play; online; health information

    © 2014 American Academy of Physician Assistants.