Secondary Logo

Journal Logo

Original Investigation

YouTube Searches for Anterior Cruciate Ligament Injury Promote Biased, Low-Quality Videos

Ward, Brittany1; Lin, Alex1; Lin, Jasmine1; Warren, Christopher1; Mattern, Patrick1; Paskhover, Boris2; Chu, Alice1

Author Information
Translational Journal of the ACSM: Fall 2020 - Volume 5 - Issue 12 - e000131
doi: 10.1249/TJX.0000000000000131
  • Free

Abstract

INTRODUCTION

The anterior cruciate ligament (ACL) is the most commonly injured ligament in the knee (1). Each year, there are between 100,000 to 200,000 ACL ruptures in the United States alone and a reported annual incidence of approximately 68.6 per 100,000 person-years (2,3). A sprain or tear of the ACL is a common knee injury among children and adolescents, of which women and girls are at a greater risk of injury (4,5). Recent studies have shown that the incidence of pediatric ACL injury has significantly risen over the past 20 yr, with peak incidence observed during the high school years (4,5). Similarly, ACL reconstruction, one of the most frequently performed orthopedic procedures worldwide, has seen a steady rise over the past decade as well (6–8).

The Internet has been a major means of searching for and acquiring health information by patients for over a decade; however, the information is not always of high quality (9–11). In recent years, patients have been using not only standard Internet search engines to perform these searches, but also social media platforms such as YouTube to watch videos to augment their knowledge of medical procedures (12). A survey demonstrated that 90% of 18 to 24-yr-olds said they would trust medical information shared through social media sites (13). Today, YouTube is the world’s most popular online video-sharing community, with over 1.9 billion logged-in users each month (14), and according to the Pew Research Center, 90% of 18- to 24-yr-old Americans use YouTube (15). Although there is high authorship diversity among YouTube videos, the most popular videos are published by individual creators sharing their own perspective and experience, whereas the minority of videos are published by knowledgeable, unbiased medical professionals (16). Thus, the content and quality of health information on YouTube is highly variable.

In the United States, online health information is sought by 72% of Internet users, and ACL injuries are a significant interest for younger patients (17). It is important to consistently evaluate the quality of YouTube videos because its search algorithms are continually evolving, which results in different videos being presented to the user, as well as a large quantity of new content being uploaded daily. A substantial number of videos (33%) evaluated in our study did not exist at the time of the search in a recent study (18). There are also additional sources that would likely be recommended by physicians as a reliable resource for information on ACL injury, such as the American Orthopaedic Society for Sports Medicine (AOSSM) website and OrthoInfo from the American Academy of Orthopaedic Surgeons (AAOS). In this study, we evaluated the content and quality of ACL injury and treatment–related videos published on YouTube and compared them with those on the AOSSM and AAOS websites.

METHODS

YouTube (YouTube.com) was searched in “Incognito Mode” using Google Chrome on June 3, 2019, to compile a list of 40 video results using the following search terms: “ACL tear” and “ACL reconstruction.” Each term was searched using the default search method, “Relevance,” and an alternative search method, “View Count.” The top 10 results for each search term and method pairing were recorded. Non-English search results were excluded. Videos appearing as a result for more than one search term and method pairing were categorized as duplicate videos and rated once. Thirty unique videos were rated by four trained medical school reviewers (B.W., A.L., J.L., C.L.) using the DISCERN criterion, which utilizes a 5-point Likert scale for evaluating the quality of health information (19). An overall DISCERN score (DS) of 1 indicates a poor-quality video, and an overall score of 5 is the highest-quality video. A bias DS of 1 corresponds to a high-bias video, and a bias score of 5 corresponds to a low-bias video. The bias score was determined based on the impartiality of the presented video information. For example, a video describing one patient’s experience and providing guidance based solely on that singular experience would receive a 1 rating. Duplicate videos were included for individual search term result analysis to show the overall video quality and characteristics for each search term separately (Table 1). In the overall analysis, duplicate videos were excluded to better investigate the true number of unique videos that include US-board certified orthopedic surgeons and the prevalence of different types of video authors, such as hospitals or advertisers (Table 2,3).

TABLE 1 - Results for 40 Videos When Searching for “ACL Reconstruction” or “ACL Tear” on YouTube (youtube.com), Ranked by Either Relevance or View Count.
Search Term and Type No. Videos Age, Mean ± SD (yr) Video Length, Mean ± SD (min) Total No. Views No. Views, Mean ± SD DISCERN Bias, Mean ± SD DISCERN Overall Quality, Mean ± SD
ACL reconstruction 20 6.05 ± 2.83 7.07 ± 8.30 6,314,368 315,718.40 ± 639,584.29 1.85 ± 1.57 2.05 ± 1.19
 Relevance 10 5.69 ± 2.95 6.03 ± 8.34 3,191,288 319,128.80 ± 655,945.45 2.10 ± 1.79 2.10 ± 1.19
 View count 10 6.42 ± 2.81 8.11 ± 8.57 3,123,080 312,308.00 ± 658,253.57 1.60 ± 1.25 2.00 ± 1.25
ACL tear 20 5.76 ± 3.63 6.04 ± 6.63 26,284,939 1,314,246.95± 1,340,062.19 2.30 ± 1.78 2.00 ± 1.12
 Relevance 10 5.05 ± 3.59 6.30 ± 8.78 2,767,191 276,719.10 ± 670,262.72 3.20 ± 1.81 2.60 ± 1.26
 View count 10 6.47 ± 3.71 5.78 ± 3.95 23,517,748 2,351,774.80± 974,509.39 1.40 ± 1.26 1.40 ± 0.52
Total 40 5.91 ± 3.21 6.55 ± 7.43 32,599,307 814,982.68 ± 1,153,172.35 2.08 ± 1.67 2.03 ± 1.14

TABLE 2 - Mean Quality and Bias in Unique Videos Found Using “ACL Reconstruction” or “ACL Tear” Search Terms, Stratified by Video Publisher Categories.
Video Author No. Videos DISCERN Bias, Mean ± SD DISCERN Overall, Mean ± SD
Academic institution 1 5.00 4.00
Company/Advertiser 9 1.44 ± 1.33 1.44 ± 0.53
Hospital 11 2.82 ± 1.72 2.73 ± 1.35
Non-MD medical professional 1 1.00 1.00
Orthopedic surgeon 3 3.33 ± 2.08 2.33 ± 0.58
Patient 1 1.00 1.00
Physical therapist 1 1.00 1.00
YouTube user 3 1.00 ± 0.00 1.00 ± 0.00
Total 30 2.17 ± 1.68 2.00 ± 1.17

The video publication date, video length and total video views were recorded. Video publishers were categorized as: company/advertiser, hospital, orthopedic surgeon, patient, academic institution, non-MD medical professional, physical therapist, and YouTube user. Certificationmatters.org was used to determine whether the video contained commentary from a US board-certified physician.

The overall mean video age, length, number of views, bias DS, and overall DS were calculated as well as the total number of video views for each search term. The mean bias DS and mean overall DS were compared for videos including commentary from US board-certified physicians and those without and video results for the search terms “ACL reconstruction” using two-sample t-tests. OrthoInfo (orthoinfo.aaos.org) was searched for “ACL,” and the video result was rated using the DISCERN criteria. The AOSSM website (sportsmed.org/aossmimis) was searched for “ACL,” and the Resources for Patients–Surgical Animations–Knee videos were also rated using DISCERN. Videos from these sources were rated by all reviewers independently.

Internal rater consistency was evaluated using the interrater reliability coefficient by assessing the average of the ratings for five individual videos by the medical school reviewers and comparing them with ratings from a senior orthopedic resident (20).

RESULTS

The YouTube search resulted in 30 unique and 10 duplicate videos. The videos received a total of 25,198,961 views (range, 9432–4,194,863 views) and a mean of 839,965.37 ± 1,216,209.12 views per video. On average, videos were 5.61 ± 3.21 yr old (range, 0.09–11.52 yr) and 6.83 ± 7.20 min in length (range, 0.45–30.73 min). The mean bias DS was 2.17 ± 1.68, and the mean overall DS was 2.00 ± 1.17. When including duplicates to maintain an even number of videos per search term, the video results from the Relevance ACL Tear YouTube search scored highest for both bias and overall DS (Table 1).

A majority of videos, 37% (11), were posted by hospitals, followed by 30% (9) by companies or advertisers (Table 2). Fifty-three percent of the YouTube videos included commentary by US board-certified orthopedic surgeons. The mean bias (2.88 ± 1.89) and overall (2.44 ± 1.36) DS for these videos was significantly higher (P < 0.05) than the mean bias (1.36 ± 0.93) and overall (1.50 ± 0.65) DS for videos that did not include US board-certified orthopedic surgeons (Table 3).

TABLE 3 - Descriptive Statistics Comparing Videos with US Board-Certified Orthopedic Surgeons and Those Without.
Specialty All Videos DISCERN Bias, Mean ± SD DISCERN Overall, Mean ± SD
Orthopedic surgery 16 (53.33) 2.88 ± 1.89 2.44 ± 1.36
N/A 14 (46.67) 1.36 ± 0.93 1.50 ± 0.65
Total 30 2.17 ± 1.68 2.00 ± 1.17
Values in the “All Videos” column are expressed as n (%).

One video was found on the topic of “ACL” on AAOS/OrthoInfo and received a mean bias DS of 3.00 ± 0.00 and mean overall DS of 2.00 ± 0.00. Five surgical animation videos were found on the AOSSM website using the “ACL” search and scored a mean bias DS of 3.00 ± 0.00 and mean overall DS of 2.00 ± 0.00.

DISCUSSION

Because of the rising numbers of annual ACL injuries and reconstruction procedures, along with the increasing popularity of social media platforms being used as a trusted source of medical information, it is important to continually assess the quality of the content patients are consuming on these evolving online sources (4,6–9,13). ACL videos on YouTube may influence patient perception of certain diagnoses and treatment options, suggesting that videos of poor quality and with biased information may negatively affect patients and providers alike (21).

Patients turn to YouTube to gain valuable health information in a more understandable and accessible way; however, 66% of the videos in this study were posted by hospitals and advertisers (Table 2). These videos contained health information, but also aimed to persuade viewers, which is suboptimal in a video being watched to gain unbiased medical information. We found that a majority of the hospital videos contained phrases about the quality of their institution or program interwoven with their information for patients. In addition, none of the videos included external references, which made it difficult for viewers to analyze the accuracy of the information presented.

When the videos were searched by view count, the results for “ACL tear” resulted in more views (23,516,748) than “ACL reconstruction” (3,123,080); when searched by relevance, the results for “ACL tear” resulted in the fewer viewed videos (2,767,191) than “ACL reconstruction” (3,191,288). This view count difference of 20.75 million views is higher than the view counts of all other term-search type pairs combined. We speculate that YouTube algorithms favor popular videos, which continue to generate more views, irrespective of quality.

Overall, videos including US board-certified orthopedic surgeons were found to be of higher overall quality but were still relatively low quality and biased. This is consistent with results from prior studies in ADHD and facial plastics that found videos including US board-certified physicians were of significantly higher quality than videos that did not include medical professionals (22,23). Despite the higher quality, these videos with orthopedic surgeons only received 5,492,250 total views, whereas videos that did not include physicians totaled 19,706,711 views. This highlights another key issue with YouTube, which is that quality content does not always lead to a higher view count.

Despite a recent previous analysis of YouTube videos on ACL injury and reconstruction in 2018, 10 of the 30 unique videos evaluated in our study did not exist at the time of the previous study’s YouTube search, resulting in 897,242 new topic views (18). It is vital to continue to evaluate the quality of YouTube content as more than 300 h of new video content are uploaded every minute (16). We concur with other authors that the overall quality of the evaluated videos was poor and the bias was high (21). The largest factors that affect a YouTube video’s search ranking are relevance and keywords in relation to the search term (24). If practitioners who produce videos are not aware of the qualities necessary to appear in more user search results or those that increase the likelihood of the user watching the video, then the quality of the content is inconsequential. Even practitioners who do not make videos are affected by patients who are negatively affected by viewing poor-quality videos.

Educational videos on AOSSM and AAOS websites were also evaluated and compared with videos found via YouTube search. This study showed that, based on the DISCERN criteria, videos provided by the AOSSM and AAOS were less biased than, and of similar overall quality, to the average YouTube video. Both AOSSM and AAOS offer high-quality written information on the topic; however, patients who are seeking video sources may turn elsewhere, such as to YouTube, if not provided with enough video content. As such, we recommend that additional, higher-quality educational video content is included on both websites. In addition, this will allow physicians to recommend these resources as a key resource for video information for patients interested in learning more about ACL tears or reconstruction procedures.

Patients are seeking video sources for medical issues such as ACL tears in greater numbers. YouTube search terms are producing video suggestions that are highly viewed but of overall poor quality and high bias. Official sites such as those managed by AAOS and AOSSM have limited video selection and are infrequently used. Patients and practitioners should both be aware of the restricted availability of high-quality videos as well as the way search engines in YouTube propagate content. Future studies are necessary to further investigate the video qualities that will increase the likelihood of patients finding and watching high quality medical content.

In conclusion, physicians of patients who want to further educate themselves on certain diagnoses or procedures should be aware of the types of results their patients are likely to see based on the search term and video source. It would be helpful for patients seeking video information if the video libraries on ACL procedures supplied by AOSSM and AAOS were expanded. In addition, physicians should continue to upload high-quality, unbiased content to the most widely used video website, YouTube.

The authors have no financial interests to declare in relation to the content of this article. There are no conflicts of interest to report. The results presented in this study do not constitute endorsement by the American College of Sports Medicine.

All authors contributed equally to this work.

REFERENCES

1. Evans J, Nielson J. Anterior Cruciate Ligament (ACL) Knee Injuries. Treasure Island (FL): StatPearls; 2019.
2. Sanders TL, Maradit Kremers H, Bryan AJ, et al. Incidence of anterior cruciate ligament tears and reconstruction: a 21-year population-based study. Am J Sports Med. 2016;44(6):1502–7.
3. Garrick JG. OKU, Orthopaedic Knowledge Update: Sports Medicine 3. Rosemont (IL): American Academy of Orthopaedic Surgeons; 2004.
4. Beck NA, Lawrence JTR, Nordin JD, DeFor TA, Tompkins M. ACL tears in school-aged children and adolescents over 20 Years. Pediatrics. 2017;139(3):1–9.
5. Herzog MM, Marshall SW, Lund JL, Pate V, Mack CD, Spang JT. Incidence of anterior cruciate ligament reconstruction among adolescent females in the United States, 2002 through 2014. JAMA Pediatr. 2017;171(8):808–10.
6. Paschos NK, Howell SM. Anterior cruciate ligament reconstruction: principles of treatment. EFORT Open Rev. 2016;1(11):398–408.
7. Herzog MM, Marshall SW, Lund JL, Pate V, Spang JT. Cost of outpatient arthroscopic anterior cruciate ligament reconstruction among commercially insured patients in the United States, 2005–2013. Orthop J Sports Med. 2017;5(1):2325967116684776.
8. Mall NA, Chalmers PN, Moric M, et al. Incidence and trends of anterior cruciate ligament reconstruction in the United States. Am J Sports Med. 2014;42(10):2363–70.
9. Shaw RJ, Johnson CM. Health information seeking and social media use on the Internet among people with diabetes. Online J Public Health Inform. 2011;3(1):ojphi.v3i1.3561.
10. Gualtieri LN. The doctor as the second opinion and the internet as the first. CHI '09 Extended Abstracts on Human Factors in Computing Systems. ACM: Boston (MA); 2009. p. 2489–98.
11. Lu X, Zhang R, Wu W, Shang X, Liu M. Relationship between internet health information and patient compliance based on trust: empirical study. J Med Internet Res. 2018;20(8):e253.
12. Madathil KC, Rivera-Rodriguez AJ, Greenstein JS, Gramopadhye AK. Healthcare information on YouTube: a systematic review. Health Informatics J. 2015;21(3):173–94.
13. Miller M. 33% of U.S. consumers use social media for health care info [survey]: search engine watch; 2012 [cited 1 July 2019]. Available from: https://searchenginewatch.com/sew/news/2169462/-consumers-social-media-health-care-info-survey.
14. YouTube for press: YouTube; 2019 [cited 1 July 2019]Available from: https://www.youtube.com/about/press/.
15. Perrin A, Anderson M. Share of U.S. adults using social media, including Facebook, is mostly unchanged since 2018: Pew Research Center; 2019 [cited 1 July 2019]. Available from: https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/.
16. Aslam S. YouTube by the Numbers: Stats, Demographics & Fun Facts: Omnicore; 2019 [updated January 6, 2019; cited 22 June 2019]. Available from: https://www.omnicoreagency.com/youtube-statistics/.
17. Fox S. The social life of health information: Pew Research Center; 2014 [updated January 15, 2014; cited 1 July 2019]. Available from: https://www.pewresearch.org/fact-tank/2014/01/15/the-social-life-of-health-information/.
18. Cassidy JT, Fitzgerald E, Cassidy ES, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018;26(3):840–5.
19. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105–11.
20. Mantel N, Fleiss JL. The equivalence of the generalized McNemar tests for marginal homogeneity in 2 (3) and 3 (2) tables. Biometrics. 1975;31(3):727–9.
21. Cassidy JT, Baker JF. Orthopaedic patient information on the world wide web: an essential review. J Bone Joint Surg Am. 2016;98(4):325–38.
22. Ward B, Ward M, Nicheporuck A, Alaeddin I, Paskhover B. Assessment of YouTube as an informative resource on facial plastic surgery procedures. JAMA Facial Plast Surg. 2019;21(1):75–6.
23. Ward M, Ward B, Warren C, et al. The quality of YouTube videos as an educational resource for attention-deficit/hyperactivity disorder [published online ahead of print June 17, 2019]. Pediatr Neurol. 2019. doi: 10.1016/j.pediatrneurol.2019.04.001, 103:84–85.
24. Chi C. YouTube Algorithm: The Constantly Updated Guide to YouTube's Updates & Changes: HubSpot; 2019 [updated March 29, 2019; cited 1 July 2019]. Available from: https://blog.hubspot.com/marketing/youtube-algorithm.
Copyright © 2020 by the American College of Sports Medicine