Secondary Logo

Journal Logo

BRIEF COMMUNICATION

Dissemination of Gastroenterology and Hepatology Research on Social Media Platforms Is Associated With Increased Citation Count

Ayoub, Fares MD1; Ouni, Ahmed MD2; Case, Robert MD3; Ladna, Michael MD3; Shah, Harsh MD3; Rubin, David T. MD1

Author Information
The American Journal of Gastroenterology: October 2021 - Volume 116 - Issue 10 - p 2137-2139
doi: 10.14309/ajg.0000000000001240

Abstract

INTRODUCTION

Social media networks have gained popularity in the gastroenterology and hepatology (GI) community for the dissemination of research. A number of societies and journals maintain successful Facebook and Twitter pages (1). Altmetrics (alternative metrics of impact) measure how broadly scholarly work has been mentioned outside traditional literature including social media (Twitter and Facebook), mainstream news, Wikipedia, policy documents, and other sources (2). The Altmetric Attention Score (AAS) is a proprietary score (Digital Science Group, United Kingdom) increasingly used by journals to provide an objective measure of scholarly work mentions outside traditional literature.

Whether the number of citations a scholarly work receives in peer-reviewed literature correlates with how widely shared it is on social media has been investigated in other fields (3–6) but has not been fully investigated in GI, with the only analysis finding a significant correlation limited to 1 endoscopy journal (7). We examined the correlation between AAS, its social media components, and citations at 5 years from publication.

METHODS

“Scholarly work” hereby refers to content published in journals and indexed by Altmetric Explorer (a data exploration tool provided by Almetric.com) including original articles, editorials, reviews, conference proceedings, letters to the editor, and other published material. Scholarly work published in the 10 highest Web of Science impact factor (IF) journals in 2014 according to Journal Citation Reports category Gastroenterology and Hepatology (Clarivate Analytics, Philadelphia, PA) was included. We extracted the AAS and citation count in November 2019 (approximately 5 years after publication). Spearman correlation was used to study the correlation between AAS, its individual components, and citations. For multivariable analysis, logarithmic transformation of AAS and citations was performed to adjust for skewness and allow for linear regression.

RESULTS

We analyzed 4,026 scholarly works from the top 10 IF journals in 2014 (Table 1). Most scholarly work was mentioned on at least 1 social media platform (2,198 articles, 54.5%), followed by 2 platforms (509 articles, 12.6%) and ≥3 platforms (67, 1.7%). The correlation (r) between AAS and total citations at 5 years for all scholarly work from the top 10 journals was 0.62 (P < 0.001), representing strong correlation. Using linear regression, AAS was a positive predictor of total 5-year citations (P < 0.001) (Figure 1).

Table 1.
Table 1.:
Ten highest Web of Science impact factor journals in 2014, impact factor in 2014, and number of mentioned scholarly works included in analysis
Figure 1.
Figure 1.:
Scatter plot examining the relationship between log (Altmetric Attention Score + 1) and log (total 5-year citations + 1); fitted line represents the linear regression model and 95% confidence interval (CI). Log (Altmetric Attention Score +1) was a positive predictor of log (total 5-year citations +1) (P < 0.001); the overall model was statistically significant (P < 0.001), explaining 35% of the relationship (R 2 = 0.35).

In a multivariable model adjusting for journal impact factor and article open access status, every 1% increase in AAS resulted in a 0.95% increase in citations at 5 years. AAS, journal impact factor, and article open-access status were all significant predictors of 5-year citations in the multivariable model (see Supplementary Table 1, https://links.lww.com/AJG/B948).

Correlation between individual sources of mentions and citations

The strongest correlation between mentions and 5-year citations was for Twitter r = 0.54 (P < 0.001), followed by Facebook r = 0.24 (P < 0.001), suggesting that Twitter is the primary driver of social media–related mentions and associated correlation with citation counts (Table 2).

Table 2.
Table 2.:
Summary statistics and correlation coefficients between individual components of the Altmetric Attention Score (nonweighted) and 5-year citations

Sensitivity analyses

The association between AAS and 5-year citations remained strong when we excluded outliers (citations >300, AAS>100) [r = 0.61], studied only hepatology-focused journals [r = 0.55], and when excluding open access articles [r = 0.64]. However, in an analysis of scholarly work published in the 10 lowest IF GI journals (see supplementary Table 2, https://links.lww.com/AJG/B948), r decreased to 0.12 indicating weak correlation between the AAS and citations at 5 years in this subset of journals.

DISCUSSION

In this analysis of all scholarly work published in the top 10 IF GI journals in 2014, we found a strong correlation between AAS (as a measure of social media dissemination) and citations at 5 years. Our findings suggest that the degree of social media attention garnered by scholarly work published in leading GI journals strongly correlates with the number of citations in subsequent years.

We found that 73% of the included scholarly work was mentioned at least once outside of peer-reviewed literature since publication, demonstrating an increasing reach across nontraditional media platforms. Twitter was the strongest driver of mentions on social media (and 5-year citations) when analyzing the individual components of the AAS, consistent with other studies (5). This further supports the dominance of Twitter as a prime venue for sharing scholarly work outside of traditional peer-reviewed literature. In an analysis by Chang et al. of pediatric surgery literature, the length of time a journal's Twitter account had been established directly influenced the strength of correlation between AAS and citation count (6).

The correlation between AAS and citations has been investigated in other fields with conflicting results. Strong correlation was found in high IF pediatric surgery journals (6), weak correlation in cardiology (8), emergency medicine (4), and no correlation in plastic surgery literature (9). The strength of association may be specialty-specific, particularly if article content appeals to a broader readership. For example, an analysis of articles published in top radiology journals found a significantly lower number of articles mentioned outside of peer-reviewed literature (10). Case in point: The highest AAS article ‘A Diet Low in FODMAPs Reduces Symptoms of Irritable Bowel Syndrome’ discusses a dietary approach to the management of irritable bowel syndrome, a highly prevalent disorder present in 11% of the general population (11). This would arguably appeal to a lay audience more than the most cited article ‘Global epidemiology and genotype distribution of the hepatitis C virus infection’. In addition, studying articles published in earlier years compared with contemporary articles is less likely to yield a strong association, since social media adoption by the medical community has only recently increased considerably (6).

Our results remained robust to several sensitivity analyses. However, studying articles published in the lowest 10 IF journals the correlation became weak with r = 0.12. One explanation may be that at the time of writing, only 2 of these journals had official social media pages as compared to most top 10 journals, limiting their social media reach. Journal characteristics or article quality may have also influenced the results.

Our analysis has limitations. First, we only focused on the top 10 journals limiting the generalizability of our findings. We selected a follow-up period of 5 years to avoid citation lag, but the nature of the correlation between AAS and citations may change over longer periods of follow-up. Altmetric uses proprietary indexing methods, and as a result, our analysis includes work beyond original research, including editorials and reviews, limiting generalizability. Finally, controversial articles highlighted on social media (e.g., criticized for lack of robust methodology) may yield a higher AAS, highlighting that not all attention may be positive.

In conclusion, we found a strong correlation between AAS and scholarly work citation counts at 5 years, suggesting a growing role for social media platforms in the dissemination of GI scholarly work.

CONFLICTS OF INTEREST

Guarantor of the article: Fares Ayoub, MD.

Specific author contributions: F.A.: study conception and design, analysis and interpretation of data, manuscript drafting, manuscript revision, and study supervision. A.O., M.L., R.C., and H.S.: study conception and design, analysis and interpretation of data, and manuscript revision. D.T.R.: study conception, manuscript revision, and study supervision.

Financial support: No grant support was used for this work. The authors wish to thank Altmetric for providing this study's data free of charge for research purposes.

Potential competing interests: None to report.

REFERENCES

1. American College of Gastroenterology. Twitter Page (@AmCollegeGastro). Twitter, twitter.com/AmCollegeGastro. Accessed March 12, 2021.
2. Brigham TJ. An introduction to altmetrics. Med Ref Serv Q 2014;33:438–47.
3. Warren HR, Raison N, Dasgupta P. The rise of altmetrics. J Am Med Assoc 2017;317:131–2.
4. Barbic D, Tubman M, Lam H, et al. An analysis of altmetrics in emergency medicine. Acad Emerg Med 2016;23:251–68.
5. Hayon S, Tripathi H, Stormont IM, et al. Twitter mentions and academic citations in the urologic literature. Urology 2019;123:28–33.
6. Chang J, Desai N, Gosain A. Correlation between altmetric score and citations in pediatric surgery core journals. J Surg Res 2019;243:52–8.
7. Smith ZL, Chiang AL, Bowman D, et al. Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy. Gastrointest Endosc 2019;90:77–83.
8. Patel RB, Vaduganathan M, Bhatt DL, et al. Characterizing high-performing articles by altmetric score in major cardiovascular journals. JAMA Cardiol 2018;3:1249–51.
9. Ruan QZ, Chen AD, Cohen JB, et al. Alternative metrics of scholarly output: The relationship among altmetric score, mendeley reader score, citations, and downloads in plastic and reconstructive surgery. Plast Reconstr Surg 2018;141:801–9.
10. Rosenkrantz AB, Ayoola A, Singh K, et al. Alternative metrics (“Altmetrics”) for assessing article impact in popular general radiology journals. Acad Radiol 2017;24:891–97.
11. Lovell RM, Ford AC. Global prevalence of and risk factors for irritable bowel syndrome: A meta-analysis. Clin Gastroenterol Hepatol 2012;10:712–721.e4.

Supplemental Digital Content

© 2021 by The American College of Gastroenterology