The 100 Most Mentioned Glaucoma Articles Online With Highest Altmetric Attention Scores

Supplemental Digital Content is available in the text. Précis: Characteristics of the most mentioned glaucoma articles on the internet were analyzed, allowing a better understanding of the dissemination of glaucoma research to the general public. Purpose: The aim was to determine the 100 most mentioned articles on the internet in the field of glaucoma and analyze their characteristics. Materials and Methods: We identified the top 100 glaucoma articles with the highest Altmetric Attention Score (AAS), an automatically calculated metric for monitoring social media. Each article was evaluated for several characteristics including year of publication, title, journal name, journal impact factor (IF), article topic, article type, affiliation, and online mentions (news, blog, policy, Twitter, Facebook, etc.). Correlation analysis was conducted for AAS with these characteristics. Results: The selected 100 articles came from 44 journals with more than half (56%) published in ophthalmology-specific journals. There was no significant correlation between IF and number of articles in a specific journal or AAS (P>0.1), but the number of articles in the top 100 was higher for ophthalmology journals with a higher IF (P<0.05). Original study was the most common study type (87%), of which clinical observation study was the most common subgroup (40%). Epidemiology/risk factor and basic science were the most common article topics (each 24%), followed by medical treatment (13%). Article topics regarding medical treatment had a significantly greater AAS than other topics (P<0.05). Of the top 5 articles, more than half (60%) were related to “Lifestyle choice” topics. Conclusions: There was no association between journal IF and AAS, consistent with previous studies. 90% of journals that had articles in the top 100 had a Twitter page. “Lifestyle choice” activities and other modifiable risk factors attracted significant online attention regarding glaucoma studies, with two of the top three most mentioned articles related to dietary intake. The present study thus provides a better understanding of online engagement with glaucoma research and the dissemination of this research to the general public.

T raditionally, the significance of scientific articles is measured by the number of citations, which affects journal impact factor (IF). 1 A citation is a reference to a published or unpublished source. Citation analysis has been conducted in various fields [2][3][4] and provides an objective metric for the intellectual impact of individual articles and a relative magnitude of the importance of particular subfields. 2 Within ophthalmology, citation analysis has been conducted in the topics of epidemiology, dry eyes, cataract surgery, glaucoma, and global ophthalmology. [5][6][7][8][9] Traditional citation analyses have limitations such as considerable lag time for citation accumulation after publication and generally only assessing relevance to researchers within the field. These limitations are becoming increasingly important as the advancement of the internet and rapid growth of social media have enabled a more diverse audience and widespread acquisition of scientific knowledge. 10,11 Altmetrics, a Web-based metric, was developed in 2010 to evaluate and monitor the early influence of scientific articles. 12 This metric measures the scientific and social influences of published scientific articles based on the number of "mentions" across various sources of online social media platforms such as Twitter and Facebook and assigns an Altmetric Attention Score (AAS), which is a weighted approximation of the online attention received by the article. 10,12 The AAS is calculated based on 3 main factors: the number of times the article is mentioned, the sources that mention the article (including news outlets and various social media sites), and the authors of each mention. 13 Altmetrics may provide a more broad assessment of research impact, diversity of underlying factors, and timeliness. 14 Recently, there have been several published studies that have used Altmetrics to identify the most mentioned articles in the fields of neurosurgery, neuroimaging, neurointervention, stroke, emergency medicine, dentistry, and critical care. [15][16][17][18][19][20][21] Findings from these studies indicate that factors driving AAS and IF may not completely overlap. Given these previous findings, it is thought that traditional metrics such as citation count and IF may predominantly represent what is important to scientists, while online attention scores represent what is important to the general population. To the best of our knowledge, no study to date has investigated the most mentioned articles in ophthalmology as a whole or in any ophthalmology subspecialty. Herein, we identify the top 100 glaucoma articles with the highest AASs and investigate the characteristics of these articles. We hypothesized that AAS of glaucoma articles would not correlate with journal IF. We did not have any hypotheses on the characteristics of the most mentioned articles in glaucoma.

MATERIALS AND METHODS
This study did not include human subjects. Therefore, the study did not require Institutional Review Board approval as per the standing policy of The George Washington School of Medicine and Health Sciences.

Altmetric Database Extraction
Altmetric Explorer (Altmetric, London, UK) is a commercially available Web-based application offering a nontraditional metric to monitor a publication's online mentions, social media discussion, and general online interaction. We accessed the Altmetric database 13 to extract article data. The database contains up-to-date information regarding online activity of millions of articles. The various online activity measures are given weighted scores from which an AAS is calculated. 22 In particular, the three categories from which the AAS is calculated are number of mentions, sources of the mentions, and the authors of each mention. Within these categories, the sources are weighted to reflect the relative reach of each type of source. 22 The weightings reflect the relative reach of each source per Altmetric and are as follows, with each source's relative weight in parentheses: news (8), blog (5), policy document (3), patent (3), Wikipedia (3), Peer review on Publons or Pubpeer (1), Weibo (1), Google+ (1), F1000 (1), syllabi on Open Syllabus (1), LinkedIn (0.5), Twitter tweets and retweets (0.25), Facebook public pages (0.25), Reddit (0.25), Pinterest (0.25), Stack Overflow Q&A (0.25), YouTube (0.25). Further modifiers are applied to the score from news outlets depending on reach, and Twitter scores are also modified based on reach (followers), how often the user tweets about research, and potential bias. In addition to the summary AAS, the database also provides the underlying mention counts in individual categories.
The following search term was used in the database:

Selection of Articles
The searches returned 11,955 unique articles, which were ordered by AAS. Starting from the highest ranked article by AAS, 2 independent reviewers (J.L.C and J.S) assessed sequentially identifying 100 articles with the primary focus determined to be glaucoma. Articles with the primary focus determined to be glaucoma were included and articles with glaucoma as secondary focus were excluded. Examples of excluded articles include those focused on general methodology, ocular disease in general, and chronic diseases without a focus in glaucoma. Article titles and abstracts were reviewed. When necessary, the full manuscript was reviewed for clarification. There was agreement regarding inclusion for all articles. All articles that were excluded and those that had an AAS below the top 100 were removed from further analysis. A total of 191 articles were reviewed to identify the top 100 glaucoma focused articles.

Extraction of Journal IF and Twitter Presence
For the top 100 mentioned glaucoma articles included in the study, we identified the journals that these articles were published in. For each journal, the Web of Science 2017 IF was identified. 23 Articles that were from journals that did not have IF available were not included in the subsequent analysis of IF correlation with number of articles in journal or AAS. IF, as extracted from Web of Science, is calculated as the number of citations received in a particular year for publications published in that journal divided by the number of publications in the journal in the two preceding years. We also calculated Average IF to take into account year each article was published. We used the website https://www.bioxbio.com/ to identify an IF for each article based on the journal and year published. We then averaged these numbers for each journal to determine the average IF.
In addition, presence of Twitter page for each journal was identified and if present, the year in which the Twitter page was created was identified for each journal. We used search terms "Twitter" + "journal name" in Google search to identify each journal twitter page.
When comparing IF and AAS directly the actual IF was used as there was a correlation. When IF was used to identify the number of articles the IF was rounded so that number of articles from journals with a similar IF could be binned.

Extraction of Article-Specific Factors
Both reviewers (J.L.C. and J.S.) reviewed all included articles to determine topic and type of article. The details of these categories are provided here. Article topic included: (1) epidemiology/risk factors, (2) diagnostics, (3) medical treatment, (4) surgical treatment, (5) genetics, (6) basic science, and (7) Other (cost, patient satisfaction, and quality of life). In the case where both medical and surgical treatment were being compared we chose the treatment method that was being investigated (not standard of care control).
Original study further subdivided to (1a) clinical observation, (1b) comparative clinical trial, (1c) experimental study, and (1d) genome-wide association study. These categories were similar to those used in other Altmetric analyses. 20 A subset of articles was categorized as "lifestyle choice." "Lifestyle choice" was defined as activities that can be modified by the patient (eg, tobacco use, diet and nutrient intake, exercise, physical activity, cardiorespiratory fitness, oral health). Two reviewers (J.L.C. and J.S.) assessed the articles and determined which articles would fall under this classification.
For each article, the continent of origin for the corresponding author was identified. An article with more than one country contributing was also identified and labeled as an international collaboration.

Statistical Analysis
Wilcoxon rank-sum test was used to compare nonnormally distributed data such as the AAS. When multiple Article publication year ranged from 2012 to 2019. There was no significant trend for correlation between year published and AAS (P > 0.5). The most frequent year was 2016 (31%). For most articles, the corresponding author was from North America (71%) and international collaboration represented almost a third of the articles (36%). The median AAS was not significantly different between articles from any specific continent or between those with and without an international collaboration (P > 0.1). Of the top 100 articles, 14% listed industry as a funding source; the median AAS for this subset was not significantly different than the rest (P > 0.1). The most frequently mentioned article was "Frequency of a diagnosis of glaucoma in individuals who consume coffee, tea and/or soft drinks," 24 a retrospective cross-sectional study. The article had 128 news mentions, 8 blog mentions, 129 Twitter mentions, 18 Facebook mentions, 1 Reddit mention, and 1 F1000 mention. Of the top 10 articles, Twitter mentions (10/10) was the most important vehicle for dissemination followed by news mentions (10/10).
The selected articles came from 44 journals. Journal name, journal Twitter presence, ophthalmology specific versus nonophthalmology focus of each journal, article count per journal, and IF are reported in Table 1. JAMA Ophthalmology (formerly Archives of Ophthalmology) had the highest number of articles (17%), followed by Ophthalmology (13%). This was followed by Investigative Ophthalmology & Visual Science (6%), American Journal of Ophthalmology (4%), and then Clinical Ophthalmology (3%), Journal of Glaucoma (3%), and The Lancet (3%). The remaining journals had less than 3 articles, with most journals having 1 article. There was no significant correlation between journal IF and number of articles contributed by the journal (P > 0.1). There was also no significant correlation between journal IF and article AAS (P > 0.2; Supplementary  Fig. 1, Supplemental Digital Content 2, http://links.lww. com/IJG/A582). Ophthalmology-specific journals accounted for 13 of the 44 (29.5%) journals represented and accounted for 56% of the top 100 articles. No significant difference in the AAS of articles was found between ophthalmology specific versus nonophthalmology-specific journals (P > 0.5). When looking only at those journals that were ophthalmology specific the IF was still not correlated with AAS, however, the number of articles in the top 100 (Table 2) was higher for journals with a higher IF (P < 0.05, Jonckheere-Terpstrsa test). Using the average IF for each journal also revealed a significant correlation with number of articles in the top 100 (P < 0.01, Jonckheere-Terpstrsa). Only 10% of articles came from journals without an identified Twitter page; however, the AAS was not significantly different between the articles from journals with Twitter presence versus those without Twitter presence (P > 0.5), though the Ophthalmology-specific journals per rounded impact factor and number of the top 100 most mentioned glaucoma articles per each impact factor, showing a greater number of articles coming from the journals with highest impact factor.
The number of articles in journals with indicated impact factor is provided. The statistical data regarding the trend analysis between number of articles and impact factor is provided in the table.
IF indicates impact factor. Breakdown of article types of the top 100 mentioned glaucoma articles and mean AAS per article type, showing that most studies were clinical observational studies but the highest AAS were seen in comparative clinical trials.
The number of articles along with mean, SD, median, and range are provided.
fact that most of the journals had a twitter presence may limit our ability to assess this. We also did not find a significant relationship between number of years having a twitter with AAS or number of articles in the top 100. The most frequent article type was original study, which comprised 87% of the articles (Table 3). This was subdivided into clinical observation study (40%), followed by experimental study (25%) and comparative clinical (11%) and genome-wide association study (11%). Systemic reviews/meta-analyses comprised 6%, editorials comprised 4%, and case reports/case series were least common at 3%. The highest median AAS was for comparative clinical trial (AAS 216), but did not reach significance compared with that of the other groups (P > 0.05).
The 2 most frequent categories were epidemiology/risk factor and basic science type, each at 24% ( Table 4). The next most common were treatment (22%), topics related to medical treatments (13%), and surgical treatments (9%). This was followed by genetics (10%), diagnostics (10%), and "other" (10%). The median AAS of topics related to medical treatments was significantly higher than genetics, basic science, and other categories (P < 0.05; Fig. 1). The difference between medical treatments and both diagnostics and surgical treatment was not significantly different (P > 0.05).
We identified a subgroup of articles associated with "lifestyle choice." "Lifestyle choice" was defined as activities that can be modified by the patient (eg tobacco use, diet and nutrient intake, exercise, physical activity, cardiorespiratory fitness, oral health). We categorized 15% of articles as falling within this subgroup, 3 articles of which were in the top 5.

DISCUSSION
In this study we identified the top 100 published articles in the field of glaucoma with the highest online attention on social media. These articles were analyzed for the journal they were published in, the journal's IF, the general topic of study, and the type of study. More than half (56%) were published in ophthalmology-specific journals, with the majority in JAMA Ophthalmology (17%) and Ophthalmology (13%). A similar analysis of most mentioned neurointervention articles found multidisciplinary journals to have more frequent mentions than specialty journals. 20 They attributed this to the fact that neurointervention is a newer field and the specialty-specific journals have less penetrance among the public. This could also be related to the interdisciplinary nature of neurointervention compared with glaucoma. Compared with classic citation counts in ophthalmology, Altmetric analysis identified more journals (44) than previous bibliometric studies. [5][6][7][8] Other Altmetric analyses in different fields also report a similar number of journals among the top 100 mentioned articles. 17 The articles were primarily original studies with most being clinical observation studies. The majority of articles focused on the following topics: epidemiology/risk factor, basic science, and treatments. The medical treatment subcategory had a significantly higher AAS. The range of AAS in this study (42 to 1009) and publication dates is comparable to those seen in Altmetric analysis in neurosurgery, oral cancer, and critical care. 15,17,19,20 Most articles were published in 2016 (31%).
As we had hypothesized, our study demonstrated a lack of association between journal IF and AAS, which is consistent with studies in other fields. 16,17,19 Even when there has been a correlation demonstrated between IF and AAS in a previous study on Altmetrics in emergency medicine, the correlation was only modest. 18 In the fields of neurology and neurosurgery, Kim et al 20 suggested that traditional metrics and AAS measure different perspectives, which also appears to apply to glaucoma research. AAS has been hypothesized to be higher when articles relate to "popular topics" or are otherwise able to garner public attention, which can be good or bad, while traditional metrics pertain to impact on the scientific community. It is important to note though IF and AAS are not correlated, ophthalmologyspecific journals with a higher IF have more articles in the top 100 ( Table 2). The discrepancy between ophthalmology specific and general journals can be explained by the fact that some recent important publications have been published in highimpact general journals, which rarely publish glaucoma-related work. Altogether, these results suggest IF is an important measure, but there are other factors which drive some articles to generate higher amounts of attention. Therefore, when assessing the intellectual and societal impact of an article, a combination of traditional metrics and attention scores may be best.  Topics of the top 100 most mentioned glaucoma articles, showing that the greatest number of articles were about epidemiology/risk factors but that the mean AAS was highest for medical treatment articles.
The number of articles along with mean, SD, median, and range are provided.
†AAS for "Treatment-medical" group (bold) was significantly greater than the identified topics (+), P < 0.05 (Wilcoxon rank-sum with correction for multiple comparison (see Materials and Methods).
Our study demonstrated that "lifestyle choice" activities and/or modifiable risk factors attracted online attention regarding glaucoma studies. Though the median AAS was not significantly higher, 2 of the top 3 most mentioned articles in our study were related to dietary intake. The article with the highest social media presence was "Frequency of a diagnosis of glaucoma in individuals who consume coffee, tea, and/or soft drinks." 24 When looking across all medical and health science articles, diet and exercise represented about 31% and 11% of articles with top Altmetric scores. 25 Altmetric analysis of stroke papers showed modifiable risk factors such as diet and exercise to make up half of the top articles. 16 The results from our study and other Altmetrics studies suggests that the medical interest of the "general public" gravitates towards modifiable "lifestyle choice" activities. It is likely that patients find these articles to be directly relatable and will mention them in their social media platforms such as Twitter or Facebook. Interestingly, these results also match classic bibliometric analysis, as a study of articles related to ophthalmic epidemiology found that the most cited article was a study of outdoor activities and prevalence of myopia among schoolchildren. 5 The distribution of article types, with most being original studies, is also similar to that of other studies of AAS. 16,19,20 Here we find most of these original studies to be clinical observation and experimental, rather than comparative clinical trials as shown in neurointervention. 20 In terms of article topic, we found topics to be unevenly distributed but without a clear leading topic. This is in contrast with other Altmetric analyses that found lower prevalence of basic science. 19,20,26 This may be because of the highly specialized nature of ophthalmology compared with other medical fields, and we speculate many online media mentions of glaucoma articles may be from scientist or physicians in ophthalmology. The medical treatment subcategory had a significantly greater mean AAS compared with that of genetics, basic science, and other subcategories, and we hypothesize that medical treatment topics may draw more online interest as these articles may be more understandable and applicable for the general public. It is possible that this is specific to glaucoma as topical medications are essential to glaucoma management. For instance, Altmetric studies in neurosurgery showed types of articles related to operative procedures were the most frequent in the top 100. 15 Future research should thus compare these findings in glaucoma to other specialties within ophthalmology. In addition, future studies with analysis based on the demographics of users mentioning these glaucoma articles on social media are warranted. Classic bibliometric analyses have shown higher citation counts for clinical research. 7 This may represent a factor more specific to glaucoma. Supporting this, a bibliographic analysis of glaucoma papers showed that though clinical and epidemiology papers were more prevalent among highly cited papers, there was an increasing amount of basic science studies more recently. 9 There are limitations to our study. We only identified Altmetric data from the Altmetric database. Other studies have shown not all articles in a journal are included in the database. 27 However, this database is widespread and more easily accessible for investigators to work with, and is the sole source of most studies in the field. Second, we excluded all articles that did not have glaucoma as the primary focus. For instance, articles with primary focus on cataract surgery (glaucoma secondary) were excluded. Further exploration may consider including all articles related to glaucoma. Moreover, we only assessed the top 100 mentioned articles, as has previously been done in published literature. The search terms need to be broad enough to include all articles related to glaucoma, but as shown in our results many of the papers from the search results were not related to glaucoma (see Materials and Methods section). In addition, since information is in real-time the scores can change in a short amount of time. This was minimized by collecting all the data in one setting and limiting to top 100 articles which are unlikely to represent new articles which will show higher shifts. Older studies that were published in a "nononline media era" will likely not have an AAS score, resulting in the limitation of using AAS to compare articles from different eras. However, because we did the analysis until May 2019 with no limit, there is a chance that older articles have had more opportunities to be read and cited. In addition, IF varies per year, though usually without significant changes. As above, we attempted to minimize this by collecting the data on 1 day, and we believe that performing the analysis with no historical limit will be more representative of the true "most mentioned" articles in glaucoma. In addition, Altmetric score tracking only is available starting from October 2011, and numerous previous Altmetrics studies have also had no historical limit or a limit close to or even preceding October 2011. Future research may consider stratifying AAS analyses by year. Another limitation for comparing number of articles is the difference in amount of the article type in the literature; for example, there are a higher number of original studies than review papers or editorials, which would impact the number in the top 100. Finally, other limitations of Altmetric database studies ones previously mentioned 27 of personal biases (individual tastes or perfunctory citations), automation bias (bots generating retweets to increase Altmetric impact), and Matthew effect of accumulated advantage bias (the "wealthy" influencers or content creators are rewarded with additional wealth). These may have increased the significant disparity between IF and AAS score; however, it is not possible to determine exactly the reasons for this discrepancy. Relatedly, it was not possible to determine how all of the articles reached social media, whether through self-citation, publication by the journal's official Twitter account, or dissemination by societies such as the American Glaucoma Society or organizations such as the American Academy of Ophthalmology. It is also not possible to determine with accuracy what populations-the general public, scientists, or medical professions-are reading the articles and most driving the higher AAS scores. However, given the lack of correlation with IF and the much larger size of the general population relative to medical professionals, it is likely that the AAS score may be more related to general interest.
In conclusion, we herein present the first Altmetric analysis in the field of ophthalmology to help quantify public engagement with glaucoma research. This study identifies factors not captured by classic citation analysis and includes a more diverse set of journals than typically seen in bibliometric studies. However, it is important to be cautious when interpreting these results as articles can be sensationalized and receive wide interest even when not properly interpreted. Almetrics reflect crowd attention, which is not necessarily related to the quality or originality of the article. However, it does have its place in scientific research as it captures valuable information beyond traditional IF. This information can help scientists and clinicians understand where the general population may be currently focused and areas of their field which need to draw more attention. Altmetric and