Secondary Logo

Journal Logo

Editorial

From traditional Bibliometrics to Altmetrics: Socialising the research metrics

Bajwa, Sukhminder Jit Singh,; Mehdiratta, Lalit1

Author Information
doi: 10.4103/ija.ija_1058_21
  • Open

The academic world is currently facing a mad pursuit for publications. Parallel to this race, there is another race going on silently—the race of authors, journals and articles to make an impact. The author and inspirational speaker, Simon Sinek, once said ‘Genius is in the idea. Impact, however, comes from action’.[1] As if to endorse this statement, researchers and journals are currently trying various means to increase the impact of their published research.

THE RACE FOR ‘IMPACT’ AND THE CHALLENGE OF ‘METRICS’

Different types of metrics are used to measure the impact of published research in the world of scientific publications. Bibliometrics include the application of quantitative analysis and statistics to publications such as journal articles and their citation counts. Traditional journal-level metrics determine the impact a journal has on the scientific community. These include measures such as the Impact Factor (IF), Eigenfactor, Scimago Journal Rank, Source Normalised Impact per Paper (SNIP), CiteScore (CS), h5-index and Immediacy Index. Author-level metrics (personal bibliometrics) such as h-index and g-index determine the impact that an author makes on the scientific community platform. Article-level metrics are a newer approach for quantifying the impact of published articles. These include article access data such as HTML views, PDF and XML downloads, citation counts, expert evaluations on the F1000 Prime system and altmetrics.[2] In the current era of digitalisation and Open Access, the channels of scholarly communication are fast expanding digitally. Individual articles are tagged with digital object identifiers, and this exposes them to numerous channels of scholarly communication. Article-level metrics, including article access data, can thus be easily obtained.[2]

ALTMETRICS: REVOLUTIONISING AND SOCIALISING THE RESEARCH

Social media are now being increasingly used in medical education and research for teaching and learning. There is a mad scramble for the usage of social media tools in almost all walks of life. Most of us have ended up being less ourselves and more devoted to social media and devices. Traditional bibliometrics does not account for the role of social media in knowledge dissemination. This has encouraged approaches to evaluate the scholarly impact of articles through alternative assessment metrics or altmetrics. Altmetrics is a tool that evaluates published articles based on the extent of their use and sharing by audiences in social networks.[3] The Altmetrics Attention Score (AAS) quantifies the digital attention received by a scientific publication on various online media platforms, including mainstream news, blogs and social media networks.[4] These platforms include general social networks such as Twitter, Facebook and YouTube; scholarly social networks such as SlideShare, LinkedIn, ResearchGate, Academia.edu and online reference managers; bookmarking sites such as Zotero and Mendeley; scientific blogs and general blogs; open access repositories; Faculty of 1000 (F1000); collaborative encyclopaedias such as Wikipedia; and social news websites such as Reddit.[5] Companies such as Altmetric.com, PlumX, Lagotto, ImpactStory and Crossref Event Data offer altmetric services, and each of these companies tracks a combination of different sources and uses different formulae for calculation of the AAS.[6] The values of the AAS are freely available and easily identifiable at the centre of the colourful donut badge embedded in the published article. The altmetric calculations depend on factors such as the value (how many times an article is mentioned), sources (where do the mentions come from) and authors (of each mention).[7] The weightage for the calculation of the AAS varies according to the source in which a mention is made about the article. A mention in an online new channel has the highest weightage followed by blogs, and so on. Each source has an associated action such as ‘Likes’ and ‘Shares’ on social media, ‘viewed’ in HTML views and PDF downloads, ‘discussed’ or ‘mentions’ in blog posts or Wikipedia, ‘saved’ or ‘captures’ in social bookmarking sites and in electronic reference managers.[8] The altmetric dashboard provides a useful tool for tracking online conversations surrounding a research output.[4] In a fast era of ‘fast food’, ‘fast track’, ‘fast forward’ and so on, authors want to make a fast impact on their articles, and altmetrics allows them to do just that. Compared to traditional metrics such as citation counts, altmetrics is a faster metric of the popularity of individual articles, and by tracking various sources of social media in real time, it measures the impact of articles from the moment they are published.[9]

THE NEVER ENDING DEBATE ON ALTMETRICS

Founded in 2011, altmetrics have steadily evolved and are increasingly being incorporated into the databases of scientific journals, including anaesthesia journals. Nevertheless, publications from social sciences, humanities, medical and life sciences are most often found on social media platforms and show the highest preface of altmetrics.[10] Faculty are nowadays using altmetric reports to show the impact of their scientific work for academic promotions and while applying for research grants. Institutions, too, are using altmetric reports to find out the attention received by the faculty.[11] Altmetrics scores, particularly, mentions in blogs, though associated with low levels of recall, are able to identify highly-cited publications with higher precision levels than journal citations scores.[12] Studies in different medical disciplines have found a positive association ranging from strong to weak correlation between AAS and citation rates/journal impact factor.[13141516171819] However, evidence on how exactly the citations and research-related social media affect each other is not clear.[20] A study has found a direct and meaningful relationship between the compliance with Consolidated Standards of Reporting Trials (CONSORT) criteria and altmetrics score.[21] Mendeley readership is the most common altmetrics compared to others.[2022] Another study found that like citations, social media metrics increase with the length of the reference lists.[10]

Not surprisingly, several pitfalls are associated with altmetrics. Manipulation by self-citation or purchasing boosted alteration from social media tools is possible. Negative discussions about articles in social media are equalled to positive reviews. Tweeters of academic articles do not always engage in intellectual discussions, and the tweets do not necessarily reflect intellectual impact.[23] Non-academic users may sometimes dominate the discussion. A survey reported statistically significant differences in the frequency of usage of different types of social media-related acts between early-stage researchers and senior researchers, including professors.[24] The altmetrics patterns vary as per the journal article type. The results of a study in this regard showed that review articles had the highest median and mean of views, saves, shares and citations. Editorials and news items are not much cited but are more popular on Twitter.[10] Article viewing, saving and citing depend on the type of the article, meaning educational articles are highly saved but not as highly cited and opinion articles though highly viewed are not as highly saved.[25]

The interpretation of altmetrics is difficult because altmetrics are shaped and influenced by technical possibilities and affordances of different underlying social media platforms.[2627] Altmetrics are much more vulnerable to manipulation than traditional bibliometrics and may have a limited lasting impact. Altmetric scores indicate the social impact of a single article rather than the scholar’s view and criticism.[28] Altmetric tracks public Facebook Pages only and cannot track personal profiles, private accounts or group pages due to data access restrictions. Mendeley readers do not contribute to the AAS and do not appear in the altmetric badges. Altmetric does not track second-order citations such as links to news stories or blog posts about scholarly outputs.[29] A review has cautioned about various limitations and issues that can have an effect on altmetrics calculations and hence the need to have an open mind in the interpretation of altmetrics.[20] Another analysis of more than 10,000 randomly chosen publications processed by altmetrics providers demonstrated coverage bias with more than 65% blogs representing English-speaking countries.[30]

A study compared the ranking of five top anaesthesia journals by journal IF with ranking by altmetric scores. The study found that altmetric scores per article differed between journals. The journal with the highest IF for the year did not receive the highest altmetric score. Citations of the five journals correlated to their cumulative altmetric scores.[28] A study published in this issue of the Indian Journal of Anaesthesia (IJA) investigates the views of editors of high-impact journals on the AAS with the help of a questionnaire. The authors conclude that a high percentage of editors believe that AAS should be used to assess a scholarly output. Further, 76% of the editors are familiar with altmetrics and 28% know how AAS is calculated.[31]

This debate leaves one wondering whether altmetrics is merely an obsession. Will it stay? Is altmetrics equivalent to leaping and shouting to be soon forgotten? The noted Ghanaian writer and teacher, Ernest Yeboah, once said ‘We only shout when we neglect what silence can do’.[32] So, do we really need to shout to be heard and remembered? Has the scientific world passively opened itself to the commercial exploitation of attention gained by the use of social media? Nevertheless, whatever their strengths and weaknesses, altmetrics are fast expanding. Their use in different scientific disciplines, including the field of anaesthesia, is growing and several open-access journals are currently partnering with altmetrics providers. As already suggested by some authors, research on different aspects of altmetrics such as long follow-ups to evaluate if a high altmetric score continues to draw attention or expires soon needs to be conducted.[28] There are still several challenges in the meaningful use of altmetrics.[20] Nevertheless, bibliometrics and altmetrics are currently used as complementary scholarship impact assessment tools.[10] The altmetrics are being explored deeper and recently, they have been said to represent a trend towards the democratisation of research and its evaluation.[33]

As has been mentioned time and again in previous editorials of the IJA, conducting genuine, good quality, methodologically correct, statistically flawless, clinically useful, novel research and reporting it correctly as per CONSORT and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines should be our prime goals at the moment.[34353637383940] At this juncture, one can only say altmetrics or traditional bibliometrics… Does it really matter? How much further they can go in socialising and popularising the research, only time can convey.

REFERENCES

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
Copyright: © 2021 Indian Journal of Anaesthesia