Impact Factor: A Call to Reason

Hernán, Miguel A.

doi: 10.1097/EDE.0b013e31819ed4a6
Impact Factors: Editorial

    A simple way to increase Epidemiology’s impact factor is to publish discussions on the impact factor. Let me explain. A year ago we published a commentary1 on the shortcomings of the bibliographic impact factor (BIF), a ratio calculated by Thomson Reuters (formerly Thomson Scientific) for many journals each year. One of these shortcomings is that the numerator of the ratio includes all citations to pieces published in a journal during a 2-year period, whereas the denominator includes only the number of original research reports and other substantive papers published during the same period. For example, the denominator of Epidemiology’s future BIF will include neither last year's commentary1 nor this editorial you are currently reading. However, the numerator will include the 1 citation that I made in the previous sentence. By the way, last year's commentary1 was published along with 3 responses2–4 and 1 editorial.5 In the previous sentence, I have managed to add 4 more citations to the numerator of our future BIF. But none of the 5 items cited in this paragraph—all of them published in Epidemiology—will be included in the denominator of the BIF. Amazingly, a journal's BIF can be increased simply by publishing commentaries and other nonsubstantive articles that cross-cite themselves.

    If you think that is not a reasonable property for a metric of scientific impact, keep reading because it gets worse. The editorial5 cited the commentary.1 That is, the editorial itself added 1 more citation to the numerator. Each of the 3 responses2–4 cited the commentary.1 That adds 3 more citations to the numerator. Further, we published 6 letters6–11 in response to the original commentary (6 additional citations in this sentence). These letters included 13 additional citations. In case you lost count, the entire series of editorials, commentaries, responses, and correspondence has so far generated 28 citations that will go to the numerator, but not to the denominator, of Epidemiology’s future BIF.

    Not bad, huh? Suppose we organize several discussions on interesting topics each year, and that we summarize them in editorials like this one. We could easily add over 100 citations and thus increase the journal's BIF. The greater the ratio of cross-referencing commentaries to original research articles, the greater the BIF. What then? Should we stop publishing discussions on hot topics so that the BIF is a more honest reflection of Epidemiology’s true impact? Well, we happen to believe that discussions of this sort have a rightful place in scientific journals. Rather than eliminating them to avoid inflating certain metric of impact factor, we would like to see a more reasonable metric that is not affected by citations to commentaries, editorials, and opinion letters. After all, as Colditz and Colditz11 argue on this issue, science needs impact metrics.

    One easy solution would be to redefine the impact factor as proposed in our previous commentary1: only citations to items in the denominator should be included in the numerator. Because Thomson Reuters already collects the raw data, recalculating this corrected BIF may only require changing a couple of lines of computer code. This simple modification would be a step in the right direction, although other problems remain.

    A big one is that even a corrected BIF would include only citations over the most recent 2-year period. Imagine that one of the articles published in this issue of Epidemiology describes a methodologic advancement that changes the design and analysis of all future follow-up studies. Unfortunately, because the publication of results from follow-up studies usually takes place more than 2 years after the inception of the cohort, an article with such a profound impact on epidemiologic research will have a null contribution to Epidemiology’s future BIF. Again, a fix to this limitation is easy: Thomson Reuters could provide not only the corrected BIF2 (ie, the corrected BIF based on the most recent 2 years) but also the corrected BIF5 and BIF10. This expansion would have no additional cost: a couple of additional columns (automatically generated after changing a few lines of code) in an electronically published database. In addition to better reflecting long-term journal impact, extending the period would likely result in more stable, harder-to-manipulate impact factors.

    Thomson Reuters, a for-profit company, is doing a great service to the scientific community by putting together a database of citations and making much of the information available to interested investigators. Yet references to Thomson Reuters in the scientific literature tend to be unflattering. Imperviousness to reasonable criticisms and unattended requests for transparency may be habitual in commercial enterprises, but are not acceptable to scientists. As a result, many scientists are working on alternative impact metrics that do not rely on proprietary information, and whose methodology is freely available. These new metrics are especially attractive for editors who cannot currently reproduce their journal's BIF, even when using the raw citation data provided by Thomson Reuters. It is possible that the BIF will not survive the emergence of these new metrics. Or perhaps Thomson Reuters can ask their programmers to press a few keys to produce a corrected BIF2, BIF5, and BIF10 that will remove some of the major criticisms of the current BIF. Why not?

    Back to Top | Article Outline


    1. Hernán MA. Epidemiologists (of all people) should question journal impact factors (with discussion). Epidemiology. 2008;19:366–368.
    2. Szklo M. Impact factor: good reasons for concern. Epidemiology. 2008;19:369.
    3. Porta M, Alvarez-Dardet C. How come scientists uncritically adopt and embody Thomson's bibliographic impact factor? Epidemiology. 2008;19:370–371.
    4. Rothenberg R. The impact factor follies. Epidemiology. 2008;19:372.
    5. Wilcox AJ. Rise and fall of the Thomson impact factor. Epidemiology. 2008;19:373–374.
    6. Castelnuovo G. More on impact factors. Epidemiology. 2008;19:762–763.
    7. Giuliani F, de Petris MP. More on impact factors. Epidemiology. 2008;19:763.
    8. Kogevinas M. More on impact factors. Epidemiology. 2008;19:876.
    9. Davey-Smith G, Shah E. More on impact factors. Epidemiology. 2008;19:876–877.
    10. von Elm E. More on impact factors. Epidemiology. 2008;19:877–878.
    11. Colditz IG, Colditz GA. What are we BIF-fing about? Science needs impact metrics. Epidemiology. 2009;20:462.

    Cited By:

    This article has been cited 4 time(s).

    “How Am I Doing?”
    Wilcox, AJ
    Epidemiology, 21(2): 163.
    PDF (52) | CrossRef
    A Correction Regarding Bibliographic Impact Factors
    Hernán, MA
    Epidemiology, 20(5): 785.
    PDF (50) | CrossRef
    Impact Factor: Let's Be Unreasonable!
    Brumback, RA
    Epidemiology, 20(6): 932-933.
    PDF (200) | CrossRef
    A Response From Thomson Reuters
    Testa, J
    Epidemiology, 20(3): 319-320.
    PDF (87) | CrossRef
    Back to Top | Article Outline
    © 2009 Lippincott Williams & Wilkins, Inc.