The public, funding agencies, and academic institutions are today increasingly interested in measuring the research quality and productivity of individual scientists as an indication of their scholarly excellence. Political or independent institutions evaluate and compare the scientific activities of individuals, departments, and universities and publish rankings as, for example, “The List of the Worldwide Champions League of Research Institutions 1994–1999” (http://www.cest.ch/en/) to use such “scientific” information for policy decisions. The growing field of studying the output of scientific and technological literature is termed “Scientometrics,” and it includes all quantitative aspects of the science of science.
A popular method for measuring the impact on the scientific community of an article or a researcher is the citation rating, which has become the modern “sales figure” for scientists. When publication A refers to publication B in the reference list, then publication A is the citing article and publication B becomes the cited article (1). Citation analysis is the area of bibliometrics that examines these relationships. The number of citations (i.e., how many times a given article is counted in the reference lists of subsequent articles) is seen as a direct measure of the recognition that this publication has had in its scientific field; in some circumstances, it is also used as a measure of quality by granting bodies. One criticism of citation analysis is that the databases do not distinguish between positive or negative credits or citations. Although there is obviously considerable debate regarding the value of citation rates used to assess the quality of research (2,3), analysis of citation rates may allow for the identification of seminal advances in a specialty and may provide a historic perspective on its scientific progress. Therefore, various specialties have recently analyzed their so-called “citation classics” (i.e., the top 100 articles most frequently cited or articles cited more than 100 times, depending on the size of the specialty) (4–6). In addition, various journals have published their own citation classics (7–11). The British Journal of Anesthesia(BJA) published the “BJA citation classics 1945–1992” of the top 50 BJA articles in 1998 (12), and Anesthesia & Analgesia published a “bestseller” list with respect to authors and articles in the field of pain and analgesia (13). Furthermore, such lists are also the basis for the so-called “landmark” series as for example published in the Journal of the American Medical Association (JAMA) during 1983–1984 (14). Although articles in the anesthesia landmark series such as “CITATION CLASSIC” published during 1998 in the BJA or “CLASSIC PAPERS REVISITED” currently published in Anesthesiology are selected based on the citation ratings of a single journal or recommendations from members of the editorial board, a comprehensive list of all top cited articles in the field of anesthesia is not yet available. The purpose of the present study was therefore to identify and examine the characteristics, such as ranking, year of publication, publishing journal, type of article, institution of origin, country, state, topic, and authorship, of the top 100 most frequently cited articles published in anesthetic journals.
Twenty-three journals dedicated to anesthesia and its subspecialties were selected from the Journal Citation Report 2001 under the subject category “Anesthesiology,” but five specialized pain journals were excluded form analysis (Table 1). The most frequently cited anesthetic articles published in the 18 analyzed anesthetic journals were identified using the database of the Science Citation Index Expanded (SCI-EXPANDED, 1945 to present). The SCI-EXPANDED is a multidisciplinary database powered by the ISI, formerly known as the Institute for Scientific Information and now owned by the Thomson Corporation, and its Web of SCIENCE®, with searchable author abstracts covering the journal literature of the sciences; it indexes more than 5700 major journals across 164 scientific disciplines, providing access to current information and retrospective data from 1945 forward, and contains a current total of over 17 million records with an average of 17,750 new records per week. The Cited Reference Search option of the Web of SCIENCE® returns a site-configured maximum number of results, which is limited to 500. To find more than the maximum number of results, the data can be searched in segments, e.g., by searching one year or one journal at a time for data. For journals publishing articles with high citation rates, even this segmented search for one year of data yielded, in certain years, more than the maximum number of results. In these cases, each author’s name, according to the author index of the respective journal and year, was “hand-searched” using the Web of SCIENCE® interface. Data stored for each reference in the database include, among other categories, the title, author names, institutions and addresses, journal, abstract, and keywords. The top 101 (101 publications are listed in Table 2 as there were 3 joint positions at the 99th rank) most frequently cited anesthetic articles were read and reviewed online (PubMed) by both authors; when relevant information was not available online, the articles were obtained in a printed format by direct library access. The articles were analyzed and the data tabulated according to their specific features.
Six-hundred-forty-one anesthetic articles were retrieved that were cited 100 times or more. Using the Web of SCIENCE® update of May 9, 2003, we selected the top 101 most frequently cited articles from this list for further analysis and ranked them according to the number of citations they received (Table 2). The most cited article received 707 citations and the 3 least cited articles received 197 citations. The mean number of citations per article was 283. The majority (50 articles) received between 197 and 250 citations, 23 articles received between 251 and 300, 12 articles received between 301 and 350, seven received between 351 and 450, six articles received between 451 and 550, and only 3 articles received more than 550 citations.
The top 101 citation classics were published from 1954 to 1997. The decade from 1980 to 1989 produced the most citation classics with 34 articles, followed by the decade from 1970 to 1979 with 29 articles. The most classic papers published within given years were 7 articles in 1991 and 6 in 1980. Sixty percent of the classic articles were published before 1984. The top-cited articles were all published in 5 high-impact anesthetic journals, led by Anesthesiology (73 articles) followed by Anesthesia & Analgesia (10 articles), BJA (10 articles), Anesthesia (6 articles), and Acta Anaesthesiologica Scandinavica (2 articles). Of the 101 articles, 78 were original publications, 22 were review articles, and one was an editorial.
The 101 top-cited articles originated from 9 countries, with the United States (USA) contributing 70 articles, followed by the United Kingdom with 15 (Table 3). Only 8 articles originated from non-English speaking countries (Germany 3, Sweden 2, and Belgium, Denmark, and Italy with 1 each). Within the USA, California leads the list of citation classics with 25 articles, followed by Massachusetts (8 articles) and Minnesota (7 articles). Twelve states contributed more than one article to the top 101 citation list (Table 4). Of the total articles, 58 originated from multi-institutional collaboration, 4 from multinational collaborations (Rank 19, 43, 49/2, and 62 in Table 2), and 43 from individual institutions. Sixty-eight different institutions produced the 101 top-cited articles; only 14 articles did not originate in an anesthetic department (Departments of Anatomy, Medical Sciences, Pharmacology, Physiology, Psychology, and Surgery). In all anesthetic citation classics, the first author was affiliated with an academic department. Ten institutions produced more than one citation classic article (Table 5). The Department of Anesthesia and Perioperative Care, University of California, San Francisco, California and the Department of Anesthesia, Stanford University, Palo Alto, California share the lead in this list with nine classic articles each. Only one nonanesthetic institution, the Department of Anatomy and Developmental Biology of the University College and Middlesex School of Medicine in London, United Kingdom, produced more than one classic paper. The number of authors of the top-cited articles ranged from 1 to 11. Fifteen articles were authored by a single author and 30 articles by 2 authors. Twenty-nine persons authored two or more of the top-cited articles. Table 6 presents a list of these “frequent authors,” which is clearly led by E. I Eger II, who authored nine classic papers. E. I Eger II and P. R. Bromage are the only authors who are first author on three anesthetic classic papers.
Pharmacology, excluding volatile anesthetics, is the main topic covered by these highly cited articles; 27 articles deal with pharmacological issues. Considerable attention also was given to volatile anesthetics (16 articles), circulation (12 articles), regional anesthesia (11 articles), lung physiology (9 articles), and analgesia and pain (8 articles). Other topics include anesthetic complications, airway management, and monitoring.
The list of the top-cited anesthetic articles (Table 2) identifies authors and topics that reflect major advances in anesthesia during the last 50 years, although some articles have been subjected to retrospective criticism for inaccuracies or unwarranted conclusions. Nevertheless, this list provides a fascinating insight into the history and development of anesthetic practice. For example, we find at position one Cousins and Mather with intrathecal and epidural administration of opioids; at position three Revill et al. with the description of a linear analog for evaluating pain; at position five Eger EI II et al. with the introduction of the minimum alveolar concentration as a standard of anesthetic potency; at 11th position Cormack and Lehane about difficult tracheal intubation; at 15th position Slogoff and Keats about perioperative myocardial ischemia; at 22nd position Owens et al. with the ASA physical status classification; at position 30 Ali and Savarese about the monitoring of neuromuscular function; at position 33 Brain and the laryngeal mask as a new concept in airway management; and at position 51 Bromage and Robson about toxicity of lidocaine. These articles are just a few examples of some of the milestone publications that have had an important impact on anesthetic practice. Other classic articles describe the pharmacology of anesthetic drugs such as the articles at positions 2 (ketamine), 14 (midazolam), 24 (isoflurane), 32 (propofol), 37 (mivacurium, isoflurane), and 94 (atracurium).
Most of the anesthetic citation classic articles are still regularly cited. The sum of the citations of an article is logically dependent on its publication year, as citations accumulate over time. Scientific papers usually are not cited until 1 or 2 years after their publication and generally reach a maximum after 3 to 10 years, at which time they continue to be cited, but at a less frequent rate (1). The decline of the citation rate (citations per year) can be described as the half-life of a publication and is determined to be the time when the citation rate has decreased to half of the maximum. It has been reported that the true impact and fame of an article often cannot be accurately assessed for at least two decades (5,7,10–12). This normal life span of a publication shows that evaluating the rank and significance of recent publications is, at best, limited; thus, articles that are not listed here will eventually deserve classic status. Another problem of citation behavior is the so-called “obliteration by incorporation,” i.e., original seminal work is absorbed in current knowledge and it is no longer explicitly cited. Interestingly, some authors of landmark series noted that the true intellectual milestones and discoveries may in fact not be found in the most cited articles themselves but in their reference lists (4,6,8), underlining the fact that the absolute number of citations an article has accumulated cannot be used as a sole measurement of its “importance.”
Several observations can be made from the articles included in our study concerning the authors and institutions of origin. Sixty-nine percent of the anesthetic citation classics originated in the USA. This is very comparable with the origin of citation classics in general surgical journals (78% United States) (4), in clinical dermatologic journals (75%) (5), and in otolaryngology-head and neck surgery journals (84%) (6). In the Citation Classics of JAMA, 95% of all articles originated in the USA (7). This finding confirms the overwhelming influence that the USA has on medical research because of its large population and the financial resources available to the scientific community. This dominance might even be fostered by a tendency for American authors to cite local papers and for European authors to publish in and cite American journals (4,6). An interesting difference can be found both between surgery and general medicine and anesthesia with respect to the institutions and states from where the classic papers originated. Whereas surgical classics (Massachusetts, New York, Pennsylvania, Washington, DC) and JAMA classics (New York, Massachusetts, Maryland, Pennsylvania) mainly originated from the East coast (4,7), California is clearly the leading state in producing anesthetic citation classics (Tables 4 and 5). Neither the University of California, San Francisco nor Stanford University, Palo Alto were among the top 12 institutions contributing to classic surgical articles, but they share the lead in producing anesthetic classics with nine articles each. The finding that academic productivity of surgical and anesthesia departments is not interlinked was not expected and lacks a good explanation; academic productivity obviously depends more on local individuals than on the cooperation between anesthetic and surgical departments.
Another observation is not surprising. The list of frequent authors on anesthetic citation classics (Table 6) gives a sampling of some of the world’s best-recognized anesthesiologists; however, not all famous anesthesiologists were mentioned, which is consistent with an earlier study in another medical specialty (6). Nevertheless it is tempting to speculate that publishing citation classics will help one to ascend the academic staircase; journal editors may consider inviting such authors to publish subsequent work or to write review articles in their journals. In many cross-discipline studies of top-cited articles, methodological publications and review articles predominate, thus reflecting the need of clinicians for a critical evaluation and summary of the rapidly expanding medical literature (5,7). In addition, referring indirectly to a multitude of additional references is another good reason for citing methodolo- gical or review publications, given the restrictions on the maximum number of allowed references of manuscripts used today by many journals. We were therefore surprised to see that original research articles accounted for 77% of all anesthetic citation classics, although 5 of the top 10 classics indeed were review articles (Rank 1, 2, 4, 6, and 9 in Table 2).
The citation analysis presented in this study has several limitations. First, our survey of the literature was limited for practical reasons to cited articles originally published in anesthetic journals. Therefore some anesthetic citation classics that were originally published in other medical or scientific journals are not recognized as such on our list in Table 2. For example, such articles include the descriptions of the development of electrodes for blood Po2 and Pco2 determination by J. W. Severinghaus and A. F. Bradley, which they published in different papers in the Journal of Applied Physiology. Their most important article (15) has been cited 820 times, which would put them somewhere at the top of anesthetic classics with respect to received citations. Other articles by these authors have been cited 370 times or more and obviously would also qualify as anesthetic citation classics (16,17). This is equally true for the article co-authored by M. A. Schumacher, another anesthesiologist, that described the cloning of the capsaicin receptor (18). This article was published in 1997 in Nature and has already received 875 citations. Furthermore, in the list of the most cited JAMA articles, two anesthetic publications were included (7). Foldes et al. (19) compared the toxicity of IV given local anesthetics and Taves et al. (20) examined the nephrotoxicity after methoxyflurane anesthesia; as of May 9, 2003, they have received 304 and 173 citations, respectively. This list of examples of classic anesthetic articles originally published in journals outside of the field of anesthesia is by no means comprehensive, and there are other such articles that are not examined in this study.
Further limitations of the study can be attributed to the inherent problems of citation analyses as previously described (2,3,13,21,22). It is well recognized that the reasons for citing specific articles may not be entirely appropriate. One major problem is “incomplete citing.” The assumption here is that scientists and authors are supposed to credit those in their reference lists who influenced them most significantly in their work; however, the truth is that many authors try to support, as best and as easily as they can, the interpretation of their own results when attempting to convince or “persuade” the readers (21). Another problem is oriented or biased citing, including various types of conscious or unconscious biases, such as self-citation (bias towards one’s own work), in-house (bias towards friends or colleagues), journal or powerful person (bias towards reviewers, editors, members of grant awarding bodies), review (bias towards secondary sources instead of the original discoverer), negative citation (bias towards potential negative credits), national and cultural (bias towards national parochialism), English language (bias towards publishing and referencing English articles), and omission bias (bias towards not referencing competitors or sources contradictory to one’s own results) (23). In addition, an editorial in the 10 January 2002 issue of Nature (Volume 415, page 101) entitled “Errors in citation statistics” points out another potential pitfall in citation analysis, i.e., inaccuracies in the database used for citation statistics, especially when articles are authored by a consortium rather than by individual scientists.
All these criticisms and biases should caution us when using citation rates to evaluate individual scientists (2,3,24); however, citation frequency can be considered as one of several valid and legitimate indicators in identifying classic work (6). We tend to remember those works that receive the greatest public recognition (7). This list of classic papers therefore represents a value or index related to how often a specific article has been used. These classic articles have influenced many people and should help to bring to our attention the many important advances in anesthesia made during the last 50 years. Truly classic articles will withstand the test of time by having a prolonged frequency of citations over decades. A citation index is not a measure of quality or importance but a measure of recognition (23) and the number of citations an article receives over the years tells us something about the impact of that article on the scientific community (2).
The authors thank Joan Etlinger for her expert editorial assistance.
1. Marx W, Schier H, Wanitschek M. Citation analysis using online databases: feasibilities and shortcomings. Scientometrics 2001; 52: 59–82.
2. Gisvold SE. Citation analysis and journal impact factors: is the tail wagging the dog? Acta Anaesthesiol Scand 1999; 43: 971–3.
3. Seglen PO. Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthop Scand 1998; 69: 224–9.
4. Paladugu R, Schein M, Gardezi S, Wise L. One hundred citation classics in general surgical journals. World J Surg 2002; 26: 1099–105.
5. Dubin D, Hafner AW, Arndt KA. Citation classics in clinical dermatologic journals: citation analysis, biomedical journals, and landmark articles, 1945–1990. Arch Dermatol 1993; 129: 1121–9.
6. Fenton JE, Roy D, Hughes JP, Jones AS. A century of citation classics in otolaryngology: head and neck surgery journals. J Laryngol Otol 2002; 116: 494–8.
7. Garfield E. 100 citation classics from the Journal of the American Medical Association. JAMA 1987; 257: 52–9.
8. Picknett T, Davis K. The 100 most-cited articles from JMB. J Mol Biol 1999; 293: 171–6.
9. Key JD. Citation classics: most-cited articles from Archives of PM&R. Arch Phys Med Rehabil 1988; 69: 1058–9.
10. Albert DM. Analysis of the Archives’ most frequently cited articles. Arch Ophthalmol 1988; 106: 465–70.
11. Key JD, Kempers RD. Citation classics: most-cited articles from Fertility and Sterility. Fertil Steril 1987; 47: 910–5.
12. Hall GM. BJA citation classics 1945–1992. Br J Anaesth 1998; 80: 4–6.
13. Strassels SA, Carr DB, Meldrum M, Cousins MJ. Toward a canon of the pain and analgesia literature: a citation analysis. Anesth Analg 1999; 89: 1528–33.
14. Lundberg GD. Landmarks. JAMA 1984; 252: 812.
15. Severinghaus JW, Bradley AF. Electrodes for blood pO2
determination. J Appl Physiol 1958; 13: 515–20.
16. Bradley AF, Stupfel M, Severinghaus JW. Effect of temperature on pCO2
of blood in vitro
. J Appl Physiol 1956; 9: 201–4.
17. Severinghaus JW, Stupfel M, Bradley AF. Variations of serum carbonic acid pK with pH and temperature. J Appl Physiol 1956; 9: 197–200.
18. Caterina MJ, Schumacher MA, Tominaga M, et al. The capsaicin receptor: a heat-activated ion channel in the pain pathway. Nature 1997; 389: 816–24.
19. Foldes FF. Comparison of toxicity of intravenously given local anesthetic agents in man. JAMA 1960; 172: 1493–8.
20. Taves DR, Fry BW, Freeman RB, Gillies AJ. Toxicity following methoxyflurane anesthesia. II. Fluoride concentrations in nephrotoxicity JAMA 1970; 214: 91–5.
21. Cole S. Citations and the evaluation of individual scientists. Trends Biochem Sci 1989; 14: 9–13.
22. MacRoberts MH, MacRoberts BR. Citation analysis and the science policy arena. Trends Biochem Sci 1989; 14: 8–12.
23. Dumont JE. The bias of citations. Trends Biochem Sci 1989; 14: 327–8.
24. Braun T. The reliability of total citation rankings. J Chem Inf Comput Sci 2003; 43: 45–6.