Osteosarcoma is the most common primary malignant neoplasm of the bone in children and young adults. It is, histologically, characterized as a “high-grade primary skeletal malignancy consisting of spindle cells of mesenchymal origin depositing immature osteoid matrix”1. Although rare overall, it still is the third most common cancer in adolescents after leukemia and lymphomas2. With a bimodal pattern of incidence in adolescence and after the age of 603, osteosarcoma is commonly found in the metaphysis of the long bones of the lower extremity. It grows rapidly and can metastasize early in the course of disease to the lungs, other bones, and brain. Once metastasized, the course of management becomes a combination of limb salvage with surgical removal of metastases and chemotherapy.
Citation analysis is a relatively burgeoning field in the domain of medical research, serving the purpose of quantifying the scientific impact of articles on a specific category of research/topic. Citation analysis reveals the extent to which research-based articles have an impact on further research in their area. The number of times an article is cited is contingent upon several factors including but not limited to the year of publication, country and institution of origin, journal and its target audience (general or specific). The last decade has seen a considerable increase in citation analysis-based research on orthopedic surgery4–7, pancreatic cancer8, breast cancer9, and more broadly, on oncology10. To the best of our knowledge, however, there has not been a citation classic research based on osteosarcoma. Considering the vast plethora of studies published on osteosarcoma, there is a dire need for identification of classic landmark articles that have been published on this topic.
Given the aforementioned developments as well as lacunas in the field of research, our research aims to delineate characteristics of the 100 most-cited articles which made a significant impact in extending the knowledge in and around osteosarcoma over time.
Materials and Methods
This study made extensive use of the SCOPUS database to retrieve articles. SCOPUS is an Elsevier Inc. owned subscription-based database of peer-reviewed articles. It is one of the major databases, among others, which are primarily used for citation analyses purposes. We used the search term—“osteosarcoma” as article title only. This was done to ensure that only articles primarily investigating “osteosarcoma” as part of their research question were retrieved from the database. A total of 12,107 articles were retrieved from the database. Following sorting based on the total number of cites, information about the top 100 articles with respect to author, country of origin, date of publication, article name and journal were retrieved and tabulated onto an excel spreadsheet. Studies were divided into 3 categories (basic science, clinical science, and epidemiological studies). The years were divided into decades (1960-1969, 1970-1979, 1980-1989 and so forth). As older studies tend to accumulate a larger number of citations, in order to correct for the time elapsed since publication the total number of citations were divided by the years since publication to gain a value termed as the CY index or citation index. For the clinical studies, levels of evidence were calculated using the Journal of Bone & Joint Surgery-American levels of evidence guidelines. These guidelines are based on the original Levels of evidence published by the Centre of Evidence Based Medicine. For statistical analysis of data, Kruskal-Wallis test was used to analyze associations between the number of studies published within different decades and the total number of citations and citation index. Kruskal-Wallis test was also utilized to determine whether levels of evidence were associated with a higher number of citations and citation indexes. In addition, we also carried out Pearson χ2 test to see if there was any association present between the decades and the levels of evidence.
A total of top 100 most-cited articles were retrieved from the database. Only 1 article was filtered out as it was a veterinarian study investigating osteosarcoma in dogs only. Therefore, the 101st article was then duly included in the final list (Supplementary Table 1, Supplemental Digital Content 1, http://links.lww.com/IJSO/A3). The articles were published between 1967 and 2014. The range of citations was from 1888 to 153. Up till the present date (November 5, 2017), all articles were cited a total of 27,117 times with a mean citation per paper of 271.2. The most-cited paper with 1888 cites was “A human DNA segment with properties of the gene that predisposes to retinoblastoma and osteosarcoma” by Friend and colleagues. The paper was published in 1986 in Nature. However, when adjusted for the time the article had been published by using the CY index, the article with the highest CY index (85.0) was “Osteosarcoma treatment—where do we stand?: A state of the art review” by Leutke et al11. Table 1 shows the top 5 articles when adjusted according to CY index.
Majority of the publications were published in recent decades with 40 papers being published between 2000 and 2009, followed by 26 papers in 1990-1999 (Fig. 1).
The journal publishing the highest number of articles was the Journal of Clinical Oncology with 20 articles, followed by Cancer with 12 articles (Table 2). Majority of the journals were oncology/medicine journals. The only surgical journal to be included in the top 5 journals publishing material related to osteosarcoma was Clinical Orthopaedics and Related Research with 5 publications.
The most prolific authors were Picci, P. and Bacci, G. from the Istituto Ortopedico Rizzoli with 9 and 8 publications, respectively (Table 3). Memorial Sloan-Kettering Cancer center topped the list of institutions with the highest productivity in terms of articles in the top 100 most-cited list with a total of 12 publications (Table 4). On second was the Istituto Orthopedico Rizzoli with 8 publications. This was followed by National Institute of Health, Bethesda and the National Cancer Institute with 5 publications.
The most productive country was United States with a total of 63 publications, followed by Germany with 13 publications and Italy with 11 publications (Fig. 2). The list primarily included original articles (N=82) followed by reviews (N=14). Interestingly, 4 conference papers also found their place in the top 100 list. Majority of the studies were clinical (N=54) followed by basic science studies involving animals, laboratory-based research or genetics (N=43). Only 3 epidemiological studies found their place in the top 100 list. For clinical studies (N=54), we evaluated the levels of evidence based on the Journal of Bone & Joint Surgery-American guidelines. Majority of the studies were level IV (35.2%), followed by level I (22.2%) and level V (20.4%).
Kruskal-Wallis test revealed that the decade of publication had no correlation with the number of citations (P=0.199). However, articles published in recent decades tended to be cited more per year as shown by the CY index (P<0.001). There was no association between the decade of publication and level of evidence (P=0.059). A statistically significant association was found for a higher number of citations for high level of evidence clinical articles (P=0.038). However, along the same lines, recent researches tend to cite level V reviews more as shown by a statistically significant association between CY index and levels of evidence (P<0.001). We found no association between the type of the study (basic science, clinical or epidemiological) and the total number of citations. Figure 3 shows the proportion of different levels of evidence of articles being published over the decades.
Citation analysis serves to identify landmark articles on a specific topic. It helps in analyzing scientific impact while recognizing significant/path-breaking contributions by predecessors and key advances in research. When combined with analyzing levels of evidence, it also helps in understanding the quality of these top 100 most-cited classics. To the best of our knowledge, this is the first citation analysis research based on osteosarcoma.
The first study that was widely cited was “A human DNA segment with properties of the gene that predisposes to retinoblastoma and osteosarcoma” by Friend et al14 published in 1986. Perhaps, one of the reasons it is widely cited is that it introduced the concept of the RB gene predisposing to osteosarcoma among other cancers. This study bore evidence to the 2-hit hypothesis now taught to medical students worldwide. This important association set a milestone showing that there is a hereditary association involved in the incidence of osteosarcoma.
The second study “Prognostic factors in high-grade osteosarcoma of the extremities or trunk: an analysis of 1702 patients treated on neoadjuvant cooperative osteosarcoma study group protocols” by Bielack et al12 was published by the American Society of Clinical Oncology in 2002. This study reported the prognostic factors for osteosarcoma by analyzing several variables including age, site (axial or appendicular), primary metastases, and poor response. Some factors were speculated but left open for future research and development. The different modalities of treatment were also compared. This study comprised 1702 sarcomas and the large sample size has provided a quantum of knowledge for others to work with and to refer to.
The third most cited study was “Estrogen binding, receptor mRNA, and biologic response in osteoblast-like osteosarcoma cells” published in 1988 by Komm et al15. This study introduced the concept of estrogen having an effect on osteoblasts by inducing increased transcription and formation of bone matrix. This research was particularly important as it lead to an avenue of studying hormone receptors in osteosarcoma cells and their therapeutic implications.
The most prevalent trend is increase in the number of articles over the years. In the period of the 1960s, few researches were published with a major hike from the 1970s to the 1980s, and then a more impressive rise at the turn of the 21st century. The current decade shows a lower number of top-cited articles but that is to be expected as these articles have been published very recently and it is a well-established fact that recent articles are often cited less in the beginning followed by a peak in next 10 years or so. This accounts for the low number of recent articles in the top 100 cited list.
Another factor that was considered relevant to the impact an article made was the country of origin of the corresponding author’s address. The United States has dominated the research on osteosarcoma. The next most productive country was Italy and produced only 10% of the input. A multitude of factors will affect the choice of doctors to focus on this field of research. The reasons for this are varied, but most likely center around the funding available for research, quality of orthopedic departments at hospitals, impact of the disease on population, academic atmosphere, platforms for sharing ideas, etc.
In addition to the number of publications, the level of evidence from a clinical perspective is of great importance as well. Overall, it was observed that low level evidence articles dominated the field of the top 100 citation classics with around 35% of the clinical articles (N=54) being level IV in evidence, followed by level I studies. It appears that though there has been a shift in the publication of high level randomized controlled trials since 1990 onwards, researchers are prone/inclined toward citing level IV case series. This needs to be highlighted as future researchers need to be aware of the importance of high-quality articles. Citation analysis like ours, provide any impetus to change the trend toward higher level-evidenced articles.
When adjusted for the time since publication using the CY index, the manuscript with the highest CY index of 85.0 was “Osteosarcoma treatment—Where do we stand? A state of the art review” by Leutke et al published in 2014. Overall we saw that articles published most recently tended to have a higher CY index. This might indicate a change in citation influence trend over time.
The keyword “osteosarcoma” used in our search yielded papers published mainly in journals of oncology rather than surgical journals. The top 3 journals to receive the most (a total of 41) articles on osteosarcoma were those that discuss oncology rather than orthopedics. In fact, the highest number of articles received by an orthopedic-centered journal was 5 which was achieved by Clinical Orthopaedics and Related Research. This finding is expected as osteosarcoma, despite being termed as a multidisciplinary with regards to its cure, significantly requires input from an oncologist in addition to orthopedic surgeons, nuclear medicine, physiotherapist, and ancillary staff. Surgical management is usually considered after neoadjuvant treatment has been given by the oncologist. Another reason is that various surgical options of managing osteosarcoma, such as endoprosthesis and rotationplasty, are still continuously evolving and research is comprises low-quality case series.
Just as some countries are more productive than others, so are certain organizations that are at the forefront of research. Memorial Sloan-Kettering Cancer Center (USA) stands out at it contributed 9 articles. The second most common is the Italian IRCCS Istituto Orthopedico Rizzoli, Bologna. The Rizzoli institute is widely considered a worldwide frontier in all academic aspects of orthopedics and has produced numerous papers over time. Interestingly enough, the SCOPUS database indexed the National Institutes of Health and National Cancer Institute as 2 separate entities, when in essence they both operate under one roof of the National Institute of Health. If combined, the total output was 10 publications, thereby making it the most productive institute. With a vast repertoire of government-based T-32 funding grants, NIH still continues to be at the forefront of most research-based medicine worldwide.
Despite the status quo where North American hospitals manage to churn out more papers, these hospitals produced one of the top 5 most influential authors with respect to contributions. The top of the list was filled with names from Europe (Italy, Austria, and Germany). This reinforces the view that research on osteosarcoma is conducted widely in many countries, even though the United States, by pure power of numbers, generates more articles due to more avenues for publication of scientific findings.
The list of researches is an amalgamation of clinical, basic science, and epidemiological studies. Mainly, the list consists of clinical studies that discuss ways to diagnose and treat the condition. Basic science journals focus on the metabolic abnormalities seen in the condition and investigate potential prognostic factors, diagnostic measures and biochemical indicators for osteosarcomas. Only 3 epidemiological studies made it to this list because substantial funding is scarce to amass enough participants to take part in such huge studies to provide statistically significant associations.
There are several caveats in this study. Firstly, we placed the word “osteosarcoma” as the article title rather than the keyword. This was done to minimize other publications that were looking at a broad-based approach toward a large number of cancers. Secondly, a lot of researches tend to cite articles from established journals or authors. Although, no such studies have been conducted to investigate a regional publication bias, it is probable that high impact journals also in turn accept articles published from resource-rich countries rather than the low-middle income countries. This may skew the worldwide productivity on research topics. Thirdly, the differences in the indexing of various databases may reflect different overall citation numbers.
This citation analysis is a useful resource for many. Firstly, it is useful for those considering beginning research on this particular cancer. It gives them an idea of where they can find the location of like-minded peers, the milestones already achieved to avoid reinventing the wheel and the target type of journal to make sure their message reaches as large a readership as possible. Secondly, it guides clinicians as to where they can find the most useful and clinically relevant information. Thirdly, it also identifies people who made major contributions to the field of osteosarcoma research.
Although citation analysis may not be entirely flawless, it still remains the most standard way of analyzing scientific impact worldwide. We identified the top 100 most-cited articles on osteosarcoma. On the basis of our study, we conclude that future researches need to be catered toward producing high-level articles with respect to the number of citations. Trainees need be taught the importance of levels of evidence in research so they may accurately identify and cite high-level articles rather than referencing highly cited low-level articles. This citation list is dynamic and introducing the analysis of levels of evidence may bring about a change in numbers over time. Our list is comprehensive in terms of identification of the top-cited classic papers on osteosarcoma.
No ethical approval was required for this study.
Sources of funding
M.S.H. was involved in manuscript writing and data collection. A.T.M. was involved in data analysis and study design/conception. S.N. was involved in supervision and critical review of manuscript.
Conflicts of interest disclosure
The authors declare that they have no financial conflict of interest with regard to the content of this report.
Research registration unique identifying number (UIN)
Azeem Tariq Malik.
The authors would like to acknowledge Saman Tariq Malik, BA (Hons), who is currently a Rhodes Scholar pursuing an MPhil Degree in South Asian Literature at the University of Oxford, for her help in grammatical proof-reading of the manuscript.
1. Sissons HA. The WHO classification of bone tumors. Recent Results Cancer Res 1976;54:104–8.
2. Damron TA, Ward WG, Stewart A. Osteosarcoma
, chondrosarcoma, and Ewing’s sarcoma: National Cancer Data Base Report. Clin Orthop Relat Res 2007;459:40–7.
3. Mirabello L, Troisi RJ, Savage SA. Osteosarcoma
incidence and survival rates from 1973 to 2004: data from the Surveillance, Epidemiology, and End Results Program. Cancer 2009;115:1531–43.
4. Ahmad SS, Evangelopoulos DS, Abbasian M, et al. The hundred most-cited publications in orthopaedic knee research. J Bone Joint Surg Am 2014;96:e190.
5. Lee S, Shin J, Haro M, et al. Fifty most cited articles for femoroacetabular impingement and hip arthroscopy. Front Surg 2015;2:41.
6. Murray MR, Wang T, Schroeder GD, et al. The 100 most cited spine articles. Eur Spine J 2012;21:2059–69.
7. Nayar SK, Dein EJ, Spiker AM, et al. The top 100
cited articles in clinical orthopedic sports medicine. Am J Orthop 2015;44:E252–61.
8. Li Q, Jiang Y. Top classic citations in pancreatic cancer research. World J Surg Oncol 2016;14:298.
9. Uysal E. Top 100
cited classic articles in breast cancer research. Eur J Breast Health 2017;13:129–37.
10. Tas F. An analysis of the most-cited research papers on oncology: which journals have they been published in? Tumour Biol 2014;35:4645–9.
11. Luetke A, Meyers PA, Lewis I, et al. Osteosarcoma
treatment - where do we stand? A state of the art review. Cancer Treat Rev 2014;40:523–32.
12. Bielack SS, Kempf-Bielack B, Delling G, et al. Prognostic factors in high-grade osteosarcoma
of the extremities or trunk: an analysis of 1,702 patients treated on neoadjuvant cooperative osteosarcoma
study group protocols. J Clin Oncol 2002;20:776–90.
13. Ottaviani G, Jaffe N. The epidemiology of osteosarcoma
. Cancer Treat Res 2009;152:3–13.
14. Friend SH, Bernards R, Rogelj S, et al. A human DNA segment with properties of the gene that predisposes to retinoblastoma and osteosarcoma
. Nature 1986;323:643–66.
15. Komm BS, Terpening CM, Benz DJ, et al. Estrogen binding, receptor mRNA, and biologic response in osteoblast-like osteosarcoma
cells. Science 1988;241:81–4.