Like many other fields in medicine, orthopaedic surgery is continuously evolving with new ideas and research. Presentations at national orthopaedic conferences provide an important avenue through which to highlight and disseminate new and innovative research. For the researchers who present at these conferences, the eventual goal is to publish their findings in a quality, well-read, peer-reviewed journal. Many barriers exist to publication, and the number of abstracts never even submitted to journals for publication may be as high as one-third.1 Especially for international researchers or private practice physicians, limited travel time and decreasing funding force many to select the optimal conference to attend or to submit research.
An important contributing factor to the practice of evidence-based medicine involves the publication of research in peer-reviewed journals. Although national conferences have their own peer review process that approves abstracts for presentation, projects presented at conferences may not be fully mature or may not pass the peer review process for a journal, and therefore never reach publication.1–6 The reported publication rates of different studies at various orthopaedic conferences have been shown to range from 28% to 67%.2,7–18
Studies have demonstrated an increasing trend in publication conversion rates for abstracts presented at the annual conference held by the American Academy of Orthopaedic Surgeons (AAOS) over the past 20 years.1,3,5,6,8,19–22 However, several factors may contribute to the increase in the number of studies published within the field of orthopaedics, such as an increase in the number of journals and online, open-access options.23 Few authors have reported impact factors (IFs) from abstracts published after presentation at orthopaedic meetings,2,5,6,24,25 and the other surrogates for quality such as CiteScore (CS) and Scopus Source Normalized Impact per Paper (SNIP), which attempt to quantify and rank the quality of journals by analyzing citations normalized by the number of publications,23,26–29 were not used.
The purpose of this study was to focus on abstracts in trauma and joint reconstruction. By evaluating studies presented at the AAOS, Orthopaedic Trauma Association (OTA), and American Association of Hip and Knee Surgeons (AAHKS) meetings, we sought to answer the following questions: What are the publication acceptance rates for abstracts presented at the AAOS, OTA, and AAHKS meetings? What are the differences in journal metrics for the published abstracts from each conference? What is the average time to publication (TTP)? What potential factors predict conversion to publication?
A review of all of the adult clinical and preclinical research abstracts accepted for either podium or poster presentation at the AAOS, OTA, and AAHKS conferences during 2015 was performed. Only research pertaining to orthopaedic trauma or arthroplasty was included in this study. Research abstracts that were filed under another subspecialty or under a forum other than trauma or arthroplasty were also included if they could also be categorized as trauma or arthroplasty. Eleven of 15 AAHKS poster abstracts filed under nonarthroplasty were excluded. Pediatric trauma was excluded, and video theater and scientific exhibits presented at AAOS were also excluded from the study. ScienceDirect, Ovid, Pubmed, and Google Scholar were queried to determine whether the research had been accepted for publication. The papers and posters were evaluated for the status of publication, publication date, journal in which the study was published, TTP, and country of origin. Studies conducted at institutions outside of the United States (US) were classified as international. In instances where authors collaborated between US and abroad institutions, the country of the senior author was used for analysis purposes. Time to publication was defined as the time between the first date of the conference and the date of publication in print of the manuscript. If this information was not available, then the acceptance date (if available) or online publication date was used. In instances of only the month and year of publication available, we used the first of the month. Initially, we searched for the published work using the complete conference abstract titles. If we could not find a paper with the same title, we pared down the title and included author names in the search criteria. If these methods did not yield results, a final effort was made searching institutional author profiles and bibliographies to review their published work for any matching criteria. When titles or authors differed between conference abstracts and journal publications, abstracts were compared to determine their similarities. Abstract titles, author groupings on abstracts, and digital object identifier for published abstracts were queried for matches between conferences using the Vlookup command with exact and approximate match options on Microsoft Excel (Redmond, WA).
Various journal metrics were used as a relative measure of the quality of the journal in which abstracts were published. The IF, recorded from the Clarivate Analytics journal database (Philadelphia, PA), the CS, the SNIP, and whether a journal was categorized as an Open Access Journal were recorded from the SCOPUS database (Elsevier, Atlanta, GA).
Time to publication, overall (any journal) publication rate, journal name, journal metrics, publication rates in the Journal of Orthopaedic Trauma (JOT), and The Journal of Arthroplasty (JOA) were compared. Descriptive statistics were performed using means and standard deviations for normally distributed data, and median and interquartile range (IQR) were used for nonparametric data. Categorical variables were compared using contingency tables. Data analysis was conducted using t-tests, Mann-Whitney U tests, chi-Squared tests, and Fisher exact tests where appropriate. A multiple logistic regression model was fit to identify any factors that may be predictive of publication. Associated odds ratios (ORs) and confidence intervals (CIs) were calculated for each factor. A Hosmer-Lemeshow test was performed on the regression model to asses for goodness of fit of the model. All analysis was conducted with an overall significance level of α = 0.05. The threshold for significance was corrected for multiple comparisons using the Bonferroni correction, αc = 0.0024.
There were a total of 516 (N = 213, AAOS; N = 303, OTA) trauma and 711 (N = 470, AAOS; N = 241 AAHKS) arthroplasty poster and podium presentations (Table 1). Regarding international abstracts, there were a total of 146 (N = 58, AAOS; N = 88, OTA) trauma and 141 (N = 132, AAOS; N = 9, AAHKS) arthroplasty presented at these conferences.
Overall, 60.4% of all abstracts accepted by OTA (N = 183) and 59.8% of abstracts accepted by AAHKS (N = 144) were published at median times of 306 (IQR: 75 to 469) days and 133 (IQR: 73 to 312) days after their respective conferences. Similarly, 62.8% of abstracts presented at AAOS (N = 429) were published at a median time of 148 (IQR: −34 to 446) days after the conference. Publication rates were similar between all conferences, P = 0.599. Fourteen published abstracts presented at OTA were also presented at AAOS. Three published abstracts presented at AAHKS were also presented at AAOS. Time to publication for abstracts presented at AAOS and AAHKS was similar, P = 0.76, and was quicker than that of abstracts presented at OTA (P = 0.002).
Twenty-nine percent, 18%, and 16% of the published AAOS, OTA, and AAHKS abstracts were published before their respective conferences. Of studies that were published, they mostly were published within a year of being presented with 70.2% AAOS, 57.4% OTA, and 78.5% AAHKS manuscripts being published within a year of their conference (Figure 1), and more than 90% from each meeting published by 2 years. The publication rates for trauma research presented at the AAOS and OTA meetings were similar at 57.2% and 60.4%, respectively (P = 0.54); likewise, the publication rates for arthroplasty research were similar at 65.3% and 59.8% for AAOS and AAHKS, respectively (P = 0.17).
For international abstracts, 57.9% (110/190) of AAOS abstracts and 62.5% (55/88) of OTA abstracts were published in a median time of 178 (IQR: −39 to 515) days and 315 (IQR: 29 to 441) days after their respective conferences. Six of 9 (66.7%) international abstracts from AAHKS were published in a median time of 71 (IQR: 98 to 157) days. Overall publication rates (P = 0.47) and TTP for international abstracts presented at AAOS and AAHKS were similar (P = 0.19), and also similar for those presented at AAOS and OTA (P = 0.47). International trauma abstracts presented at OTA (N = 55/88 published, 62.5%) were not more likely to be published than those presented at AAOS (N = 24/58 published, 41.4%), P = 0.01. International arthroplasty publication rates between AAHKS (N = 6/9 published, 66.7%) and AAOS (N = 86/132 published, 65.2%) were not different, P = 0.74. The percentages of the total accepted manuscripts as a function of publication time for international and US abstracts are presented in Figures 2 and 3, respectively.
Journal metrics and publication data for overall international presenters and US presenters are provided in Tables 1–3. Published abstracts presented at OTA had a significantly lower median IF compared with those published in AAOS (P < 0.001) or AAHKS (P < 0.001). The IF of publications between AAHKS and AAOS were similar (P = 0.003). AAOS abstracts had a significantly lower CS metric score compared with AAHKS, P < 0.001, but a significantly higher CS score compared with OTA, P < 0.001. Source Normalized Impact per Paper indicator scores were similar between abstracts published from the three conferences (P > 0.05). A total of 7.8% of published AAOS abstracts, 6.0% of OTA abstracts, and 2.8% AAHKS abstracts were published in open access journals.
Subspecialty Journal Publication
Of abstracts that were published, AAHKS arthroplasty abstracts were more likely to be published in The Journal of Arthroplasty, 69.4% (100/144), compared with OTA trauma abstracts published in the Journal of Orthopaedic Trauma, 33.3% (61/183), P < 0.001. Forty-four percent (135/306) of the published AAOS arthroplasty abstracts were published in JOA and 27.0% (33/122) of the published AAOS trauma abstracts were published in JOT. AAOS arthroplasty abstracts were less likely to be published in JOA compared with AAHKS (P < 0.001), whereas AAOS trauma abstracts had similar publication rates in JOT compared with OTA abstracts (P = 0.30). Trauma and arthroplasty abstract publication rates in The Journal of Bone and Joint Surgery (JBJS), Clinical Orthopaedics and Related Research (CORR), and Bone and Joint Journal (BJJ) were also compared. The proportions of studies published in these journals did not differ significantly between conferences (P > 0.0024 [all], Table 4). Publication differences in the Journal of the American Academy of Orthopaedic Surgeons were not assessed because the counts were too low for meaningful comparisons (less than 5 per conference).
Predictive Factors of Publication
Factors, associated ORs, and CIs when considering all abstracts in a multiple logistic model are reported in Table 5. Acceptance as a podium presentation was identified as a notable predictor of abstract conversion to publication (OR, 1.50; CI, 1.18 to 1.91). Accepted abstracts in arthroplasty trended toward having a higher likelihood of publication in comparison to trauma abstracts, (OR, 1.48; CI, 1.06 to 2.07). Neither being from a US institution nor an international institution predicted future publication (OR, 1.16, CI, 0.88 to 1.54). Acceptance by AAOS (OR, 1.17, CI, 0.84 to 1.63), OTA (OR, 1.40, CI, 0.86 to 2.28), or AAHKS was not a significant predictor of conversion to publication. The results of the Hosmer-Lemeshow test did not demonstrate a significant difference between observed and expected publication rates (P = 0.78).
The purpose of this study was to compare AAOS, OTA, and AAHKS annual conference meetings by assessing their contributions to published research in the orthopaedic subspecialties of trauma and arthroplasty. We also aimed to demonstrate the overall quality of abstracts presented at these conferences using multiple different journal metrics. To our knowledge, this is the first study to broadly evaluate and compare publication rates, TTP, quality of journal of publication for trauma, and arthroplasty research presented at these major international orthopaedic and subspecialty meetings.
Our study indicates that publication rates between AAOS, OTA, and AAHKS are similar, with 60% to 63% of conference abstracts going on to publication in a peer-reviewed journal within two years. These data are consistent with other published results, supporting an increase in publication rates of conference abstracts3,5,6,8,10,12,13,15,16,21 (Table 6). With the breadth of research submitted and presented at these meetings, review committees are seeking out higher quality research for presentation, which serves as a positive selection bias for publication. Likewise, with more options for publication, more high-quality research is likely to get published.
To assess the quality of research published from conference abstracts, various journal metrics and publication in open access journals were assessed. Based on study outcomes, acceptance by AAOS or AAHKS appears to be associated with publication in journals with higher IFs. The mean IFs of published manuscripts from all three conferences suggest that all three meetings present high-quality research because the journals of final publication have at least double the mean IF of other orthopaedic journals.23 Although the validity of the IF and other journal metric scores (ie, CS and SNIP) have come under scrutiny in recent years, the higher values suggest the studies are reaching broader audiences.23,26–29 Although there has been an increase in the number of open access journals, less than 10% of conference publications in our study were published in journals that identified as purely open access.
Our results show that podium presentations were more likely to result in full-text publication than poster presentations. This finding appears to validate the selection criteria of each of the three conferences in determining which studies are eventually going to be of the greatest impact.9,11–13,15,16 This finding should be considered by meeting attendees because they choose how to spend their time at the annual meetings.
The TTP averaged less than two years regardless of the presentation format. This finding was in contrast to other studies with the times to publication of presentations for other orthopaedic subspecialty meetings.2,6,8,12,13,15,16,20–22 Abstracts presented at AAOS and AAHKS were published in almost half the time as abstracts presented at OTA. It is unclear if submission deadlines or peer review process variations affected TTP.
Although overall abstract publication rates were comparable across the three meetings, there were some differences in where manuscripts were ultimately published. Abstracts from AAHKS were more likely to be published in JOA compared with abstracts from OTA being published in JOT. Arthroplasty abstracts from AAOS were also more likely to be published in JOA compared with trauma abstracts from AAOS being published in JOT. There was no difference in rates of publication for arthroplasty abstracts from AAHKS or AAOS or trauma abstracts from OTA or AAOS in JBJS, CORR, or BJJ. This discrepancy between arthroplasty and trauma abstracts in subspecialty journal publication rates could be due to a number of factors, including the peer review processes for the journals and conferences, choice of submission to the association-affiliated journal versus a nonaffiliated journal, intent to submit for publication, or intent to submit to general orthopaedic journals such as the Journal of the American Academy of Orthopaedic Surgeons or The Journal of Bone and Joint Surgery.
Publication rates for abstracts from outside of the United States were similar across the three conferences in our study, and did not differ markedly from publication rates for abstracts submitted from US institutions. Although publication bias is a concern for international authors, our study reported similar publication rates between international and US conference abstracts.19,30 This supports the peer review process of the three conferences in our study in selecting quality research regardless of the country of origin.
The strengths of this study are in the reporting of publication rates and journal metrics over a range of conferences. The authors, however, acknowledge several limitations. First, we assumed that the authors presenting at these conferences actively sought publication of their research soon after presenting it. Second, if a conference abstract changed markedly from presentation to publication, we may not have identified it as a publication and therefore underestimated our publication rate. Third, we used IF, CS, and SNIP as surrogates for research quality without analyzing the quality of the study design. Finally, we only analyzed 1 year's worth of abstracts and two subspecialties, and annual presentation and publication fluctuations as well as specialty-related differences may occur.
Nevertheless, the comparative publication rates along with the early distribution of accepted manuscripts and high journal metrics of the associated journals are reflective of high-quality research that is consistently being presented at the three annual meetings. Given that publication rates, TTP, and journal quality are comparable between all three conferences, the AAOS meeting appears to offer the greatest breadth of high-quality research. Thus, although we recognize that there are multiple reasons for attending these meetings beyond learning of new research, surgeons with broader practices for whom this is the primary goal may benefit from attending the Academy because it will expose them to quality material from various subspecialties.
1. Sprague S, Bhandari M, Devereaux PJ, et al.: Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. JBJS 2003;85:158-163.
2. Orr J, Dunn JC, Kusnezov N, et al.: Publication rate and impact factor for Society of Military Orthopedic Surgeons: Annual Meeting Presentations, 2009–2013. Mil Med 2017;182:e1992-6.
3. DeMola PM, Hill DL, Rogers K, Abboud JA: Publication rate of abstracts presented at the shoulder and elbow session of the American Academy of Orthopaedic Surgery. Clin Orthop Relat Res 2009;467:1629.
4. Guryel E, Durrant AW, Alakeson R, Ricketts DM: From presentation to publication: The natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J 2006;82:70-72.
5. Daruwalla ZJ, Huq SS, Wong KL, Nee PY, Murphy DP: “Publish or perish”—presentations at annual national orthopaedic meetings and their correlation with subsequent publication. J Orthop Surg Res 2015;10:58.
6. Voleti PB, Donegan DJ, Kim TW, Lee GC: Level of evidence: Does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? JBJS 2013;95:e2.
7. Amirhamzeh D, Moor MA, Baldwin K, Hosalkar HS: Publication rates of abstracts presented at pediatric orthopaedic society of North America meetings between 2002 and 2006. J Pediatr Orthop 2012;32:e6-10.
8. Donegan DJ, Kim TW, Lee GC: Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res 2010;468:1428-1435.
9. Janssen T, Bartels RH, Lind B, Tome CV, Vleggeert-Lankamp CL: Publication rate of paper and podium presentations from the European Section of the Cervical Spine Research Society Annual Meeting. Eur Spine J 2016;25:2311-2316.
10. Jasko JJ, Wood JH, Schwartz HS: Publication rates of abstracts presented at annual musculoskeletal tumor society meetings. Clin Orthop Relat Res 2003;415:98-103.
11. Kinsella SD, Menge TJ, Anderson AF, Spindler KP: Publication rates of podium versus poster presentations at the American Orthopaedic Society for Sports Medicine meetings: 2006-2010. Am J Sports Med 2015;43:1255-1259.
12. Lloyd EW, Geller JA, Iorio R, Yoon RS, Huo M, Macaulay W: Publication rates of scientific presentations at the American Association of Hip and Knee Surgeons annual meetings from 1996 to 2001. J Arthroplasty 2006;21(6 suppl):2e5.
13. Nguyen V, Tornetta P, Bkaric M: Publication rates for the scientific sessions of the OTA: Orthopaedic Trauma Association. J Orthop Trauma 1998;12:457-459.
14. Theman TA, Labow BI, Taghinia A: Discrepancies between meeting abstracts and subsequent full text publications in hand surgery. J Hand Surg 2014;39:1585-1590.
15. Yoon RS, Lloyd EW, McGrory B, Bal BS, Macaulay W: Studies presented in poster format at the annual meetings of the American Association of Hip and Knee Surgeons: How do they fare in the peer review process? J Arthroplasty 2007;22(6 suppl 2):17e20.
16. Preston CF, Bhandari M, Fulkerson E, Ginat D, Koval KJ, Egol KA: Podium versus poster publication rates at the Orthopaedic trauma association. Clin Orthop Relat Res 2005;437:260-264.
17. Gavazza JB, Foulkes GD, Meals RA: Publication pattern of papers presented at the American Society for Surgery of the Hand annual meeting. J Hand Surg Am 1996;21:742-745.
18. Scherer RW, Dickersin K, Langenberg P: Full publication of results initially presented in abstracts: A meta-analysis. JAMA 1994;272:158-162.
19. Bhandari M, Devereaux PJ, Guyatt GH, et al.: An observational study of orthopaedic abstracts and subsequent full-text publications. JBJS 2002;84:615-621.
20. Murrey DB, Wright RW, Seiler JG III, Day TE, Schwartz HS: Publication rates of abstracts presented at the 1993 annual Academy meeting. Clin Orthop Relat Res 1999;359:247-53.
21. Hamlet WP, Fletcher A, Meals RA: Publication patterns of papers presented at the Annual Meeting of the American Academy of Orthopaedic Surgeons. JBJS 1997;79:1138-1143.
22. Naziri Q, Mixa PJ, Murray DP, et al.: Adult Reconstruction Studies Presented at AAOS and AAHKS 2011-2015 Annual Meetings: Is there a difference in future publication? J Arthroplasty 2018;33:1594-1597.
23. Moverley R, Rankin KS, McNamara I, Davidson DJ, Reed M, Sprowson AP: Impact factors of orthopaedic journals between 2000 and 2010: Trends and comparisons with other surgical specialties. Int Orthop 2013;37:561-567.
24. Varghese RA, Chang J, Miyanji F, Reilly CW, Mulpuri K: Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America: Is there a difference between accepted versus rejected abstracts? J Pediatr Orthop 2011;31:334-340.
25. Whitehouse MR, Atwal NS, Blom AW: Publication rates for hip surgery-related abstracts presented at national and international meetings. Orthopedics 2009;32:407-.
26. Kodumuri P, Ollivere B, Holley J, Moran CG: The impact factor of a journal is a poor measure of the clinical relevance of its papers. Bone Joint J 2014;96:414-419.
27. Haddad FS: The impact factor: yesterday's metric? Bone Joint J 2014;96:289e90.
28. da Silva JA, Memon AR: CiteScore: A cite for sore eyes, or a valuable, transparent metric? Scientometrics 2017;111:553-556.
29. Leydesdorff L, Opthof T: Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. J Am Soc Inf Sci Technol 2010;61:2365-2369.
Copyright © 2019 Texas Tech University Health Sciences Center El Paso. Published by Wolters Kluwer Health, Inc. on behalf of the American Academy of Orthopaedic Surgeons.
30. Harris IA, Mourad M, Kadir A, Solomon MJ, Young JM: Publication bias in abstracts presented to the annual meeting of the American Academy of Orthopaedic Surgeons. J Orthop Surg 2007;15:62-66.