Characterization of the Conversion of Meeting Presentation to Publication From the 2016 and 2018 ISSVA Workshops : Journal of Vascular Anomalies

Secondary Logo

Journal Logo

Original Paper: Clinical study

Characterization of the Conversion of Meeting Presentation to Publication From the 2016 and 2018 ISSVA Workshops

Banyi, Norberta; Baweja, Sahdevb; Tuen, Young Jia,b; Bucevska, Marijaa,b; Arneja, Jugpal S.a,b,c

Author Information
Journal of Vascular Anomalies 3(4):p e050, December 2022. | DOI: 10.1097/JOVA.0000000000000050
  • Open

Abstract

Background

Scientific advances occur when research is shared through presentations at conferences and subsequently published in academic journals. Presenting novel research at annual meetings is a valuable method for sharing findings and gaining peer feedback, but many physicians and surgeons attend these meetings irregularly and abstracts are not always published. Moreover, the research presented at conferences often contains only partial results and analyses, which are finalized during conversion to a manuscript that is submitted to a peer-reviewed journal. Furthermore, the academic “bar” may be lower to have an abstract accepted at a conference compared to a journal. Peer-reviewed journal publications offer a more rigorous account of research findings and the potential to reach wider audiences than conference presentations, therefore acting as a crucial step in the scientific process.1

The underreporting of research findings impedes the advancement of clinical practice by creating a biased subset of studies represented by the literature and causing research redundancy. Despite the importance of publishing research, the majority of the work presented at scientific meetings goes unpublished. A 2018 Cochrane review of 307 028 conference abstracts found that only 37.3% resulted in full publication, which is significantly lower than the 44.5% conversion rate found in their 2007 review.2,3 Conversion rates have been found to vary between specialties ranging from 15.4% to 57.1%. Previous studies demonstrate that a lack of time is the largest barrier to journal publication.4 Other reasons for low conversions from presentation to journal publication are that this process may be of low priority for some authors, the authors presented results of an incomplete study that was never finished, and relocation or shifting responsibilities among research team members.4

While studies have characterized conversion rates of abstract presentation to publication over a wide range of specialties, there is currently no analysis of the conversion rate for abstracts presented at the International Society for the Study of Vascular Anomalies (ISSVA) conferences. This study aims to shed light on the destiny of ISSVA presentations. The primary objective of this study is to determine the rate of conversion from meeting presentations to publications from the 2016 and 2018 ISSVA Workshops. Secondary objectives include identifying the conversion rate by specialty and analyzing time from presentation to publication (TTP), as well as characterizing research study design, presenter country, publication journal, and the associated journal impact factor (JIF).

Methods

Conference programs from the ISSVA 2016 and 2018 workshops were obtained and used to identify presented work. Presentations that were keynotes, research letters, or education-related theses were excluded from further analysis. The remaining oral, poster, and difficult case presentations were searched using the PubMed interface (MEDLINE) and Google Scholar to identify if they resulted in a full-text publication.

Publications were searched for by the presentation title, keywords, and author(s) from the conference program. An initial search was conducted to identify publications yielding identical titles to presented abstracts. Where a match occurred, authors were compared to ensure overlap. For abstracts without a title match, author and keyword searches were carried out. Publications were defined as full-text papers including original research articles, reviews, letters to the editor, and opinion pieces. The criteria for conversion to publication were that at least 25% of authors remained the same, the first author or senior author from the abstract was also responsible for the publication, and that the contents of the publication were similar to the available conference abstracts. Three independent reviewers evaluated publications. Discrepancies and uncertainties were reviewed by all authors and settled by consensus. Data collected included presentation details such as title, authors’ names, type of presentation, country of the last author, and specialty of the last author. If the presentation was found to be converted into a publication, data collection also included TTP, study design and level of evidence, journal name, and corresponding 5-year JIF. Data collection occurred between September 8 and October 30, 2021.

Authorship parameters were derived from the last author of ISSVA conference presentations to maintain consistency between work that went on to be converted and work that did not. Author specialty was first reviewed in conference programs. The specialty of authors who are PhD trained and are not clinicians were coded to be “Basic Science.” If only institutional affiliations were present, authors were searched for on the institution’s websites. If the previous steps were unsuccessful, author names were searched online and past research was compared with their ISSVA materials for an overlap in subject matter. Specialty-specific conversion rates were compared between specialties with at least 15 presenters from the 2016 and 2018 meetings combined to ensure an adequate sample size.

The methods section of published work was examined to determine the study design, which was categorized as basic science or grouped based on a level of evidence hierarchy adapted from the 2011 Oxford Center for Evidence-Based Medicine classification hierarchy.5 The level of evidence assigned to a study ranges from level 1 (higher quality and lower risk of bias) to level 5 (higher risk of bias). The levels contain study types as follows: level 1 includes systematic reviews, meta-analyses, and critical appraisals; level 2 has clinical trials, cross-sectional studies, and comparative cohort studies; level 3 is made of case-control studies; level 4 is case series; level 5 includes case reports and expert opinion letters.

A database of presentations and their associated collected data was created using Google Sheets (Mountain View, CA). R version 4.0.2 (Boston, MA) was used to perform descriptive statistics to summarize the data and to perform Pearson’s chi-squared test to determine if there is an association between presentation type and conversion rate.

Results

Seven hundred two posters and oral presentations from the 2016 and 2018 ISSVA meetings were identified and 559 were eligible for our study. Of the 559 searched eligible presentations, 40.43% (226) were found to have an associated publication in a peer-reviewed journal (Table 1). The 2016 and 2018 meetings had publication rates of 42.61% (98/230) and 38.91% (128/329), respectively. Publication rates were highest in podium presentations, followed by poster presentations and then difficult case presentations with overall conversion rates of 50.6% (90/178), 37.9% (133/351), and 10.0% (3/30), respectively (P < 0.001) (Table 1). The overall median TTP was 16.1 (range: −28.9 to 66.4) where the TTP was 14.1 months (range: −13.3 to 66.4) from 2016 and 16.1 months (range: −28.9 to 41.1) from 2018. Excluding research that was published prior to presentation, the overall median TTP was 17.5 months (range: 0.1 to 66.4). Ninety percent of 2016 ISSVA presentations were published before 44 months after the meeting.

Table 1. - General Data Compared Between ISSVA 2016 and ISSVA 2018 as well as the Overall Combined Data
Parameter Analyzed ISSVA 2016 ISSVA 2018 Overall
Abstract presentations, n (%) 230 (41.14) 329 (58.86) 559 (100.00)
Publication, n (% conversion) 98 (42.61) 128 (38.91) 226 (40.43)
Months to publication, median (range) 14.12 (−13.32 to 66.41) 16.07 (−28.9 to 41.10) 16.07 (−28.9 to 66.41)
Presentation acknowledged in publication, n (% of published) 5 (5.05) 7 (5.47) 12 (5.29)
5-y journal impact factor, median (range) 3.35 (1.15, 27.47) 3.60 (1.15, 54.64) 3.49 (1.15, 54.64)
Podium presentation published, n/N (% conversion) 42/84 (50.0) 48/94 (51.1) 90/178 (50.6)*
Poster presentation published, n/N (% conversion) 53/134 (39.6) 80/217 (36.9) 133/351 (37.9)*
Difficult case presentation published, n/N (% conversion) 3/12 (25.0) 0/18 (0.00) 3/30 (10.0)*
Abbreviation: ISSVA, International Society for the Study of Vascular Anomalies.
*There is a statistically significant association between the presentation type and the conversion rate X2 (2, N = 559) = 20.1, P < 0.001.

The specialties with the highest conversion rates were Basic Science (60.5%, 23/38), Dermatology (55.42%, 46/83), and Plastic Surgery (42.48.%, 48/113) as illustrated in Figure 1. The 2 specialties with the lowest rates were Pathology (22.2%, 4/18) and Radiology (22.87%, 25/87). Overall, 226 papers were published in 111 separate journals, where the Pediatric Dermatology had the highest number of publications in a given journal (13 publications [5.8% of publications] in each journal) (Table 2). Case series and basic science studies comprised over three-quarters of publications, each accounting for 51.3% and 24.3% of study designs, respectively (Table 3). The median 5-year JIF of the journals where authors published was 3.35 (range: 1.15 to 27.47) for the 2016 meeting and 3.60 (range: 1.15 to 54.64) for 2018. Presenters from the 2016 and 2018 meetings were from 26 and 35 different countries, respectively. Of the 230 presenters in 2016 and 329 presenters in 2018, most were from the United States (114 in 2016; 144 in 2018), followed by China (18 in 2016; 23 in 2018) as seen in Table 4.

Table 2. - Journal of Peer-reviewed Publication for Converted Works at ISSVA 2016 and ISSVA 2018 as well as the Overall Combined Data
ISSVA 2016 ISSVA 2018 Overall
Characteristic N = 98 Characteristic N = 128 Characteristic N = 226
Journals n (%) Journals n (%) Journals n (%)
Pediatric Dermatology 8 (8.2) Journal of the American Academy of Dermatology 10 (7.8) Pediatric Dermatology 13 (5.8)
Annals of Plastic Surgery 5 (5.1) Pediatric Blood and Cancer 6 (4.7) Journal of the American Academy of Dermatology 12 (5.3)
Pediatric Blood and Cancer 5 (5.1) Pediatric Dermatology 5 (3.9) Pediatric Blood and Cancer 11 (4.8)
Journal of Investigative Dermatology 4 (4.1) Journal of Vascular Surgery: Venous and Lymphatic Disorders 4 (3.1) Annals of Plastic Surgery 8 (3.5)
The British Journal of Dermatology 4 (4.1) Annals of Plastic Surgery 3 (2.3) Journal of Vascular and Interventional Radiology 6 (2.7)
Journal of Vascular and Interventional Radiology 3 (3.1) JAMA Dermatology 3 (2.3) Journal of Vascular Surgery: Venous and Lymphatic Disorders 6 (2.7)
Pediatrics 3 (3.1) Journal of Vascular and Interventional Radiology 3 (2.3) The British Journal of Dermatology 6 (2.7)
The Journal of Pediatrics 3 (3.1) Lasers in Medical Sciences 3 (2.3) The Journal of Dermatology 5 (2.2)
Child’s Nervous System 2 (2.0) PLoS One 3 (2.3) International Journal of Pediatric Otorhinolaryngology 4 (1.8)
Frontiers in Surgery 2 (2.0) The Journal of Dermatology 3 (2.3) Journal of Investigative Dermatology 4 (1.8)
Other 59 (60.2) Other 86 (67.2) Other 182 (81.0)
49 other journals each containing < 2 frequency. 74 other journals each containing < 2 frequency. 101 other journals each containing < 4 frequency.
Abbreviation: ISSVA, International Society for the Study of Vascular Anomalies.

Table 3. - Study Design and Level of Evidence for Converted Works at ISSVA 2016 and ISSVA 2018 as well as the Overall Data
Study Design ISSVA 2016 (N = 98) ISSVA 2018 (N = 128) Overall (N = 226)
Level 1
 Critical appraisals 0 (0%) 0 (0%) 0 (0%)
 Meta-analysis and systematic reviews 4 (4.1%) 3 (2.3%) 7 (3.10%)
Level 2
 Cross-sectional study 4 (4.1%) 4 (3.1%) 8 (3.54%)
 Comparative cohort study 4 (4.1%) 8 (6.2%) 12 (5.31%)
 Clinical trial 2 (2.0%) 3 (2.3%) 5 (2.21%)
Level 3
 Case-control study 2 (2.0%) 3 (2.3%) 5 (2.21%)
Level 4
 Case series 41 (42%) 75 (59%) 116 (51.3%)
Level 5
 Case report 7 (7.1%) 2 (1.6%) 9 (3.98%)
 Expert opinion 3 (3.1%) 3 (2.3%) 6 (2.65%)
Basic science 29 (30%) 26 (20%) 55 (24.34%)
Other 2 (2%) 1 (0.8%) 3 (1.33%)
Abbreviation: ISSVA, International Society for the Study of Vascular Anomalies.

Table 4. - Country of Presenting Last Author Compared Between ISSVA 2016 and ISSVA 2018 as well as the Overall Combined Data
ISSVA 2016 ISSVA 2018 Overall
Characteristic N = 230 Characteristic N = 329 Characteristic N = 559
Country n (%) Country n (%) Country n (%)
USA 114 (49.0) USA 144 (44.0) USA 258 (46)
China 18 (7.7) China 23 (7.0) China 41 (7.3)
France 15 (7.7) The Netherlands 18 (5.5) France 27 (5.3)
Argentina 12 (5.1) Canada 16 (4.9) Canada 27 (5.3)
Canada 11 (4.8) Spain 14 (4.3) The Netherlands 24 (4.3)
Spain 9 (3.9) France 12 (3.6) Spain 23 (4.1)
Belgium 7 (3.0) Japan 12 (3.6) Argentina 20 (3.6)
The Netherlands 6 (2.6) Germany 10 (3.0) Japan 17 (3.0)
Japan 5 (2.2) Argentina 8 (2.4) Belgium 14 (2.5)
Germany 4 (1.7) Belgium 7 (2.1) Germany 14 (2.5)
Other 29 (12.6) Other 114 (34.7%) Other 94 (16.8)
16 other countries, each with< 4 authors. 25 other countries, each with< 6 authors. 29 other countries, each with< 9 authors.
Abbreviation: ISSVA, International Society for the Study of Vascular Anomalies.

F1
Figure 1.:
Total number of presenters and conversions by specialty of presenting senior author combined of the 2016 and 2018 ISSVA workshops. Conversion rate as a percentage is displayed on the x-axis adjacent to x-axis bar labels. Only specialties with a presenting frequency of greater than 10 were included. Abbreviation: ISSVA, International Society for the Study of Vascular Anomalies.

Discussion

Conversions

Similar to other academic conferences, less than half of the abstracts were converted at both the 2016 meeting (42.6%) and the 2018 meeting (38.9%). The higher conversion rate from 2016 is likely due to collecting data after a longer period since presentation. Approximately 10% of 2016 presentations were published more than 44 months after the meeting but our data collection only occurred 42 months after the 2018 meeting. Therefore, we estimate that the true conversion rate of the 2018 meeting will be closer to 43% than 39%. The overall 40.4% conversion rate found in this study is similar to the 37.3% conversion rate reported by Scherer et al in their 2018 Cochrane review that analyzed all specialties.2 While ISSVA’s conversion rates are in keeping with others, a previous study of three major plastic surgery annual meetings from 2003 to 2007 found conversion rates of up to 57.7%, suggesting that there is room for improvement at ISSVA.6

The higher conversion rates of podium presentations than poster presentations is congruent with the findings of Scherer et al.2 This was expected given that conference organizers tend to reserve the highest quality submissions for podium presentation. Nonetheless, our findings show a higher rate of conversion for both podium and poster presenters than the 43.1% podium and 24.6% poster presentation conversion rates found by Scherer et al.2 Compared to the others, difficult case presentations had a very low publication rate, but this may be because the primary intention of these presentations is to obtain professional advice from other conference attendees.

The consequences of underconverting are far-reaching. Some describe the underreporting of research findings as scientific misconduct because it disrupts discovery and understanding.7 Research redundancy utilizes resources that would have otherwise gone towards novel findings. Moreover, the reliance on incomplete or biased evidence for clinical decisions can lead to underestimated negative effects and overestimated positive effects, which can have detrimental impacts on patient health unbeknownst to healthcare providers.8 Additionally, there is an ethical obligation to ensure the dissemination of research where human participants sacrifice their time and face risks of harm with the understanding that they will be contributing to science.7

Past studies found that the reasons for underconverting were primarily due to a lack of time, followed by publication being a low priority, having incomplete results, and troubles between co-authors.4 It is important to foster a culture that places emphasis on ensuring that research is seen through to publication with sufficient protected time for research teams to publish their findings.

Level of evidence

The majority of the studies published were level 4 evidence, which is unsurprising given that case series are more feasible than randomized-controlled trials (RCTs), case-control, or cohort studies because of their lower cost and time requirements.9 It is important that science consumers are aware of the potential bias of case series, in particular selection bias. Moreover, while RCTs are thought of as the gold standard for primary literature, this study design is not uniformly available over different specialties. It is particularly difficult to perform an RCT in surgical settings due to difficulties in blinding, issues with generalizing results, and ethical limitations.10 Studies looking at the level of evidence of journal publications have likewise demonstrated a majority of publications are level 4.11–13 Initiatives to increase study designs to higher levels must be promoted and with ISSVA trending toward medical therapy for certain conditions, an opportunity may emerge for higher levels of evidence.14,15 Furthermore, registries and multicentered studies do have the potential to increase a study’s power and ISSVA is well-positioned to champion this work.

Subspecialties

Although many years of work often go into a PhD laboratory project and only a snapshot of this work can be feasible in a short abstract presentation, Basic Science investigators yielded the highest conversion rate at ISSVA. This may be due to basic science investigators having a higher academic requirements than clinicians.

The dermatology-specific abstract-to-publication conversion rate of 55.4% aligns remarkably well with the 55.7% conversion rate observed by Bolac et al studying conversion rates of a dermatology conference.16 The dermatology conversion rate is far above the average of clinical specialties and may be explained by relatively lower work hours as compared to other medical specialties.17 Similarly, our finding that 42.5% of plastic surgery presentations are converted to publications, is reflected by a 45% conversion rate characterized by a previous study looking at 888 presentations at plastic surgery conferences.6

Time to publication

Our overall 16-month median time to publication, which includes negative publication rates (journal publication prior to ISSVA presentation), was in alignment with the results of the 2018 Cochrane review by Scherer et al which also included negative time to publication (2). We identified 39 out of 226 (17.3%) presentations from the 2016 and 2018 meetings that were published 0.2 to 28.9 months before the meeting. Our search in October 2021 allowed for 58 months after the 2016 meeting and 42 months after the 2018 meeting. It is almost certain that the median time to publication following the 2018 ISSVA meeting is an underestimate given that the 90th percentile of TTP at the 2016 meeting is 44 months. Ultimately, it will be several years before a project is conceived to its final publication.

Journal and JIF

Of most significance, studies were published in 111 of different journals, with any one journal having a maximum of 13 publications. Publishing vascular anomalies literature in a wide range of journals may increase the difficulty for practitioners to keep up to date on new vascular anomalies research. Collating vascular anomalies research in a journal that can encompass all of these studies may improve the discoverability and readability of these publications in a one-stop shop. Furthermore, given the breadth and scope of subspecialties, an argument can be made for “program-based” rather than “specialty-based” journals since clinicians generally work in programs due the supra-specialization of medicine and the emergence of precision medicine.

The journals had a fairly high overall median 5-year JIF (3.5) with a wide range and no significant difference in JIFs between the 2 meetings. High-quality and impactful research is being presented at the studied meetings.

Limitations

This study may be limited in that conversions to publications were searched manually. While we developed a comprehensive search protocol to mitigate the failure to capture publications, it is possible that the MEDLINE and Google Scholar databases did not contain certain publications, the published research may have had a different title or list of authors from that of the abstracts, and works unavailable in English were missed. Furthermore, searching for publications 42 months after the 2018 ISSVA meeting may have limited our results and underrepresented the true abstract-to-publication conversion rates and underestimated the median TTP. There is strong evidence that presentations at meetings continue to be published upward of 4 years following presentations (2, 3, 6). Moreover, we did not account for the possibility of multiple abstracts being used for the same publication. Limitations may also exist in our search for authors. It is possible that authors’ online information is outdated or different than at the time of publication.

Future studies may examine, through surveys, why ISSVA presenters do not go on to publish their work to further characterize the barriers to publishing. Moreover, it would be interesting to see if the stage of a career is correlated to the publication rate.

Conclusions

The majority of studies presented at the 2016 and 2018 ISSVA meetings have not been converted to publications. This is in line with previous studies; nonetheless, there is an opportunity for improvement. Podium presentations are published at a higher rate than poster presentations, which in turn are published at a much higher rate than difficult cases. Moreover, a large portion of studies that were published had a low level of evidence. Initiatives to increase the publication rates and the level of evidence of research are critical to ensuring that research will be available and reliable to inform guidelines and clinical decision-making. To increase the discoverability of published work presented at ISSVA, capturing these works in a single, easily discoverable location should be promoted.

References

1. Foster C, Wager E, Marchington J, et al. Good practice for conference abstracts and presentations: GPCAP. Res Integr Peer Rev. 2019;4:11.
2. Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2018;11:MR000005.
3. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;2:MR000005. doi: 10.1002/14651858.MR000005.pub3.
4. Scherer RW, Ugarte-Gil C, Schmucker C, Meerpohl JJ. Authors report lack of time as main reason for unpublished research presented at biomedical conferences: a systematic review. J Clin Epidemiol. 2015;68:803–810.
5. OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evidence. Oxford Centre for Evidence-Based Medicine. https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf. Accessed January 12, 2022.
6. Gregory TN, Liu T, Machuk A, Arneja JS. What is the ultimate fate of presented abstracts? The conversion rates of presentations to publications over a five-year period from three North American plastic surgery meetings. Can J Plast Surg. 2012;20:33–36.
7. Chalmers L. Underreporting research is scientific misconduct. JAMA. 1990;263:1405–1408.
8. Dickersin K, Chalmers I. Recognizing, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the WHO. J R Soc Med. 2011;104:532–538.
9. Bhandari M, Joensson A, Ledet E, et al. Clinical Research for Surgeons. Thieme Verlag, Stuttgart, Germany; 2009. doi:10.1055/b-002-72200
10. Fregni F MK,. Limits to clinical trials in surgical areas. Clinics. 2011;66:159–161.
11. Molina-Leyva A, Descalzo MA, García-Doval I. Clinical research in dermatology and venereology in Spanish research centers in 2005 through 2014: results of the MaIND study. Actas Dermosifiliogr. 2018;109:52–57.
12. Chaudhry K, Bali RK, Kaur A, Tiwari RVC, Patnana AK. Level of evidence analysis in journal of maxillofacial oral surgery: a twelve-year bibliometric analysis of 1300 publications (2009–2020). J Maxillofac Oral Surg. 2021;20:364–372.
13. Sugrue CM, Joyce CW, Carroll SM. Levels of evidence in plastic and reconstructive surgery research: have we improved over the past 10 years? Plast Reconstr Surg Glob Open. 2019;7:e2408.
14. Sinno H, Neel OF, Lutfy J, Bartlett G, Gilardino M. Level of evidence in plastic surgery research. Plast Reconstr Surg. 2011;127:974–980.
15. Naji F, Javidan AP, Khan S, Srivatsav V, Rapanos T, Harlock J. Level of clinical evidence presented at the Society for Vascular Surgery Annual Meeting during a 5-year period (2012-2016). J Vasc Surg. 2018;67(3):951–959.
16. Bolac C, Orosco A, Guillet G, Quist D, Derancourt C. Publication des communications orales présentées aux « Journées dermatologiques de Paris ». Ann Dermatol Vénéréologie. 2009;136:21–27.
17. Leigh JP, Tancredi D, Jerant A, Kravitz RL. Annual work hours across physician specialties. Arch Intern Med. 2011;171:1211–1213.
Keywords:

abstract publication rates; vascular anomalies; conference presentation; peer-reviewed publication; conversion rate

Copyright © 2022 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of The International Society for the Study of Vascular Anomalies.