ArticlePlus
Click on the links below to access all the ArticlePlus for this article.
Please note that ArticlePlus files may launch a viewer application outside of your web browser.
The opportunities for successful publication of scientific manuscripts would seem to be abundant, given the proliferation and expansion of scientific journals. However, very few longitudinal cohort studies of research manuscripts have been conducted, and there are few data available on the actual publication rate of scientific papers. Such studies are difficult because there is no practical way to enumerate a cohort of initially submitted manuscripts.
Likewise, little is known about the path that successful manuscripts take on the road to publication. A few studies have described selected cohorts of manuscripts that were identified during the review process. These studies suggest that the majority of manuscripts are eventually published, although success rates vary.1–4 We use data from a survey of authors who submitted manuscripts to Epidemiology to reconstruct the journey and final status of this cohort of manuscripts.
METHODS
Our study population comprised all authors who submitted an original scientific manuscript to Epidemiology in calendar year 2002. Names and e-mail addresses of corresponding authors were identified using journal records. We contacted authors via e-mail in late October and early November 2004, providing them with an explanation of the survey’s purpose, the title of their submitted manuscript, and a direct link to a web-based questionnaire (set up through Zoomerang/ZPro5). We chose a web-based questionnaire for its convenience, and also because we thought its novelty would appeal to this research-savvy group of participants. Survey questions are provided in an online supplement to this paper, available on the journal’s web site. In a pilot survey among epidemiologists, the questionnaire required 5–10 minutes to complete.
In the survey, we asked participants whether their manuscript had been submitted to other journals before its submission to Epidemiology and, if so, to which journals. Authors whose manuscripts were declined by Epidemiology were asked additional questions about journals (if any) to which their manuscripts were subsequently submitted, and the current status of their manuscripts. Authors were invited to contact us with any questions or concerns. We sent one reminder e-mail notice to nonresponders.
An explicit question to obtain informed consent was built into the survey tool. The study was approved by the Institutional Review Board of the School of Public Health of the University of North Carolina at Chapel Hill (September 2004). In addition to the survey data, we used impact-factor data from Thomson Scientific as of 2004. We conducted simple descriptive analyses, which included the calculation of mean and median impact factors for groups of journals.
RESULTS
Epidemiology received 371 unsolicited original scientific manuscripts between 1 January and 31 December 2002 (not including letters, commentaries, book reviews, and other material). Of these, 101 were accepted for publication in Epidemiology (27%). Our survey response rate was 68% (n = 69) among the corresponding authors of the 101 accepted papers, and 58% (n = 157) among the authors of 270 rejected manuscripts. Of these respondents, we had to exclude 3 who later told us their response was in regard to a manuscript other than the one named. This left a total of 223 manuscripts for analysis. The follow-up period for manuscripts declined by Epidemiology (ie, time from rejection to survey interview) ranged from 19 to 34 months.
Of the 223 manuscripts with response data, we had complete submission history for 220. (We assigned the minimum number of possible rejections to the 3 manuscripts with incomplete history; these have trivial effects on the subsequent analyses.)
A total of 137 manuscripts (62%) had no history of submissions before their submission to Epidemiology. The remaining 83 manuscripts had been previously rejected by at least one journal. Within the total cohort of 223, the acceptance rate at Epidemiology was 30% (n = 68). (This acceptance rate is slightly higher than the acceptance rate for all submissions to Epidemiology in 2002 [27%], owing to the higher response rate among authors whose manuscript had been accepted.) The acceptance rate was the same for manuscripts previously rejected by another journal (30%) and for manuscripts being submitted for the first time (31%).
The subsequent history of manuscripts rejected by Epidemiology is shown in the flow chart (Fig. 1). Of the 155 manuscripts rejected, 140 (90%) were subsequently resubmitted or being prepared for resubmission elsewhere at the time of follow-up. The remaining 15 (10%) were never resubmitted (“inactive”). After a minimum of 19 months follow-up, 116 of the rejected manuscripts (75%) had been published or accepted for publication, 11 (7%) were being prepared for resubmission, while a total of 23 (15%) had become “inactive.” Taking the cohort of 223 manuscripts as a whole, 83% (n = 184) were eventually accepted for publication (by Epidemiology or by another journal), 7% were still being worked on, and 10% (n = 23) were inactive.
;)
FIGURE 1.:
The fate of manuscripts submitted to Epidemiology in 2002 (for which authors responded to our survey). In summary, 184 (83%) of 223 were accepted somewhere, 5 (2%) were under review elsewhere, 11 (5%) were being prepared for resubmission, and 23 (10%) were inactive.
We also considered the subcohort of 137 manuscripts that had no previous history of submission before Epidemiology. Although we accepted only 31%, 82% (n = 113) were eventually accepted within the period of follow-up. (Another 6% [n = 8] were still in process, and 12% [n = 16] had become inactive.) Among the 113 published papers in this subcohort, 70 (62%) were rejected by at least one journal (Epidemiology), and 11% were rejected by at least 2 journals before being published.
Journals and Their Impact Factors
Among manuscripts submitted to more than one journal, there was a general pattern of progression from higher- to lower-impact journals. In 2004, Epidemiology had an impact factor of 4.2. The 27 journals that rejected manuscripts before their submission to Epidemiology had a mean impact factor of 8.3 (median 4.5, range = 0.9 to 35). (Eleven percent of authors did not provide the title of the rejecting journal.) Manuscripts declined by Epidemiology were subsequently submitted to 95 journals with a mean impact factor of 2.5 (median 2.3, range = 0.3 to 7.2). (19% of the authors reporting a subsequent rejection after Epidemiology did not provide the journal title, while only 1 of the 116 with a successful subsequent submission did not provide the publishing journal’s name.)
The American Journal of Epidemiology (AJE) was the journal most likely to see a manuscript either before or after it had been submitted to Epidemiology. Of 26 manuscripts that had been previously rejected by AJE, Epidemiology accepted 6; in turn, AJE accepted 5 of 9 manuscripts that had previously been rejected by Epidemiology. The other journals most likely to review a manuscript before Epidemiology were The Lancet (n = 12) and the British Medical Journal (n = 7). Other journals likely to receive a manuscript after Epidemiology were Annals of Epidemiology (n = 6) and the European Journal of Epidemiology (n = 6).
DISCUSSION
Our cohort of manuscripts is a selected sample in several respects. Obviously, as a cohort of manuscripts submitted to Epidemiology, it represents a narrow spectrum of all biomedical manuscripts, and an incomplete sample even of all epidemiologic manuscripts. The effects of this selection cannot be addressed within our data.
Furthermore, the observed cohort does not represent all manuscripts that might have been submitted to Epidemiology, but rather only those that were not accepted elsewhere before being submitted to Epidemiology. Our cohort of manuscripts has thus been filtered by the priority authors gave to Epidemiology among all the journals that that they regarded as appropriate and by the manuscript’s success at any higher priority journals. (Of the manuscripts submitted elsewhere before being submitted to Epidemiology, we can observe only those that were rejected. This amounts to informative left-censoring of our cohort.)
A complete description of the fate of a cohort of manuscripts would require the identification of a cohort of manuscripts from their creation – that is, from the time of their preparation for first submission. Such a study is unlikely ever to be done. Unfortunately, all other designs (including ours) raise questions about selection bias. Information on rejection status and access to authors resides with journal editors, and previous studies have been conducted by or in conjunction with journal staff. Ray et al1 examined a sample of manuscripts rejected by Annals of Internal Medicine in 1993 and 1994 and found that 69% were eventually published, with a trend towards more specialized journals and decreasing impact factor. Opthof et al4 examined manuscripts submitted to Cardiovascular Research in 1995 and 1996, and found 70% of original manuscripts were rejected. OVID was subsequently searched for titles of rejected manuscripts, and 47% were found to be published in other journals, with a lower mean impact factor. Finally, Nemery1 considered manuscripts rejected by Occupational and Environmental Medicine from 1995 to 1997. Of the 44% of manuscripts that were rejected over these years, 54% were traced later in MEDLINE, with 90% of those published papers appearing in a journal with a lower impact factor.
In our study, we relied on author report of publication, and we did not validate author report with Medline tracing. As has been previously noted, not all journals are contained within MEDLINE, so that MEDLINE searches may underestimate the subsequent acceptance rate.1,3,4 Furthermore, manuscripts may evolve in their author order or title in ways that make the paper difficult to identify through searches.
We are unaware of specific descriptions of success rates for epidemiologic manuscripts, other than an editorial in AJE that compared acceptance rates with Epidemiology. Considering one volume of AJE (June–December 2004), the editors reported receiving 446 original contributions and accepting 15% of these.2
Perhaps the most striking limitation of our own study is the poor response rate (58% among the authors of manuscripts rejected by Epidemiology, and 68% among authors of accepted manuscripts). Other studies have reported low response rates for web surveys of professionals. A web survey of British general practitioners reported a response rate of 52% following 5 e-mail reminders.6 A study of response rates among surgeons reported a lower response rate in the web-survey arm (45%) compared with the mailed questionnaire arm (58%).7 Similarly, in studies conducted among staff of U.S. federal statistical agencies, poorer overall response rates were found for e-mailed surveys (43%) versus mailed paper surveys (70%).8 Although convenient to use, our e-mail survey may have appeared less legitimate than a paper survey on journal letterhead. Also, 3 authors reported having technical difficulties with the survey.
The lower response rate among the authors of rejected manuscripts is particularly problematic. Authors who ultimately failed to publish their manuscripts may have been particularly unmotivated to participate in our survey. To the degree that such “failed” manuscripts are underrepresented in our cohort, our estimates of ultimate success rates will be inflated. We explored this possible bias by assuming that the entire difference in response rates between the 2 groups of authors (58% versus 68%) was due to authors whose rejected manuscripts were never published (an additional 10% from among the 270 rejected manuscripts). If we add these hypothetical 27 nonresponders with failed manuscripts to our cohort of 223, the total proportion published would be 74% (184/250) instead of 83%, and the total inactive would be 20% (50/250) instead of 10%, with another 6% still in process. Similarly, among the hypothetical cohort of 182 manuscripts (155 + 27 inferred) rejected by Epidemiology, 64% were ultimately published and 27% were inactive.
The generalizability of our data is limited in other ways as well. Our cohort is a mix of manuscripts previously submitted to (and rejected by) other journals and manuscripts being submitted for the first time. As discussed above, the former group is selected in that we cannot observe papers that would have been submitted to Epidemiology had the manuscript not been accepted elsewhere. Interestingly, the proportion of papers eventually achieving publication did not differ for these 2 groups (83% and 82%). However, even the subgroup of manuscripts being submitted for the first time cannot be regarded as representative of all epidemiologic manuscripts, since this group includes only those for which our journal was the author’s first choice. Author’s preference is probably not unbiased with regard to the author’s previous experience with journals, and the topic and quality of the paper, any of which might be related to ultimate success of the paper.
While our results provide incomplete and no doubt biased estimates for the unknown cohort of all epidemiologic manuscripts, these data do suggest certain patterns. First, as has been observed in previous studies, authors of a rejected manuscript on average turn to journals with lower impact factors. This may be because authors presume such journals have higher acceptance rates (although we know of no data regarding that correlation). This trend was not universal: some manuscripts rejected by Epidemiology were subsequently published in journals with higher impact factor, and some papers rejected by a journal with a lower impact factor were subsequently accepted by Epidemiology. Our impact factor analysis was limited in that authors were less able to recall of the name of journals that rejected their papers than those that accepted their papers.
More importantly, it appears that authors who continue to submit their manuscripts to journals after rejection have a reasonable chance of publication. The direction of cause-and-effect is not clear; successful publication may be the reward of persistent authors, or better papers may provide the authors with confidence to continue to resubmit. Also, helpful reviewers may allow the manuscript to be improved along its path to acceptance.
While no amount of persistence can be expected to compensate for a fatally flawed paper, authors should be reassured by the fact that most published papers have a history of prior rejection. Among the papers we could follow from their first submission, 62% of those eventually published had been rejected at least once. If researchers are condemned to publish or perish, they should be encouraged to know that one or 2 rejections are not a death sentence.
ACKNOWLEDGMENTS
The authors thank Teresa P. Edwards of the Odum Institute for Research in Social Science for her assistance with this research.
REFERENCES
1. Ray J, Berkwits M, Davidoff F. The fate of manuscripts rejected by a general medical journal.
Am J Med. 2000;109:131–135.
2. What happens to your manuscript: characteristics of paper published in volume 160.
Am J Epidemiol. 2004;160:1234–1236.
3. Nemery B. What happens to the manuscripts that have not been accepted for publication in Occupational and Environmental Medicine?
Occup Environ Med. 58:604–607, 2001.
4. Opthof T, Furstner F, van Geer M, et al. Regrets or no regrets? No regrets! The fate of rejected manuscripts.
Cardiovas Res. 2000;45:255–258.
5. MarketTools, Inc. Zoomerang/Zpro. San Francisco CA: MarketTools, Inc.; 2004–2006.
6. Braithwaite D, Emery J, De Lusignan S, et al. Using the Internet to conduct surveys of health professionals: a valid alternative?
Fam Practice. 2003;20:545–551.
7. Leece P, Bhandari M, Sprague S, et al. Internet versus mailed questionnaires: a controlled comparison (2).
J Med Internet Res 2004;6:e39.
8. Couper MP, Blair J, Triplett T. A comparison of mail and e-mail for a survey of employees in U.S. statistical agencies.
J Official Stats. 1999;15:39056.
