Aaron J Barlow declared in 2012 that: “Blind peer review is dead. It just doesn’t know it yet” (Jaschik, 2012). He gave a number of reasons to support his view: pre-print online sharing is now the norm in certain sciences (for example, arXiv.org); pre-internet, there was serious limitations on printing and distribution; fairness is problematic in blind review; editors can use the review process unethically; and with the speed of the internet, reviewing slows down publishing, which hinders the careers of academics who need to publish or perish. Barlow’s opinions are echoed by Walker and da Silva (2015) who add more items to the charges against blind review. So is blind review dead in 2015; has Barlow been proven right?
One thing Barlow didn’t refer to was the notion of the gatekeeper, the quality control aspect of the refereeing process. Robson et al. (2015) don’t mention it either in their short list of the two:
“Purposes of peer:
• to assist editors who, given the diverse nature of the manuscripts, cannot expertly evaluate all that crosses their desks; and
• to assist authors in developing papers that will better contribute to scholarly knowledge” (p. 9).
In the comments section in to Jaschik’s article, the role of the gatekeeper was explicitly discussed. Readers are concerned about quality in the refereed journals they read. One source (Morrison, 2011) determines that there are over 45, 000 peer-reviewed journals producing in 2010 1.47 million papers (Meuschke, 2011). Our time is precious and when we read we need to know that what we read is useful.
How can readers be assured of quality in academic journals? Robson et al. (2015) claim: “there has been no published work that provides a detailed analysis of reviewer feedback in organizational or social sciences” (p. 11), and <irony>as their work was peer-reviewed, I do not need to check that statement</irony>. They do repeat a common sentiment made by top-notch journals: “On average, 85% of submissions are rejected, a testament to the high standards of the journal” (Robson, et al., 2015, p. 9) as an indication of the gatekeeper role reviewers have.
Yet, unsupported, these statements are meaningless. How many submissions are inappropriate to the topic of the journal but are of an acceptable standard of writing? How many have topics that have just been covered and that the editors feel are not presently useful for the readership? How many are of articles that present negative information, for example a failed replication study: often difficult to publish when editors want groundbreaking studies, not copies and certainly not negative replication studies?
Robson et al.’s article is highly recommendable as it aims to answer these questions as the submission process related to the social sciences. Even though the term ‘gatekeeper’ is missing, the detail in their analysis points to many aspects of the gatekeeper’s role. As a novice graduate student, I found much to take away as advice for my own career, including the obvious: “The fatal flaws—or those flaws of an article that lead to rejection—commonly have been found to relate to technical merit” (p. 16) and the comforting; a reviewer goes “into each paper with the attitude of saying yes to publishing and let the author talk me out of it” (p. 15).
Peer review may change, and there are serious pressures working for that change. However, novice writers need to know the system before they can change it. Peer review has many strong advantages. Wendler and Miller (2014) recognise the present reality of ‘publish or perish’ and that scholars are both the “contributors to and consumers of the literature” (p. 698). They note that the peer review process itself can be educational as requests for revision by reviewers can point out flaws in thinking, writing style and methodology. My own last peer-reviewed paper was sent back to me twice with over 1000 words of reviewers’ comments each. These comments helped me significantly, and I learnt much about subtle errors in how I conceptualised Rasch analytical methodology, and I learnt better how to communicate through reading the suggestions for rephrasing my text.
Jaschik, S. (2012). Kill peer review or reform it? Inside Higher Education. Retrieved from:
Meuschke, N. (2011). How many academic papers are published each year. Quora.com. Retrieved from:
Morrison, H. (2011). How many active scholarly peer reviewed journals. Simon Fraser University. Retrieved
Robson, K., Pitt, L., & West, D. C. (2015). Navigating the peer-review process: Reviewers’ suggestions for a
manuscript. Journal of Advertising Research, 55(1), 9-17.
Walker, R. & da Silva, P.R. (2015). Emerging trends in peer review: a survey. Frontiers in Neuroscience,
Wendler, D. & Miller, F. (2014). The ethics of peer review in bioethics. Journal of Medical Ethics, 40, 697-701.