Abstract
Peer review is widely regarded as the cornerstone of modern scientific publication. But as academic publishing undergoes a digital transformation, the traditional peer review model has been increasingly challenged for its reliability. Reforms have been proposed and implemented to improve the robustness of peer review. This book looks behind one such set of reforms in peer review and discusses the potential institutional change it has brought to scientific communication in general.
The idea of peer review originated in the 17th century, and it has become the gold standard for quality assurance in scholarly publication. However, the peer review system is not without shortcomings. One possible issue is pertinent to accuracy: decisions can be prone to errors and potentially biased because the judgement is subjective and totally dependent on reviewers’ expertise in the field. Another possible issue is efficiency: it takes a large amount of time and effort for a peer-reviewed paper to be published, which in turn slows the speed of scientific communication. Furthermore, it remains largely unclear how a manuscript is assessed because the review process is not always transparent.
PLOS ONE, a multidisciplinary open access journal, was launched in 2006, aiming to publish papers that are methodologically and ethically sound, regardless of the novelty or significance of the findings. The journal operates on the basis that editors and reviewers evaluate how a piece of research was conducted and reported, not on what it has added to the existing literature. It is then up to the readers to decide whether the findings reported in a paper is innovative or important. This idea may sound radical, but it does make a clear point on what deserves to be published as a scientific paper. Over the past 15 years, PLOS ONE has grown into one of the largest peer-reviewed open access journals in the world, boasting more than 260,000 published articles in total.
To what extent has the success of PLOS ONE brought about institutional change in peer reviewing and academic publishing? Eve and colleagues seek to answer this question by analysing referee reports sampled from a corpus of peer-review reports provided by the journal. Several interesting findings concerning the new peer reviewing process are revealed. First, although the journal set new guidelines for peer reviewing, most reviewers still follow the traditional report structure and patterns in writing their comments. Second, despite PLOS ONE’s redefinition of the long-standing selection criteria from originality to ‘technical soundness’, most of the reviewers still comment, either in a positive or a negative manner, on the originality and potential impact of the paper they review. Last, similar to previous findings on peer review, feedback tends to be written in a rather direct and blunt manner to avoid misunderstanding.
Many other newer academic journals also use novel peer-review models. For example, journals based on platforms such as F1000Research, Copernicus Publications, and Open Research Europe implement various post-publication peer review models to evaluate whether a paper should be accepted for publication. Under such models, editorial and peer-review process are open and transparent to the public. Specifically, a manuscript is immediately posted online or ‘published’ as a preprint once it has passed a set of prepublication checks (e.g. format, completeness, ethics issues). Then the paper is subjected to peer review by invited reviewers or volunteer reviewers, and authors are encouraged to respond to reviewers’ comments and revise their manuscript if necessary. Finally, the article, review comments, along with author’s responses will be officially published online if the paper is accepted. In this regard, the authors of the present book say, ‘PLOS’s early radicalism paved the road for these types of platforms in ways that would not have otherwise been possible’.
As long as PLOS ONE attracts a huge number of submissions, its influence on ‘government policies, funding mandates, and even the economics of scholarly communications’ will continue to grow. On the other hand, the idea that PLOS ONE wants to promote has, it seems, not been fully adopted by referees who are also members of a broader research community. The implication here is that in challenging referees’ behaviour and the community culture, any reform of peer review faces a long road.