Rebound Peer Review: A Viable Recourse for Aggrieved Authors?

“ T he worst except all the others that have been tried”—Winston Churchill's quote on the democratic form of government may apply to the process of peer review. The art and science of knowledge generation has a long history. Indians, Babylonians, and Egyptians recorded some of the earliest empirical scientific ideas. Ancient Greek philosophers refined the process of knowledge generation by introducing the concepts of measurement (Aristotle) and reasoning (Plato). Aristotle (384 BC) erected the pillars of measurement and observation to bear and further science. He pioneered what we today know as scientific methodology by advocating study of relevant work of others, seeking general consensus about the subject, and systematic study of all information directly or even tangentially related to the topic in question. This approach captures the essence of what we know today as peer review and relies much on access to archives of knowledge. Archiving of knowledge is on record since the Syrian Ebla tablets of 2500 BC. The great library of Nalanda University (427 BC; Nalanda in Sanskrit means “provider of knowledge” and originated from nalam [lotus, representing knowledge] and da [to give]) was so vast that it is reported to have burned for 3 months after Persian invaders set fire to it. The Greek library of Alexandria (200 BC) is recognized as the first cataloged library, an essential resource enabling scholarly peer review [peer: someone “of equal standing with another…especially belonging to the same societal group…or having the same status” (1); review: a critical inspection or examination, or a second or repeated viewing of past events, circumstances, or facts (3)].

In the 19th century, biomedical journals followed the model of general journalism and appeared as personal organs. Only sometime after World War II did the practice of editorial peer reviewing get generally adopted. The development of editorial peer review process was chaotic as they were not developed from editorial boards and passed on from journal to journal. Casual referring out of articles on an individual basis started in the early to mid-19th century. Institutionalization of the process in the 20th century is suspected to have been driven by sharply higher submission volume and/or a growing demand for expert authority and objectivity in an increasingly specialized world (9). Interestingly, The Lancet was conducted independent of any peer review process until 1976 as the editors considered it unimportant. The Journal of the American Medical Association conducted in-house peer review and only on rare occasions would they send articles to outside experts (8). In 1893, the British Medical Journal is believed to have pioneered the peer review process by sending every noneditorial submission to an external recognized expert (9).

Today, refereeing or scholarly peer review represents the linchpin of academic publishing. Benefits of the peer review system are many including helping improve article quality (11, 14, 17, 28, 30) to providing a credible seal of approval by the community of science. It is the process of peer review that validates a scientific pronouncement and separates it from self-serving publications such as an advertisement or press release (12). The quality judgment of peers tightly correlates with citation scores of the publication (29). Critics of the system raise several valid concerns that deserve serious attention (21, 23). Several studies show that the current peer review system lacks robustness (10, 15, 19, 22) and is subject to expert bias (27). The rate of agreement between reviewers is low. Peer review is often viewed as being biased toward well-established research groups and the scientific status quo. Well-established and widely known authors have an unfair advantage to dominate as reviewers are unwilling to reject papers from such contributors out of fear. Dissent discoveries and findings against “mainstream” theories are likely to be suppressed. This is reminiscent of Galileo's championing of heliocentrism when most of his contemporary peers subscribed to either geocentrism or the Tychonic system (26). Reviewers tend to be especially critical of conclusions that contradict their own views and theory, even if the supporting data is good. Ideas that harmonize with that of the leaders in the field are more likely to be accepted for publication in prestigious journals than heretic or radical ones risking leapfrog innovation. Unscrupulous reviewers can reject papers and then quickly publish similar work themselves (13). Authors of a recent Cochrane review concluded, “at present, little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research” (18). In July of 2011, the House of Commons Science and Technology Committee published a comprehensive report “Peer review in scientific publication.” The committee acknowledged flaws but concluded that prepublication scholarly peer review is vital and cannot be dismantled. The report highlights much-needed improvements to the process (7, 24, 25).

Vast majority of current prepublication scholarly peer review is partially blinded such that reviewer identity remains concealed while author identity is known to reviewers. This approach is also referred to as simple blind review. In a completely blinded setting, reviewers should not know the author's identity, as any identifying information is stripped from the document before review. In the partially blinded or blinded review system, referees do not act as a group, do not communicate with each other, and generally have no knowledge of the identity or the results of others. Generally, there is no need for consensus. The group dynamic is very different from that of a jury where consensus in sought. Frequently, there are situations in which the reviewer's opinion is inconsistent puzzling authors who are invited to revise the article. Recent work by Jackson et al. (17) identified that although logistics may represent a practical barrier, a larger number of reviewers improves accuracy of the partially blinded peer review system. A diverse pool of reviewers to guard against overrepresentation of a narrow viewpoint is essential. When possible, ARS engages a panel of 4–6 reviewers to evaluate its original research communications. Extreme opinion of any single reviewer is viewed in light of comments by other peers. Such approach helps preserve author interest as long as they are willing to accept being reviewed by a panel that is double the size of current standards. Use of a larger panel also helps evaluate work representing interdisciplinary sciences. There is hardly any aspect of human health where redox biology is not of outstanding significance. ARS commonly receives articles that represent a variety of disciplines including but not limited to cell and molecular biology, omics sciences, chemistry, structural biology, systems biology, biophysics, translational sciences, and clinical sciences. The field of redox biology continues to be in a highly proliferative phase. In these developmental years, outright rejection of promising (but not ready for publication) original research articles does not seem to be in the best interest of the discipline. ARS utilizes the strengths of an interdisciplinary panel of 4–6 peer reviewers to substantially improve the quality of original research that is eventually published. This, we believe, is pivotal in elevating the niche field of redox biology to a higher level of prominence in the broader biomedical research community. The game plan relies on enhanced reviewer-author cooperation in improving the process of peer review and elevating the quality of science being peer reviewed. Willing peer reviewers and authors accepting the burden of a higher level of scrutiny represent the bedrock supporting the success of ARS as an impactful journal and providing the emergent discipline of redox biology a respectable home. Indeed, a recent study demonstrates that improved author-reviewer cooperation is highly desirable (20). The ARS system of evaluating original research communications is yet another example of how peer reviewers, without being awarded any formal credit, help improve science perhaps more so than some with authorship credit named on the publication.

“The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability—not the validity—of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong”—Richard Horton, editor of The Lancet (16).

Acknowledging the drawbacks of partially blinded or blinded peer review systems, open peer review has been adopted by a few journals (6). Here, authors know who the reviewers of their work are. Advantages to the author include the ability to openly discuss their work with reviewers and to deal with reviewers' suggestions on their perceived merit, without fear of rejection affecting the response. Further, the reviewers are publicly acknowledged for their contributions which often play a critical role in elevating the overall quality of the published work compared to version initially submitted. On the downside, in the open review system reviewers may not feel comfortable to submit their brutally honest comments. As a result, open reviewers may be more tolerant of mediocre science. ARS utilizes a variant of the open peer review system for the evaluation of Comprehensive Invited Reviews. These articles are extensive and aimed at comprehensively covering the topic in question. A panel of 6–10 peers, credited on the article as Reviewing Editors, representing different facets of the topic in question reviews the work and provides guidance to the authors such that the final review article is balanced in nature and helps shape the emergent field.

Impactful improvements of the peer review process are of outstanding significance and have been attempted. For example, Nature conducted a peer review trial that lasted for 4 months, from June to September 2006 (2). Authors could choose to have their submissions posted on a preprint server for open comments, in parallel with the conventional peer review process. Anyone in the field could then post comments, provided they were prepared to identify themselves. Once the usual confidential peer review process was complete, the public open peer review process was closed and the editors made their decision about publication with the help of all reports and comments. Nature reported on the results of the trial in December 2006. Despite the significant interest in the trial, a paltry 5% of authors opted to participate. There was a significant level of expressed interest in open peer review among those authors who opted to openly post their articles and who responded after the event, in contrast to the views of the editors. A small majority of those authors who did participate received comments, but typically very few, despite significant Web traffic. According to the editors, most comments were not technically substantive. Marked reluctance among researchers to offer open comments was recorded. Nature chose to continue to explore participative uses of the Web and decided against implementing open peer review. The report was discussed in an editorial published in tight succession (5). To cater to the needs of those authors who would like to more enthusiastically adopt the open peer review system, WebMedCentral was founded in 2011 as a generalist online Journal in Biology and Medical Sciences (4). Authors can submit their articles online. The article is rapidly published and then the review process starts. Whether the open peer review system is here to displace the conventional systems of peer review remains to be seen. Meanwhile, strengthening the existing partially blind review system with an open review supplementary option seems like a prudent choice.

Postreview rejection can be frustrating for both authors and editors, particularly when they are victims of limitations of the blinded forms of review as discussed above. Akin to the judiciary system where protection of innocents from execution is of prime importance, safeguarding author interest and publication of quality science is a responsibility of the editor that is of extraordinary significance. To provide recourse for authors who felt that their work has been rejected not because of the quality of science but because of the limitations of the peer review system, ARS introduces a rebound track for peer review (www.liebertpub.com/ars). The rebound peer review track is a two-tier process that represents a hybrid of partially blinded and open peer review systems. This track enables authors, whose work has been rejected by ARS, to seek top experts who the authors think are best suited to understand the science in question to serve as open peer reviewers. The reviewers must meet the required criteria to be defined as acceptable expert and have no conflict of interest. If four such reviewers are willing to overturn the reject decision based on their review of the article and review comments, ARS will be open to considering the article for publication based on outcomes of exchange between the editor and the open reviewers. If accepted for publication, the comments of the open reviewers will be published as supplementary text together with the names of the reviewers cited in the article as sponsoring peers. Here, the assumption is that the open reviewers will exercise a high level of rigor in scrutinizing the case. The goal is to make sure that every author has the opportunity to rescue their work which they felt was victimized by the glitches of the current peer review system. I invite affected authors to make full use of this experimental mechanism so we know whether the rebound peer review should prevail as a viable recourse.