Abstract
Quality peer reviewers of manuscripts are the bedrock of any good journal. They play an essential role in assuring the high caliber and accuracy of the science that a journal publishes. A good peer review is used by the journal to help the editorial team make informed decisions and to improve the quality of published manuscripts.
However, good peer reviewers are not always easy to find. Surprisingly, one study has shown that scientific training and increasing practical experience reviewing papers are not reflected in the quality of subsequent reviews performed by a reviewer (Bruce et al. 2016). Other studies have shown that some peer reviewers are not able to appropriately detect errors (Ghimire et al. 2012; Boutron et al. 2010), improve the completeness of reporting (Hopewell et al. 2014), or decrease the distortion of the study results (i.e. “spin”; Turner et al. 2008; Melander et al. 2003; Lazarus et al. 2015). So, what’s a journal to do?
In 2014, Toxicologic Pathology expanded its editorial board (EB) from 25 to 52. These members are drawn from all the societies who have selected Toxicologic Pathology as their professional journal. The goal was to have a sufficient number of experienced reviewers available to review manuscripts based on the number of papers received in 1 year. Another goal was to have EB expertise in as many specialty areas as possible. The associate editors (AEs), currently 17 in number, are encouraged to determine which specialty reviews a manuscript might need (e.g., statistics or subject matter expert for a given organ, species, or technology) and invite appropriate authorities in the field. If an AE cannot find a peer reviewer in a specific specialty area, then someone outside of the EB may be invited; in general, society members are considered before external scientists. An effort was also made to clarify the goal of the Toxicologic Pathology peer review process, provide a definition of a good-quality peer review report, and list the outcomes that should be used in making publication decisions. We address these points by providing clear instructions to reviewers (Online Supplementary Table I) and in 2015 added a point-by-point checklist (Table 1) for the peer reviewer to use. Once the initial reviews have been completed and evaluated by the AE, the anonymous remarks and AE’s comments (if any) are sent either to the authors with a recommendation for a minor or major revision or to the editor in chief (EIC) with a recommendation and rationale for rejection. The reviewers receive an anonymous carbon copy of the decision e-mail that is sent to the authors and therefore have the chance to see other appraisals of the same manuscript. An astute peer reviewer will seize this opportunity to discover where they might have done a better job in performing their review. Such learning facilitates better reviewing in the future, especially when it is time to look at the revised version of the same manuscript.
Reviewer Checklista.
aThese instructions are available online at https://mc.manuscriptcentral.com/toxpath under “Instructions & Forms.”
If a minor revision was recommended by the peer reviewers, then the AE will generally assess the revised manuscript to ensure that all recommended edits were appropriately addressed. If issues were addressed effectively, the AE may send a recommendation to the EIC to accept the manuscript without having the revision undergo a second round of formal peer review. The journal has adopted this option as a means of speeding up publication decisions while ensuring that the quality of accepted papers remains high. If a major revision was recommended, however, the reviewers will have an opportunity to review the revised manuscript to ensure that their proposed edits were made or read the authors response, indicating why one or more proposed edits were not done. Outcomes for manuscripts that required a major revision may range from another revision (often minor) to an AE recommendation to the EIC for acceptance or rejection.
The journal has incorporated a 2-tier screening process for all initial submissions to avoid peer-reviewing unsuitable manuscripts. The first screen is done by the managing editor, who manages the submission and peer review process for the journal. She will determine if (1) all figures and tables are submitted at the appropriate size and resolution according to the author guidelines, (2) a cover letter has been submitted, (3) the appropriate format was used, according to author guidelines, (4) the authorship form is completed, and (5) the author has disclosed any conflicts of interest. Flagging of any of these items will warrant sending the manuscript back to the author prior to undergoing peer review, so that the manuscript format and/or figures/tables can be prepared according to journal standards, that the appropriate forms have been submitted, and that the author has disclosed any real or potential conflicts of interest or lack thereof.
The second screen is done by the EIC and is designed to ensure that the subject matter is appropriate for the journal and the science is generally appropriate. The journal mission statement was also revised to ensure clarity of acceptable journal content and to attempt to decrease the rejections based on improper subject matter. Problems flagged during the second screen will warrant sending to a journal committee for review, which is composed of 2 to 3 AEs who serve on the committee for 2 months each year.
In some cases, a final screen is done by an assigned associate editor and is for general readability in terms of the correct use of English grammar and sentence structure. If needed, a professional medical English editing service is recommended before submission.
Not sending these problematic submissions out initially for review can help to prevent “reviewer fatigue.” In general, once a reviewer receives an invitation from Toxicologic Pathology to review a manuscript, they can feel confident that the topic is suitable, the science is probably appropriate after a superficial screening, and that the figures and tables are of the acceptable size and resolution. However, a detailed review by the assigned reviewers is still necessary, and the figures and tables must still be closely evaluated by the reviewers to determine if the magnification is appropriate to see the described lesion and the annotation is sufficient to illustrate key points.
So, has the quality of peer reviews improved for our journal? Overall, I’d say, “Yes.” A good review may take the better part of a workday to complete, and a reasonable workload for a peer reviewer (especially an EB member) is 3 to 4 manuscripts a year. It may be hard for some to commit this amount of time—even on an occasional basis. But the time spent on the review is reflected in the quality of the review and the overall scientific quality of our profession’s flagship journal. Most of our peer reviews are very comprehensive. At first glance, an author may feel somewhat offended and feel that the reviewer was being “picky.” No one wants to be told that their baby isn’t beautiful! However, once the edits are done, most authors are quite grateful that someone took the time to read their paper so comprehensively and provided such a thorough and useful critique. Most authors would agree that the review process has improved the quality of their manuscript.
Our anonymous author surveys began in July 2016 and are still ongoing. They were designed as a checklist that can take as little as 5 minutes to complete or slightly longer if comments are provided. The peer review process is one of the items that we assess in this survey. One question is, “Were you satisfied with the review process in terms of timeliness and the quality of the reviews you received?” From a total of 46 responses to date, 41 (89%) replied yes and 5 replied no. Another question is, “Did the reviewers’ comments and suggestions help to strengthen your manuscript?” Again, of the 46 responses, 41 replied yes and 5 replied no. We have a comment section in the survey and look forward to hearing about specific problems that our authors encounter; the names of the authors and specific manuscript titles are not revealed to ensure the anonymity of the authors and reviewers. In general, we receive far more compliments than complaints, which indicate that our review process is working well. However, this does not mean that we don’t always strive to improve the process, so we encourage continued feedback from our authors.
The peer review process seems to be doing quite well, but the AEs are challenged at times in fulfilling their roles. The AEs are required to collect at least 2 scientific reviews for each manuscript. Sometimes the AEs will receive a review that suggests rejection (and will state the reasons why) while another reviewer will recommend accept with major, minor, or no revisions. In situations such as this, the AE can either consult the EIC and/or can make a determination after carefully reviewing the manuscript and the reviewers’ comments. The difficulty comes when one reviewer provides a substandard review. In such a situation, a replacement reviewer must be found; this requirement will cause a delay in the process, and the author must be notified. Since the reviewer instructions and checklist have been incorporated, these situations have become the exception.
So, how are good peer reviewers identified? It starts at the end of each year when it is time to cycle off a subset of reviewers. The typical time for a reviewer to serve on the EB is 2 years. It is the EIC’s responsibility to identify high-quality reviewers who can replace those that are cycling off. One goal is to keep as many subspecialties available for the AEs to choose from, and this can be determined by identifying colleagues who are known experts in the field or by consulting with the societies to find individuals with appropriate expertise. For example, the Society of Toxicologic Pathology member website possesses a “demographics” search tool for species and/or technical expertise or systemic pathology areas of expertise as well as a “communities” search tool for memberships in special interest groups or working groups. Either search tool can help identify individuals with suitable capabilities for addressing a particular peer review need. Inviting up-and-coming pathologists is another consideration. Perhaps surprisingly, this relatively inexperienced group nonetheless produces some of the very best peer reviews. Regardless of the years of training and experience or level of expertise, a literature search for first-author publications is always done. If you know how to be a good author, then you will most likely understand how to be a good reviewer. But it is not just knowledge and skill of the review process that determines who will be a good reviewer. One must also be able to commit a fair amount of time to reviewing papers. Before inviting a reviewer, the AEs are asked to look up the number of times that person has done a peer review. If someone has completed a review within the past 3 months or several within the past year, then the AE is advised not to choose that person. One thing we don’t want is reviewer fatigue. Once a review is completed, the AE is asked to rate both the timeliness and quality of review, so that other AEs can see this score and decide if that person should be invited to review again.
One request I have for those in academia or training institutions is to provide a formal comprehensive training program that will enable pathology residents and graduate students to develop the fundamental skills of manuscript peer review. This task might be done by having the trainees review the same manuscripts that the advisors are asked to review (with the permission of the EIC), critiquing the trainee’s reviews and comparing with the advisor’s own review, and/or by having trainees provide reviews of their peers submitted manuscripts. Journal clubs with faculty participation are another great way to discuss and review current literature and develop necessary skills for evaluating publications. There are also many blogs and publications with guidance on how to be a good peer reviewer (Allen 2013; Hill 2016). Knowing how to be a good reviewer will teach someone how to be a good author and vice versa.
An invitation to review is not a demand. One always can choose not to accept the invitation. Keep in mind, however, that the willingness of you and your colleagues to serve as reviewers is critical to the continued vibrancy and relevance of Toxicologic Pathology—and to the timely publication of your own submissions to our profession’s premier scientific journal.
In summary, the peer review process at Toxicologic Pathology has been honed to permit the consistent production of rapid and thoughtful feedback to authors submitting their work to our journal. The careful choice of EB members is based on subspecialty and publishing history and a commitment to review at least 4 manuscripts per year. We try to assist reviewers by providing explicit instructions and a checklist for the review process and providing an anonymous copy of the other reviews for the same paper to facilitate sharpening of one’s reviewing skills. We believe that this refinement has both allowed our EB members to develop competent review skills and has permitted our journal to provide more timely feedback to authors. The result of this evolving peer review process is reflected in the continued publication of high-quality manuscripts—reflected in the high impact factor for Toxicologic Pathology relative to other journals with similar scientific content—and high author satisfaction.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplementary material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
