Demystifying Peer Review

The Peer Review Process at HE&B

The purposes of peer review of manuscripts submitted to HE&B are to select and prioritize articles with promise and then to improve their quality through the peer review process. That process begins when an author submits a manuscript to HE&B (as shown in Figure 1). Manuscripts are submitted at the ScholarOne website, which is managed by Sage Publications. Each manuscript, whether unsolicited or solicited, is first reviewed by the editorial manager for completeness and to ensure the manuscript meets prescribed style elements and other requirements of submission before going to the editor-in-chief for substantive prereview. Last year, HE&B received more than 400 unsolicited manuscript submissions. Thus, each month, the editor-in-chief routinely reads and evaluates, on average, as many as 40 papers, with between 10% and 20% ultimately resulting in publication.

OPEN IN VIEWER

Figure 1.

Flow diagram of the peer review process at HE&B

The editor-in-chief reviews each paper and advances to full review only those manuscripts that, in his judgment, promise to make a significant contribution to the scholarship of theory, practice, or policy in behavioral science, health education, and public health. Unfortunately, HE&B receives many more manuscripts than can be published and thus the editors must make difficult decisions about the scope of valuable work that the journal can place into full peer review. Manuscripts that are declined may be judged to fall outside the main scope of focus of the journal, suffer from serious methodological limitations (e.g., the lack of sufficient sample size for power), or display serious deficiencies in logic or clarity of presentation. Authors whose manuscripts deemed by the editor-in-chief to fall outside the scope of HE&B, but which are of high quality and potentially more compatible with the scope and mission of the Society for Public Health Education’s sister publication, Health Promotion Practice, are often returned to the author with encouragement to submit to that journal. In the end, the editors must decide whether a manuscript rises to a level of quality that merits priority for publication and promises to be of as much interest to our readership as other manuscripts that are competing for precious journal space.

Once the decision is made to review, the manuscript is assigned to an associate editor, who also considers the appropriateness of each paper in the context of the journal’s mission and decides to review or, occasionally, reject without review. Working in tandem with the editor-in-chief, each of the four associate editors annually manages the review of more than 50 manuscripts that go on to full review.

The next step of the HE&B review process is the selection of referees. Initially, we invite three peer reviewers—who are blinded to the identity of the authors—to read each manuscript. Editors can identify potential reviewers in several ways: from the HE&B reviewer database using keywords that authors enter on submission to the ScholarOne manuscript management system, by viewing the work cited among the references, and by searching for the names of scholars whose work has yielded relevant publications on similar topics or populations. Authors also can identify preferred and nonpreferred reviewers via ScholarOne. The editor in charge of the review considers these requests when deciding whether to invite additional, ad hoc reviewers. All invited referees are asked to complete their reviews within 1 month. ScholarOne sends automated email reminders to referees when a review is overdue.

Once the associate editor in charge of a manuscript’s review receives the completed reviews—at least two—the editor must make a determination as to whether the manuscript should be accepted (usually with some revisions), revised for further consideration, or rejected. Importantly, each editor also scores the timeliness and usefulness of the review received from a referee for each submission. The ScholarOne system archives these scores, which enables the editors to aggregate and rate reviewer quality. The editors strive to return reviews along with a decision to the corresponding author within a week of receiving the final review. The referees also receive each other’s reviews and the editor’s comments. Challenges arise when the editor faces the problem of reconciling mixed reviews in which there appears to be no clear consensus among the referees. As a consequence, most papers are revised at least once in response to reviewer feedback and often two or more times prior to a final decision about publication. Once accepted, manuscripts are sent to the production editor for formatting and scheduled for publication. In general, it may take up to 6 or more months before a manuscript completes the process and goes to publication. Knowing the importance of timely publication, we are working to ensure that it takes no more than 12 months before a manuscript is published. Moreover, with the advent of First Online Publishing, articles are available in electronic form for citation in PubMed Central and other databases ahead of print publication.

Revised papers generally, but not always, go back to the same editor, who either sends it out again for review or makes a final determination. Sometimes the editor who is managing the manuscript will consult with one or more other associate editors, or with the editor-in-chief, or both, before making a final decision. Thus, the reviews and the merits of the manuscript are debated and discussed before a final judgment is made regarding the disposition of the manuscript.

What Makes a Good Review?

Good reviews come in all shapes and sizes, but the best reviews are easily recognizable. The best and most professional and reliable reviewers decide within a day or so if they will review the paper and complete their reviews within the prescribed time period. A good review generally requires an investment of approximately 2 to 3 hours, and sometimes longer when a manuscript’s methods are complex or the writing is not clear. Most HE&B referees review only a few manuscripts a year, but those with unique areas of expertise review as many as six or more. The least professional reviewers do not respond to the email solicitation at all, or even worse, agree to conduct the review and never actually complete it.

Editorial decisions are largely dependent on the reviewers’ recommendations. Reviewers’ comments and criticisms serve both to justify the recommendations and improve the manuscripts. The best reviews are timely; describe the extent to which the paper makes a contribution to research, practice, and/or policy; and identify the manuscript’s strengths, weaknesses, and limitations. Editors are guided by the reviewers’ opinions of the strengths of the paper, what the paper contributes to the research and theory base, and why it should or should not be accepted for publication. Of course, the weaknesses should also be clearly described. There are aspects of any study that cannot be changed, for example, the design, measures, and sample, although frequently additional details are needed and can be added. However, the writing, orientation, and content of the background, methods, findings, and discussion sections are potentially changeable and these elements of the manuscript commonly benefit from detailed reviews that serve to guide revision.

In general, reviewers should treat each manuscript as if it had been written by a valued colleague, and the comments should be designed to improve the paper so that it has the best possible chance for acceptance. Quite simply, the golden rule of peer review is to review another author’s work the same way you, as an author, would want your work to be reviewed. Having a paper rejected can be disheartening for an author and reviewers should not be overtly critical or mean-spirited. Most authors appreciate constructive criticism that enables them to improve their work, whether or not the paper is ultimately accepted for publication. The more thoughtful and specific the reviewers’ comments, the more helpful they are to the authors. Therefore, reviewers should strive to identify the ways the manuscript could be improved in as detailed and civil a manner as possible.

The Elements of a Good Review

Although reviewing is an important professional activity, regrettably it is an underdeveloped part of most professional preparation. Many reviewers and potential referees receive no formal preparation or experience in reviewing, nor do they receive advice about how to conduct a professional review. Below, we provide some guidance about what makes a review most helpful (Cummings & Rivera, 2002). Although not all HE&B manuscripts constitute research papers, many are, and for convenience and illustrative purposes, we orient these remarks toward research papers, recognizing that not all manuscripts are based on a particular study design and research methods.

Comments to the Editors

In addition to a reviewer’s recommendation that the paper be accepted, revised with minor revisions, revised with major revisions, or rejected (which should only be shared with the editors and should not be part of the critique that goes to the authors), it is useful to provide the editors with confidential comments that justify and explain the recommendation. Usually, such comments focus on the main reason or reasons for the reviewer’s recommendation, note the particular importance of the research, or note a fatal flaw in the methods that cannot be addressed through revisions. In some cases, a referee will have reviewed the manuscript and the work it represents in a previous incarnation for another journal or otherwise know things about it and the work it represents about which the editor should know. Sometimes a referee will have many criticisms of a paper, but will nevertheless recommend the paper be revised and resubmitted because, although flawed, the manuscript may still have potential. Sometimes a reviewer will have few criticisms, but view these as fatal and recommend rejection. The reviewer is likely to be more familiar than the editor with the state of the research on the topic and thus is in the best position to offer the editor informed advice about how to consider the paper within context.

When a potential referee is invited to review a manuscript, he or she should consider the following points:

Why Serve as a Reviewer for HE&B?

Reviewing manuscripts demands a great deal of work, a high level of expertise in the topic of the work, and focused attention to details of writing and methodological strengths and weaknesses. Nonetheless, it is highly rewarding when a referee realizes that he or she has contributed to the dissemination of scholarly or scientific literature through his or her review efforts.

There are several good reasons for serving as a reviewer. First, reviewing is an important professional service that results in improved quality of published articles and thus advances both scholarship and professional practice. Second, participating as a peer reviewer is considered a highly valued and prestigious activity, uniformly recognized as an important contribution by other scholars, promotion and tenure committees, grant review committees, and accrediting bodies concerned with quality assurance. Finally, and perhaps most important, reviewing can serve to improve the reviewer’s own work, by helping him or her to become a more precise writer, to improve the design of his or her own studies, to develop new ideas, and to expand current working knowledge of the literature.

Coda

Health Education & Behavior has a proud tradition of publishing high-quality articles, which is due in large part to the commitment of the referees who have reviewed for the journal and a system of peer review that is the raison d’être of science and scholarship. One mark of such quality currently resides in a journal’s “impact factor” (a measure of the frequency with which the average article published in a journal is cited in a year), which is calculated annually by Thomson Reuters (2011). Maintaining high standards and a high impact factor, which has largely trended upward for HE&B, depends greatly on the quality of peer review. But, like most journals, HE&B faces major challenges in identifying enough professionally qualified and available reviewers who are able to respond to our requests. In addition, because of the proliferation of new journals, there are now many scholarly publication outlets and the competition for good manuscripts is intense. Finally, the publishing enterprise is changing rapidly, requiring that the journal and our peer review process adapt accordingly. Despite these challenges and the shifting landscape of academic publishing, HE&B strives to provide an exceptional and expeditious review experience for both our authors and referees because the quality and integrity of peer review remains the key to our success.