The Next Chapter for Psychological Science

Increasing Equity and Inclusion

A major barrier to greater inclusion in Psychological Science is the “hidden curriculum” of journal peer review and publishing. Outside of formal policies, there are many unwritten rules about what authors, reviewers, and editors typically do or are permitted to do. Not everyone has equal access to this information or equal opportunity to take advantage of the informal avenues. Journals can help reduce this inequity by making as many of these practices as possible explicit, avoiding informal channels of communication that are not open to all, and putting safeguards in place to reduce the influence of biases. To this end, we are introducing public, drop-in online “office hours” that will be available across a wide range of time zones. I will be available at these office hours to answer any questions authors might have about the process, policies, and practices at Psychological Science. We will also strive to articulate our policies and practices in as much detail as possible on the journal website in order to make it as easy as possible for authors to anticipate how their submissions will be evaluated.

Another important barrier to equity is the biases that come into play when evaluating manuscripts that do not fit the traditional mold. We want to encourage authors to submit manuscripts that broaden the diversity of our published literature, whether that is with respect to the population studied, the authors’ identities, the topics studied, the methods used, and so on. These are often the very contributions that challenge norms that hold back scientific progress. However, if we encourage such submissions, we must take steps to ensure that they are evaluated in a just and equitable way. I am delighted to have recruited a team of editors who are committed to doing so. Our team, as a group, is quite diverse and has broad representation on a number of important dimensions, and we share the value of recognizing the diversity of ways that research can make an important contribution to the literature. Nevertheless, our group also reflects many of the same disparities in representation that exist in our field at large and especially in leadership positions (e.g., underrepresentation of researchers of color; Roberts et al., 2020).

To help build a pipeline for more diverse editorial teams at Psychological Science, we are taking two steps. First, we will strive to diversify our editorial board. In the coming months, we will be soliciting nominations for editorial board members, with an eye toward increasing representation of groups that have so far been underrepresented in journals. Second, Psychological Science is participating in APS’s new Editorial Fellows program, where two fellows will be appointed in part on the basis of their membership in groups that have been historically underrepresented among psychology journal editors. These fellows will be paired with mentors, be given a small stipend, and handle a few manuscripts. These steps will help address inequities in who is reviewing and handling manuscripts, inequities that impact authors as well as readers. Finally, we will continue the current practice of masking authors’ identities from editors until a decision has been made to desk reject the manuscript or send it out for review, further reducing the chances of bias affecting editorial decisions.

Perhaps the biggest challenge in redressing inequities in scientific publishing is how to provide assistance to authors who did not have access to resources that are most relevant to our journal’s specific requirements (e.g., open-science training, English language support). Researchers in this position often bring unique, complementary skills and perspectives that would greatly benefit the journal, but our requirements pose unique obstacles. To address this, we are compiling a list of easy-to-access resources to help authors meet our requirements. We also hope to partner with other organizations that provide similar support, prior to journal submission (e.g., Reviewer Zero; Aly et al., 2023; prereview.org).

Finally, the primary goal of peer review is to evaluate the scientific merit of submitted manuscripts, but journals also have a responsibility to ensure that the articles they publish do not cause unnecessary harm or unnecessarily exacerbate societal inequities. Striking a balance between academic freedom (and the corresponding value in science that no topic or scientific claim should be treated as inherently off-limits) and the duty not to inflict harm is a struggle for all editors. However, an easy step to take is to offer our editors training and resources for handling content (manuscripts, reviews, and editors’ own decision letters) in a way that is conscious of and minimizes potential harm. We will provide training and just-in-time support to handling editors so that claims and language that could be harmful are reviewed carefully. This includes ensuring that our team of handling editors and STAR editors has the representation and breadth of expertise necessary to assist each other in such cases.

Deterring Hype

Authors submitting to a very selective journal face a dilemma. To have a chance to have their manuscript accepted, they need to make a strong case for its importance, emphasize the strengths of the work, and articulate the implications of the findings. However, selective journals should also, in principle, be selective on rigor, and these practices, taken too far, can undermine the rigor of the work (Corneille et al., 2023; Hoekstra & Vazire, 2021).

Striking the right balance requires a great deal of trust in the editors of the journal. The author who admits the limitations of their work and carefully incorporates those limitations into their conclusions (e.g., including qualifiers in their abstract or title) is taking the risk that editors will be more attracted to the submissions with bold claims and fewer caveats. We cannot ask authors to embrace intellectual humility and calibration unless the editors are prepared to follow through—to prefer manuscripts with well-calibrated claims to those that overclaim.

Our team of editors at Psychological Science is committed to walking the walk. We appreciate that all research designs require trade-offs. Although we will have high standards when it comes to the methodological rigor of the study and analyses, we will not expect perfection. Indeed, unwarranted bold claims will harm chances of acceptance, and exaggeration will be considered a potential basis for desk rejection. We are looking for excellent research, but we expect even the best research to have flaws, and we want those flaws to be factored into the whole manuscript, including the conclusions drawn. We hope authors will trust us to see the value in work that is presented accurately and not to be fooled by claims that do not match the evidence.

Importantly, science benefits when authors share their opinions, speculations, and extrapolations about what their results might mean, and readers want to know these things. Similarly, authors should not shy away from explicitly stating their assumptions and their justification for those assumptions. However, speculations, assumptions, and other claims that go beyond the evidence should be clearly flagged as such wherever they appear (including in abstracts).

Finally, we are also continuing to find ways to keep hype in check in public-facing reports about Psychological Science articles. We are working with the APS Scientific and Public Affairs office to have editorial input into press releases and other media reports of our articles. In addition, authors will no longer be asked to submit a statement of relevance—if we reintroduce something like a public abstract, those will be written by members of the editorial team in collaboration with authors.

Making Ourselves Accountable

The open-science movement has made great strides in increasing the transparent reporting of research, making scientific claims easier to evaluate and making research more accountable. However, less progress has been made in making journal operations more transparent and accountable. Of course, not everything can or should be made transparent, but enough should be shared to make it possible for interested parties to evaluate the fairness and rigor of our processes and for authors and reviewers to know what to expect. We are making several changes to increase the transparency and accountability of our journal operations.

First, we will strive to make as much information public as possible about our evaluation criteria. The submission guidelines will be updated to reflect our editorial team’s new policies and priorities. We will also share as much information as we can on the journal’s website about the journal’s informal practices, statistics about the journal, and common reasons for rejection.

We will also introduce Registered Reports as an article type, providing authors the option to submit their work for peer review, and potentially be granted “in-principle acceptance,” before the data are collected. We will also offer “Registered Report with Existing Data” for cases where the data already exist, with some restrictions based on the authors’ prior opportunity to access the data. Registered Reports help reduce the opportunity for bias stemming from authors’ decisions during data collection and analysis and for bias stemming from editors’ decisions during peer review.

To further reduce the chances of bias playing a role in decisions, we will continue to enforce conflict-of-interest rules—no editor or reviewer will be involved in the peer review of a manuscript where they have a conflict of interest with an author (see our Contributor FAQ for more details about what counts as a conflict of interest), and authors are responsible for not suggesting reviewers who have a conflict with any of the authors. Handling editors (senior and associate editors) will be discouraged from submitting manuscripts as authors, their submissions will be capped at two per year, they will be required to disclose their conflict of interest in the published article, and their submissions will continue to be handled by an editor outside of the journal.

We can also make ourselves more accountable by giving readers access to more information about the peer review that went into vetting our published articles. Of course, we want to preserve the important aspects of anonymity and confidentiality that allow authors and reviewers to trust the peer-review process. We will never disclose reviewers’ identities unless reviewers choose to sign their reviews, and we will keep the content of peer reviews, and authors’ responses, confidential until the peer-review process is complete. We will, however, transition to “transparent peer review” (already adopted by Advances in Methods and Practices in Psychological Science). This means that if an article is accepted for publication, we will publish the content of peer-review histories (the reviewers’ reports [without reviewers’ names, unless they choose to sign their reviews], editors’ decision letters, and authors’ responses). We will not publish peer-review histories for rejected manuscripts, and instead we will give authors the freedom to share their manuscripts’ peer-review histories however they wish. These changes will help surface any problematic patterns or incidents in our peer-review processes and will help authors make the most use out of their reviews.

Our increased investment in statistical, transparency, and rigor checks prior to publication (through the STAR editors; see the accompanying editorial, Hardwicke & Vazire, 2023) will help reduce the number of errors that make it into print, but there will nevertheless still be errors that we don’t catch. We are taking steps to incentivize high-quality postpublication critiques of Psychological Science articles. We are folding the Letter to the Editor article type back into the Commentary article type, which means that commentaries will no longer be required to present new data or analyses. Further, we will strive to publish all commentaries that meet our threshold for quality and value (though other restrictions, such as the prohibition on authors submitting commentaries about their own articles, will remain in place). We will develop and communicate more explicit criteria for our evaluations of commentaries as we learn from experience. Our partnership with the Institute for Replication (detailed in the accompanying editorial) will also help detect and correct errors in our published articles.

Finally, we are encouraging replication submissions by allowing them to be submitted as a Research Article (the main article type) or a Registered Report. That is, we are discontinuing the Preregistered Direct Replication article type and allowing replication work to be submitted like any other empirical work. As with any other submission, these will be evaluated on the basis of whether they are of broad interest to the field, their importance, and their methodological rigor. However, to preserve the self-correcting function of the Preregistered Direct Replication initiative (i.e., to follow the spirit of the “Pottery Barn rule”; Srivastava, 2012), submissions of close replications of articles previously published in Psychological Science, when submitted as a Registered Report, will be evaluated only on methodological rigor (i.e., their general-interest value and importance will be granted).

Increasing Efficiency

Finally, an important priority at Psychological Science is that we respect authors’ and reviewers’ time. To this end, we will continue the practice of triaging submissions and desk rejecting those that two or more editors judge as being unlikely to be accepted for publication, providing fast responses to authors and keeping the burden on reviewers manageable. In addition, editors will aim to make accept-or-reject decisions as early in the peer-review process as possible. This means erring on the side of not inviting major revisions and not having multiple rounds of review and revisions (though each of these will sometimes be appropriate).

There is an often-overlooked inefficiency in common peer-review practices: Authors may often submit work that is not quite publication ready because they (accurately) expect that, even in the best case, they will be asked to make substantial revisions before the manuscript is accepted. This dynamic is in nobody’s interest—not authors’, not reviewers’, and not the journal’s. However, it is rational for authors to continue to do this unless they can reasonably expect that their manuscripts will stand a chance of being accepted as is or with only very minor revisions. Thus, we will try to break this cycle by being open to accepting submissions after the first round of review whenever appropriate—there is no reason why this shouldn’t happen some of the time. In exchange, we expect authors to submit publication-ready work, and we will be prepared to reject submissions that are far from publishable as is even if the issues identified in peer review are fixable (if they are substantial and/or could easily have been fixed before submission).

Some of the changes outlined in this editorial and in the accompanying editorial (Hardwicke & Vazire, 2023) present new burdens for authors, and some will slow down the evaluation process (e.g., transparency checks before acceptance). We will do our best to keep things moving efficiently on our end (which will be enabled by our large and very capable team of STAR editors), and we will provide detailed information to authors about what they can do to make the process as smooth as possible. In addition, we will look for ways to eliminate or reduce other points of friction. As a start, we are eliminating cover letters (authors can still write to the editor in an optional open-ended text box during the submission process) and statements of relevance.

To reduce the burden on reviewers, we are bringing some of the checks that reviewers may feel obligated to do (e.g., checking that links to data, code, and so on work) in-house, with our STAR editor team. We will still welcome reviewers who choose to scrutinize these aspects of manuscripts, of course, and indeed, we plan to recruit some editorial board members who specialize in STAR-related evaluation (e.g., assessing flexibility in data analysis) and call on them on a case-by-case basis.

The editors of Psychological Science are very grateful to authors who choose to submit their work to us and to reviewers who donate their time to helping us evaluate and select the best work for publication. We welcome any suggestions for ways to improve the experience for authors and reviewers, to improve the quality and diversity of articles we publish, to make our processes more equitable and just, and to correct and learn from our mistakes.