Book Review: Reporting Research in Psychology: How to Meet Journal Article Reporting Standards

Reporting Research in Psychology is a valuable addition to the growing library of reporting guidelines within the behavioral, social, educational, and medical sciences. In this reference book, Cooper thoroughly discusses the items found in the Journal Article Reporting Standards (JARS) of the American Psychological Association (APA Publications and Communications Board Working Group on JARS, 2008). Examining the content of JARS in depth, Cooper provides rich explanations and examples to help the reader adhere to each reporting standard found in this guideline. Adherence to these standards ultimately promotes transparent, accurate reports of studies, which facilitates the proper critical appraisal, replication, and application of studies to policy and practice.

Cooper explains that the impetus for these reporting standards was the push for evidence-based decision making in areas of health and public policy. Events such as the establishment of the Cochrane and Campbell Collaborations, and policies such as the 2001 No Child Left Behind Act in the United States, marked the pervasiveness of this new paradigm. However, over the past few decades, numerous reviews have revealed consistently poor reporting of research that is meant to inform evidence-based decision making in health, social, and public policy (Simera, Altman, Moher, Schulz, & Hoey, 2008b). In response, collaborations of researchers and journal editors have made a concerted effort to develop reporting standards that assist authors in writing high-quality journal articles. These reporting standards are not requirements that determine the format for writing or designing a study, but rather are recommendations that highlight essential details that journal articles should consistently and explicitly include. Reporting standards are generally collated and disseminated through reporting guidelines. A reporting guideline is “a checklist, flow diagram, or explicit text to guide authors in reporting a specific type of research, developed using explicit methodology” (Moher, Schulz, Simera, & Altman, 2010, p. 1). Guidelines usually recommend that authors report a minimum set of study details that are related to important research biases, are often poorly reported, and/or are considered by expert opinion to be essential for all reports about a specific type of research.

In 2006, the APA Publications and Communications Board commissioned a group of five former APA journal editors to investigate existing reporting standards in disciplines related to psychology and to consider how reporting standards might be incorporated into the sixth edition of the Publication Manual of the American Psychological Association (APA, 2010). Rather than develop a completely new set of standards, the JARS Group sought to consolidate existing standards and apply them to psychology. They also considered adding standards that are particularly important in psychology but have not been included in previous guidelines.

Development of JARS began in 2007 with the systematic identification, collection, and expansion of previous reporting standards. Content for JARS was derived mainly from a previous version of the Consolidated Standards of Reporting Trials (CONSORT) Statement in medicine (Moher, Schulz, & Altman, 2001), the Transparent Reporting of Evaluations with Non-Randomized Designs (TREND) Statement in public health (Des Jarlais, Lyles, Crepaz, & the TREND Group, 2004), and the American Educational Research Association (AERA, 2006) “Standards for Reporting Empirical Social Science Research in AERA Publications.” The JARS Group compared the content of these guidelines, developed a nonredundant list of their reporting standards, and then modified and added to this list so that it may better apply to psychological research (p. 109). The draft list of standards was then shared and revised with a group of APA editors, Publication Manual Revision Task Force members, anonymous reviewers from American Psychologist, and members of the APA Publications and Communications Board. The JARS Group then finalized the list into its current version, and also developed meta-analysis reporting standards (MARS) in a similar fashion. Reporting Research in Psychology is a complementary publication that offers a detailed discussion of the items found in JARS and MARS.

Compared to other guidelines, this book provides practical guidance for reporting empirical research studies specifically in psychology and related disciplines, such as social work. The main JARS table provides an “all-purpose” list of reporting standards to address in any article that reports new data. In addition, due to the breadth of research designs used by social and behavioral scientists, the JARS recommendations involve several modules (pp. 9–10), each of which contain a set of standards for a specific research design or topic (e.g., experiments with randomized designs). Researchers using the JARS recommendations should add any relevant module’s items to the “all-purpose” list of standards and then use this combined list when writing an article about that study (pp. 7–8). In accord with other guidelines, Cooper notes that these standards are meant to delineate essential information to report in an empirical research article, not to dictate the style of a paper.

The first chapter of this book clarifies the need for reporting standards in psychological research and explains how to use JARS. Most of the remaining chapters explain and provide examples for each of the items found in JARS. The organization of these chapters follows the structure of a journal article for ease of use. Chapter 2 focuses on the title, abstract, and introduction; Chapters 3 and 4 on the methods and results; and Chapter 5 on the discussion section. In Chapter 6, Cooper introduces MARS. Unlike his discussion of JARS, Cooper does not explain all of the MARS items due to space constraints, but instead discusses the most important standards as identified by a survey of members of the Society for Research Synthesis Methodology (Cooper & Dent, 2011). As Cooper indicates (p. 91), a full discussion of all of the MARS items in another book or online supplement would be useful.

In the final chapter, Cooper reviews the explicit methods used to develop JARS and MARS and discusses plans for revising these guidelines in the future. Recognizing the importance of updating guidelines, Cooper conveys the expectation that JARS can be amended when consensus about other standards emerges. Namely, the JARS Group has and will continue to encourage other authoritative groups of researchers, practitioners, and editorial teams to use JARS as a starting point for modifying standards for their specific subdisciplines and topic areas. Thus, a major strength of JARS and this book is that, like other influential reporting guidelines, these already robust standards accommodate future revision in light of new evidence. Such revisions could then be incorporated into future versions of JARS and this book (p. 110), keeping its standards current and based on the best available evidence. For example, several reporting guidelines have been published after JARS was first written, many of which are updates of the standards on which JARS is based. These include an update (Schulz, Altman, & Moher, 2010) and several extensions (e.g., Boutron, Moher, Altman, Schulz, & Ravaud, 2008) of the CONSORT Statement, as well as standards for reporting behavioral change interventions (Abraham, 2009). These guidelines could provide additional or modified standards for future revisions of JARS, particularly those standards related to intervention research.

This book has several features that make it a significant contribution to research in psychology and related disciplines such as social work. First, Cooper methodically explains and elaborates on items in JARS using accessible language. He engages with the rationale for each reporting standard, revealing why they are important and how to adhere to them. Cooper also provides numerous examples from articles published in APA journals to illustrate how to meet (and not meet) these reporting standards. Furthermore, JARS rests on a strong evidence base. Reporting standards should reflect the consensus opinion of a wide group of experts in a particular field, while also drawing upon relevant empirical evidence (Moher, Schulz, et al., 2010). The JARS Group based its standards on a thorough review of the reporting guidelines literature, and many of the standards adapted from previous guidelines are based on empirical evidence of poor reporting or risk of bias (Schulz, Altman, & Moher, 2010).

Given its adaptability, future versions of this book and JARS could benefit from newly identified best practices in guideline development and dissemination (Moher, Schulz, et al., 2010). For instance, the JARS Group could modify the content of future revisions by employing formal consensus development methods, such as the Delphi survey (Murphy et al., 1998). These methods allow multiple stakeholders to be involved, while also providing robust ways to reach agreement about guideline content by countering common biases associated with group-decision making (Moher, Schulz, et al., 2010). As a result, a wider group of stakeholders could be consulted when identifying standards for revised versions. A review of 37 reporting guideline developers suggests that involving various stakeholders in guideline development and dissemination leads to more uptake of the guideline and greater influence on the quality of research publications (Simera, Altman, Moher, Schulz, & Hoey, 2008a). Important stakeholders include primary researchers, systematic reviewers, funding agency representatives, policy makers, consumer group representatives, and any others who have an interest in the research area targeted by a guideline.

Given the importance of a reporting guideline’s accessibility to its overall impact, one potential barrier to the influence of this book relates to the issue of open access. Unrestricted online availability is a critical publication strategy used for the majority of prominent guidelines to better facilitate the implementation and uptake of reporting standards. Online open access facilitates the widespread use and appraisal of a reporting guideline by all interested stakeholders, which is essential both to its impact and to its feedback for future updates of the guideline. While the scholarly article introducing JARS is freely available online, the need to pay for and order a physical copy of Reporting Research in Psychology could prevent researchers from accessing and using it. This in turn may hamper the impact of JARS, for the detailed explication of each reporting standard (found only in Reporting Research in Psychology) is essential to effective adherence. Cooper and other representatives from the APA and JARS Group should consider online open access for Reporting Research in Psychology, similar to the availability of the CONSORT Explanation and Elaboration Document (Moher, Hopewell, et al., 2010), to remove these hurdles to its use and influence in the field.

Given the wide range of study designs that this book aims to cover, another potential barrier to its effective use is the breadth of its standards. Cooper notes that some standards may be irrelevant for a particular study, so authors should decide which to employ for each article (p. 24). The discretionary use of standards obstructs the purpose of reporting guidelines: to provide a minimal, essential set of recommendations to report in any study for a given research method (Moher, Schulz, et al., 2010). Having authors go through a comprehensive list of standards and assess which are relevant for their article may lead to improper use of the guideline, due to poor judgment or fatigue, for example. Future versions of JARS could employ the aforementioned consensus methods to highlight the most essential items on the “all-purpose” list, or to divide this list into more modules for particular research methods if length proves a barrier to uptake. Evaluations of the impact of the guideline—a key practice for keeping guidelines robust and current (Moher, Schulz, et al., 2010)—could help determine whether this issue proves important and, if so, how to address it.

Though primarily targeted toward psychological research and APA journals, Reporting Research in Psychology is a helpful tool for many stakeholders in the wider social and behavioral sciences, particularly social work. First and foremost, it provides a checklist for early career and senior scientists to use when writing reports of their research. A variety of research designs, settings, topics, and populations encountered in social work are covered in this book, giving it the potential to increase the quality of journal articles in this discipline. Transparent reporting of research has the capability to improve the way researchers conduct studies as well. Accountability of study quality via clear and complete reporting at the closing stages of a project may incentivize researchers to design more rigorous studies at the outset.

It is also a helpful reference book for undergraduate and graduate students taking research methods courses and learning about the essential features of empirical studies. Educating young researchers about basic principles of research reporting can improve the quality of submitted manuscripts in the future. Furthermore, the use of this book by peer reviewers and editors can strengthen and make more efficient the entire editorial process (Simera et al., 2008b). It can also assist research funders in appraising manuscripts and grant applications, facilitating the acceptance of high-quality submissions.

Reporting Research in Psychology is a welcome guideline in social work that can help authors clearly, completely, and accurately report their research. Journal articles are the primary means of disseminating research findings with the wider social work community. As a result, high quality reporting of research is essential for evidence-based decision making; inadequately detailed articles could be used improperly in guiding future research, practice, and policy decisions. Better reporting of primary and secondary research as a result of standards like those found in this book can improve the evidence base for social work practice and policy, leading to better outcomes for the vulnerable populations that this profession serves.