The introduction of Registered Reports in the Autism journal: The what,why and how

What are Registered Reports?

Registered Reports are a publication format in which the peer review process occurs in two stages. At Stage 1, the introduction, research questions and hypotheses (as appropriate), methods and analysis plan are reviewed, and if necessary, revised, before data collection and/or analysis. After this review process, the manuscript may be granted ‘In-Principle Acceptance’, which means the authors can conduct the research with the assurance that the manuscript will be accepted at the journal if the authors implement their pre-registered plans and interpret the results accordingly. The authors are asked to post the Stage 1 manuscript on a recognised online repository after In-Principle Acceptance (it is not published by the journal). After conducting the research, the authors then submit a Stage 2 manuscript for review for publication in the journal, which is a full paper with Results and Discussion sections, resembling a typical Research Report. At this stage, reviewers evaluate whether the pre-registered protocol has been followed (or if deviations were documented, whether these were justified and reasonable; Lakens, 2024) and whether the conclusions are consistent with the data. The Stage 2 manuscript cannot be rejected on the basis of hypotheses being unsupported or concerns with methodology or novelty (which were reviewed at Stage 1). However, deviations from the pre-registration without a well-reasoned rationale could lead to rejection.

Registered Reports are only one type of pre-registration available to our authors. Other methods include pre-registering hypotheses and analysis plans on the Open Science Framework or aspredicted.org and registering clinical trials in a public trials repository. However, unlike Registered Reports, these pre-registrations are not peer-reviewed, and there is no assurance the results will be published in a journal. Registered Reports were first offered as an article type by the journal, ‘Cortex’ in 2013 (Chambers, 2013). There are now over 300 participating journals, covering an expanding range of fields (Chambers & Tzavella, 2022).

Full details of how we are applying the Registered Reports format are included in our Submission Guidelines. ¹ Importantly, we are offering both Pre-Data Registered Reports, for manuscripts where no data have been gathered, and Post-Data Registered Reports, for when data have already been gathered but not yet examined or analysed. While the review process is similar, methodological concerns about data collection in Post-Data Registered Reports cannot be corrected, which may lead to rejection at Stage 1.

Why do we now offer Registered Reports as an article type?

The Registered Reports movement aims to counteract publication biases in academic publishing. Positive, clear-cut findings that support a hypothesis are more likely to be published than null or inconclusive findings (Franco et al., 2014; Greenwald, 1975). Academics therefore sometimes do not, or cannot, publish their null results (the ‘file drawer problem’; Rosenthal, 1979). Traditionally, academics are incentivised to publish as many papers as possible, particularly in ‘high-impact’ journals, as publication metrics influence hiring, promotion and grant funding and other markers of esteem (Nosek et al., 2012). Relatedly, authors sometimes engage in dubious research practices to increase their perceived chances of getting their paper published, such as selectively reporting parts of a study’s results that support their claims, running multiple variants of their analyses until they obtain a positive result (‘p-hacking’; Simmons et al., 2011) or changing their hypotheses to fit their results, to make a more compelling narrative (HARKing, or ‘hypothesising after the results are known’; Kerr, 1998).

These practices significantly limit the credibility of published results. Registered Reports counteract this by ensuring that publication decisions are made on the basis of the research questions, methodological rigour and planned analyses, without being biased by the results, and by maintaining a clear distinction between confirmatory and exploratory results. Consequently, the proportion of positive results is considerably smaller for Registered Reports than for other article types (Scheel et al., 2021). Pre-Data Registered Reports also allow methodological flaws to be identified and corrected before data collection, which can lead to increased research quality (Soderberg et al., 2021).

Increasing the credibility of research findings is particularly important for our journal’s focus on publishing research of direct and practical relevance to improve autistic people’s quality of life. Practical applications based on a weak, distorted evidence base may not improve autistic people’s quality of life and could even lead to harm. By correcting methodological flaws before data collection, and ensuring that results are published regardless of their strength and direction, we can also reduce the chance of participants’ time being wasted.

Registered Reports can confer further benefits to authors, reviewers and journals. Authors are freed from the pressure to publish positive, conclusive results. Acceptance rates tend to be relatively high for Registered Reports, and the timeline for publication is more predictable following data collection (Bishop, 2016). For early career researchers who need to build their publication records to secure research funding or salaried positions, a Stage 1 manuscript with In-Principle Acceptance may carry more weight with funding and hiring committees than an ‘in-preparation’ manuscript, with no assurance of whether it will be published (Bartlett, 2016). Registered Reports also allow authors to demonstrate their commitment to open, reproducible research practices, which are increasingly emphasised by awards, hiring and promotion committees (Allen & Mehler, 2019; McKiernan et al., 2016). Accordingly, early career researchers have been some of the earliest authors of Registered Reports (Bartlett, 2016; Chambers & Tzavella, 2022). For reviewers, Pre-Data Registered Reports allow input into the publication process before it is too late, when the data have already been conducted. For journals, the adoption of Registered Reports could ultimately lead to fewer retractions and author misconduct allegations.

Despite these benefits, our field has been quite slow to adopt Registered Reports. No autism-specific journals offered Registered Reports in May 2021, when Hobson et al. (2021) called for their introduction in an open letter with over 100 signatories. Later that year, the journal, ‘Research in Autism Spectrum Disorders’ opened a Special Issue for Registered Reports. We are pleased to now offer another Registered Reports option for autism researchers. We recognise that Registered Reports are not the only way to improve the credibility and quality of autism research (Hobson et al., 2022, 2023), and we understand that not all types of research will lend themselves to the Registered Report format, as discussed below. This is why Registered Reports are being introduced alongside our existing formats, as part of our ongoing effort to improve transparency and rigour (Fletcher-Watson et al., 2021).

How do authors complete a Registered Report for publication in ‘Autism’?

Before submitting a Stage 1 Registered Report to our journal, authors first need to consider whether their research question(s) fits within our aims and scope. As Editors, we will be applying the same criteria at Stage 1 as for other article types, that is, does the proposed work make an important contribution to the literature and fit within our scope, which prioritises research of direct and practical relevance to autistic people’s quality of life? We consider and publish replication attempts where a clear case is made for why a replication effort is necessary, in relation to our scope. Stage 1 manuscripts not aligning with our scope will be rejected.

Next, authors must decide whether a Registered Report is the most appropriate article type. A Registered Report may not be suitable if the research is purely exploratory and if analytical choices cannot be specified up-front (Chambers, 2019). Registered Reports are most commonly used for quantitative studies but can also be used for qualitative research (Karhulahti, 2022; Karhulahti et al., 2023). Authors will also need to consider the timeline, allowing time for the Stage 1 review process before data collection and/or analysis. As Editors, we will endeavour to make this review process as quick as possible, but will still be subject to reviewer availability, as for all article types. If the manuscript is not rejected at Stage 1, reviewers and Editors will likely suggest some revisions, which may need to go back to reviewers, before we can grant In-Principle Acceptance. This timeline could make it difficult to complete within short, fixed-term positions or degree courses (Bartlett, 2016; Maizey & Tzavella, 2019), or when the data collection and/or analysis needs to begin at a predefined time (Chambers, 2019). The upside to the extra time taken before data collection and/or analysis is that the review process at Stage 2 should be quicker, more straightforward and predictable (Bishop, 2016).

If authors decide a Registered Report is appropriate, they can use our detailed author guidelines ² to prepare their Stage 1 submission. Following these guidelines will reduce the chance of desk rejection and minimise delays in the review process. Here, I highlight three tips:

If granted In-Principle Acceptance at Stage 1, authors must then upload their Stage 1 manuscript onto a recognised online repository and conduct the research as planned. They can then prepare their Stage 2 manuscript. Authors should not make changes to the Introduction and Methods from the Stage 1 manuscript, apart from changing the tense, correcting typographical errors or other minor stylistic revisions. Authors’ Stage 2 submission should link to their Stage 1 protocol, and any deviations from the pre-registration must be transparently explained and justified, with substantial deviations discussed with the Editor. Authors are welcome to include additional analyses that were not included in the pre-registered protocol, but these must be clearly demarcated as exploratory analyses and interpreted accordingly. Authors should also include links to the data and materials. Accepted Stage 2 manuscripts will be published in the journal clearly marked with the ‘Registered Reports’ article type. A diagram outlining possible outcomes following Stage 1 and Stage 2 submissions is provided in our Submission Guidelines. ²

How should reviewers review Registered Reports?

Reviewers who are unfamiliar with Registered Reports will also benefit from our Submission Guidelines, ² which include lists of what reviewers need to look for at each Stage. We hope to be able to call on the same reviewers for Stage 1 and 2, but we understand that occasionally this will not be possible. At Stage 1, reviewers should judge the importance of the proposed research for the field and for improving autistic people’s and their families’ lives, following our journal’s aims. They should also evaluate the connections between the theory, empirical evidence to date, research questions and hypotheses with the proposed methods and analyses. Reviewers should pay particular attention to methodological rigour, including sample size justification, the existence of appropriate control conditions and the analytical pipeline in different scenarios (e.g. violation of assumptions, missing data). Reviewers should also assess whether the manuscript is presented clearly and transparently.

At Stage 2, reviewers should re-read the Introduction and Methods sections to verify that nothing has changed, but not ask for revisions to these sections, apart from minor stylistic or typographical edits. Reviewers should mainly focus on the Results and Discussion, and in particular, whether the authors have followed their pre-registered plans (or transparently explained where they have deviated from them, and why), and whether they have reached reasonable conclusions in data interpretation. Reviewers do not need to give views on the novelty, perceived importance or conclusiveness of results at Stage 2, as these will not affect the Editorial decision following In-Principle Acceptance. Reviewers could suggest additional post hoc analyses at this stage, but these must be clearly labelled as exploratory, and acceptance of the Stage 2 Registered Report will not depend on the authors completing these.

Conclusion

We are pleased to now be offering Registered Reports at our journal, as one way to promote open and reproducible research practices and counteract publication biases within autism research. Registered Reports represent a different way of publishing which entails new considerations for authors, reviewers and Editors. We hope that this Editorial has explained what Registered Reports are, why we are introducing them and how to get started. We look forward to publishing our first Registered Report.