CyberSightings

In the Spotlight

Replicability is a critical requirement of good scientific research. When a study is successfully replicated by other investigators, the underlying theory and findings are corroborated. To ensure replicability, reports of experiments published in scientific journals have to include detailed descriptions of what has been done, how the study was carried out, sampling procedures, materials, data analysis, and so on. Recently, replicability has become a hotly discussed topic in psychology due to a highly cited experiment that failed to replicate. The journal Perspectives in Psychological Science dedicated a special section (November 2012; 7/6) to this issue, collecting a set of articles that discuss the causes of the replicability problems in psychology and suggesting possible strategies to overcome them. One of the key outcomes of this debate is that replicability problems are not easily addressable, yet neither are they insurmountable. In particular, the emerging Science 2.0 paradigm can offer interesting solutions to address these challenges. An interesting initiative in this direction is the Reproducibility Project (http://openscienceframework.org/project/EZcUj/wiki/home/), which is attempting to replicate every single study published in 2008 in three leading academic psychology journals (Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition). The individual researchers or scientific teams who wish to replicate an effect from a selected article are invited to follow a set of instructions for designing and conducting it. Collaboration can take different forms, from helping with data analysis to reviewing replication protocols, to creating stimulus presentation scripts. This effort is rewarded by receiving authorship on project reports. So far, the project has seen the collaborations of several dozens of researchers from a number of international institutions, and it has been featured in prestigious scientific journals, including Science and Nature. Following the same line, the Web-based archive PsychFileDrawer (http://psychfiledrawer.org/) allows psychologists to post results of experiments that have failed to replicate, which would otherwise be ignored because most journals decline to publish straightforward replication studies. Furthermore, in this case, the objective is to encourage researchers to try to replicate the work of others, filtering out weak hypotheses and strengthening experimental psychology. On one hand, initiatives like the Reproducibility Project and PsychFileDrawer represent an innovative and potentially effective solution for the problem of replicability in psychology; on the other hand, they raise concerns about the possible risks brought about by this approach. These include, for example, the difficulties in identifying the reasons when a study fails to replicate, and the fact that low reproducibility rate may lead to funding cuts for basic psychological research. However, these risks appear acceptable when compared to the benefits that this strategy can bring to the advancement of our discipline.

Upcoming Meetings