Abstract
The open-science movement has brought many benefits to psychological science. One of the most salient is an increase in the number of articles with publicly accessible data, materials, analysis code, and preregistration information. This greater openness allows other researchers to “look under the hood” to see how the science was conducted. It also allows a more cumulative science to be built more quickly by letting other researchers use data and materials already developed and vetted through the peer-review process. By building directly on prior work, we have the luxury of more valid comparison across studies, permitting us to determine where studies were and were not replicated, and even sometimes why replication failed.
There are many ways we can measure the adoption of open-science practices. A readily accessible metric is the number of badges that are awarded for Preregistration, Open Materials, and Open Data (see the Center for Open Science website for more information about badges; https://www.cos.io/initiatives/badges). Psychological Science began awarding badges for these practices in 2014. Since the introduction of badges, the percentage of articles that have earned them, and thus contributed to the opening of science, has grown substantially. As reflected in Figure 1, a whopping 78% of articles published in 2020 earned a badge in at least one of these categories. There also has been impressive growth in the percentage of articles that have earned all three badges, with a threefold increase since 2017.
Percentage of articles published in Psychological Science that have earned between one and three of the badges (blue line), any two of the badges (orange line), or all three of the badges (gray line; 2014–2020).
The most common form of sharing is the subject of this Editorial. As reflected in Table 1, in 2020, the authors of almost three quarters of articles published in Psychological Science made their data publicly accessible (or used publicly accessible data portals or sets). The practice of making data publicly—and permanently—available improves transparency and also brings efficiency to psychological science. Rather than duplicating data-collection efforts, we can make use of others’ data, thus accumulating observations more quickly. Indeed, we can think about open data as a force multiplier. In military terms (the origin of the descriptor), a force multiplier is a tool that significantly increases combat potential. More generally, force multipliers help to amplify effort to produce more output or get more done with the same amount of effort. A simple, concrete example is a hammer, which makes driving a nail faster and more efficient relative to doing it by hand. Open data can do the same. The use of open data also democratizes psychological science by making larger amounts of information available to individuals and research groups than they could ever hope to collect on their own. Instead, they can use data provided by a third party.
Articles Published in Psychological Science That Earned Each of the Three Badges (2014–2020)
As we celebrate the growth of open data and the opening of psychological science it affords, we also need to attend to some sobering truths about using data that we did not personally collect. We readily recognize the risks associated with, for example, eating food we did not cook ourselves. It could have been handled in an unsafe manner, conceal ingredients to which we are allergic, or be heavily saturated with salt or fats, to name a few possibilities. The same risks lurk within data we did not personally collect. Moreover, even if the food was prepared under pristine conditions and contains no allergens, and we are clear about its salt and fat content, there is still opportunity for risk in the ways that the food is stored, reheated, and arranged on the plate. Even in these simple actions, we must behave in a responsible manner. The same applies to the use of publicly accessible data and data portals. Unless we are responsible and know how the data were prepared and how they should—and should not—be used, we risk harm to our science.
In light of the growing use of open data, I commissioned two Data Briefs, in which I asked authors to reflect on some of the benefits, and also the risks and responsibilities, associated with the use of publicly accessible data sets. I am indebted to Manuel Eisner and Pasco Fearon for taking up this challenge in the context of large-scale, cross-national data portals on homicide rates. In their Data Brief, Eisner and Fearon point out the “force multiplying” features of such data. They also highlight the need for users to be intimately familiar with how the data were generated and whether they are adequate and well-suited for the research purpose. I also am indebted to Cortney Simmons and her colleagues for taking up this challenge in the context of The Adolescent Brain Cognitive Development (ABCD) Study, the largest longitudinal study of brain development and youth health in the United States to date. In their Data Brief, Simmons and colleagues suggest best practices for responsible use of data from the ABCD Study, including consideration of the broader social context in which development occurs and the potential for developmental change. I thank both groups of authors for their contributions to opening science.
The two Data Briefs in this issue are written in the context of specific data portals and sets, yet the lessons to be learned from these examples generalize well beyond them to the proliferation of large-scale data portals in which nations are scored on a variety of attributes—such as their values and attitudes and levels of democracy, poverty, and religiosity—and data sets are provided on dimensions such as child well-being, the economy, education, the environment, gender inequality, and health, to name only a few. Just as we should not cite research from a secondary source without reading the original source, we should not use data without “reading” the original sources and without considering the cultural, developmental, and social context in which they were collected.
Psychological science in general and Psychological Science in particular have benefited enormously from the progress made toward opening science and increasing the transparency with which we conduct our craft. We should and do celebrate the empirical and theoretical advances that have come from these developments. I thank Manuel Eisner and Pasco Fearon as well as Cortney Simmons and her colleagues for encouraging us to continue to make progress in opening science and to do so with our eyes wide open to the benefits but also to the attendant risks and to our responsibilities.