Open Science and Single-Case Design Research

Inductive and Deductive Approaches to SCD

As proposed by Johnson and Cook (2019), SCD researchers have varied perspectives regarding the purpose and conduct of SCD research, ranging from deductive-static to inductive-dynamic approaches. The inductive-dynamic perspective is closely aligned with the core principles of applied behavior analysis, including the primacy of achieving socially significant outcomes for the participant (Wolf, 1978). Researchers who are more on the inductive-dynamic side of the continuum primarily seek to identify contingencies controlling a participant’s behavior rather than examine the effects of a specific intervention on a specified outcome. As such, these researchers may modify independent and dependent variables during the study if the (a) intervention does not result in therapeutic and observable change of the dependent variable or (b) the participant begins exhibiting another behavior requiring immediate attention, respectively.

In contrast, SCD researchers who utilize more deductive-static perspectives maintain the independent and dependent variables throughout their study to test an a priori hypothesis (Johnson & Cook, 2019). For example, a researcher may aim to understand whether an intervention affects participant responding in ways consistent with prior research. Such research may be considered deductive because it poses an a priori hypothesis regarding the effects of an intervention, a study is designed to test the hypothesis, and then data are collected and evaluated. The defensibility of findings of such SCD studies requires the possibility of the intervention not causing improved outcomes (i.e., non-effects). SCD studies utilizing a deductive-static approach seem appropriate when examining the effectiveness of interventions and potentially establishing those interventions as evidence-based practices.

Masked Visual Analysis and Randomization

SCD researchers have traditionally used a response-guided approach to inform decisions made during the course of a study, such as when to initiate intervention (Johnston & Pennypacker, 2009). Although the intent of researchers making decisions in response to data is to reduce ambiguity associated with variable response patterns, they may not always be fully reported (perhaps due to page limits), can be influenced by knowledge of environmental contexts (e.g., waiting to intervene until the day after a class party), and can influence study findings (e.g., increase the likelihood of data supporting a functional relationship; Allison et al., 1992). Similarly, procedures for conducting visual analysis, the primary approach for examining the presence of a functional relationship, are often not clearly reported and inherently involve some level of subjectivity (Ninci et al., 2015). Some researchers have proposed randomization and masked visual analysis to increase the transparency and reproducibility of SCD decision-making and data analysis.

Randomization can be used in SCD research to determine the order of participants to receive the intervention, the session number at which to intervene, and whether the participant will receive the intervention or baseline condition first—thereby making these decisions fully transparent and objective (Edgington & Onghena, 2007). Masked visual analysis uses randomization in combination with an independent researcher who is not aware of conditions and other study details to make decisions about phase changes (Ferron & Jones, 2006). For instance, in a multiple baseline design, a masked visual analyst inspects graphed data in an ongoing, response-guided manner. Once stability is reached for participants in baseline, the visual analyst instructs the research team to randomly select a participant to receive the intervention. Following implementation of the independent variable to the selected participant, the masked visual analyst—unaware of which participant has received the intervention—continues to review outcome data until there is evidence of a meaningful change in outcomes for one participant, at which point they instruct the research team to randomly select a second participant to receive the intervention, and so on. Masked visual analysis guards against researchers capitalizing on their intimate knowledge of the study by (a) reducing the likelihood of researchers introducing the intervention based on a brief period of stable responding not representative of the participant’s true performance, and (b) providing an objective and transparent means for determining when and with whom to intervene.

Randomization (with or without masked visual analysis) also allows for using a widely accepted standard for estimating the probability of rejecting the null hypothesis (statistical significance testing), rather than relying solely on visual analysis. Applicable to a wide range of SCDs, randomization testing establishes the context for statistical permutation tests to evaluate intervention effects (Ferron & Levin, 2014). When using randomization testing, researchers construct a distribution of all possible outcomes for the test statistic to compare with the observed performance of the participants (Todman & Dugard, 2001). The resulting p value represents the probability of obtaining the observed data given the distribution of possible outcomes if there was no actual improvement during the intervention compared with the baseline phase (i.e., the null hypothesis). Although masked visual analysis and randomization testing provide opportunities for researchers to investigate interventions with increased transparency in decision-making and analyses, response-guided approaches and visual analysis remain important features in SCD.

Meta-Analysis and Effect Sizes

Methodological advances in meta-analyzing SCDs have focused on supporting credible causal relations in the SCD literature used to establish generally effective, evidence-based practices (Shadish et al., 2015). The inductive tradition of single-case research, as well as core design features such as individuals as the unit of analysis, repeated measures, and reliance on visual analysis, present barriers to meta-analyses. In response, researchers have developed meta-analytic methods to measure the magnitude of treatment effects in SCD research. Although multiple SCD effect sizes have been proposed (Maggin et al., 2019), some consensus is developing in support of effect sizes for generalized inferences (Shadish et al., 2015) that represent the average magnitude of change across study cases in units comparable with those used in experimental group research (Barton et al., 2017; Odom et al., 2018).

Non-Effects

SCD researchers have traditionally provided evidence for internal validity via study results (Johnston & Pennypacker, 2009). Internal validity is evident when changes in participant outcomes reliably coincide with the manipulation of an independent variable. This inductive approach makes it difficult to interpret SCD study results when changes in the independent variable do not correspond reliably with changes in the dependent variable (i.e., non-effects; Tincani & Travers, 2018). Thus, few SCDs with non-effects have been published (Kittelman et al., 2018; Therrien & Cook, 2018), and SCDs have not traditionally been used to identify ineffective interventions or the boundaries of effective interventions (Kratochwill et al., 2018; Tincani & Travers, 2018). The scarcity of published SCDs with non-effects may undermine the credibility of syntheses and meta-analyses of the published SCD research base as well as efforts to establish evidence-based practices using SCD research.

Tincani and Travers (2018) suggested SCD researchers design their experiments to allow for the possibility of valid findings of non-effects. For example, SCD researchers may use specific designs (e.g., alternating-treatments, A-B-C-B-C designs) to both (a) establish a functional relationship between an independent and dependent variable and also (b) permit claims about whether another intervention was partially or entirely ineffective (Ledford & Gast, 2018). Another approach entails using a well-established design and addressing critical indicators of internal validity for SCDs (e.g., document intervention intensity; establish adequate treatment integrity, control condition integrity, and procedural fidelity; conduct a moderator analysis). To meaningfully test the effectiveness of interventions and contribute to establishing generally effective interventions, deductive-static approaches to SCD research should allow for the possibility of valid findings of non-effects (Kratochwill et al., 2018; Shadish et al., 2016).

Collectively, these developments in SCD apply a deductive-static perspective to enhance the ability of SCD researchers to establish the generalizable effectiveness of interventions. In the following section, we describe open-science reforms and the issues in the research enterprise they aim to address, and whether and how prominent open-science practices might build on the developments described in this section to advance the credibility and impact of the SCD research base, especially in relation to establishing generally effective practices using a deductive-static orientation.

Issues in Contemporary Research

Over the past decade, there have been multiple reports about the limited reproducibility of scientific research (e.g., Open Science Collaboration, 2015), raising concerns about the credibility of scientific claims. Evidence suggests relatively few replication studies are conducted, and when independent replications are conducted, they often fail to reproduce the results reported in the original studies (Makel et al., 2016; Makel & Plucker, 2014). One factor underlying low levels of reproducibility may be pressures on researchers to obtain positive results to publish their studies in influential journals (Nosek et al., 2012). Such pressures may foster publication bias and other questionable research practices, resulting in irreproducible study findings and research bases biased toward positive findings (Giner-Sorolla, 2012).

Indeed, evidence suggests publication bias—or the over-representation of studies with positive findings in the published literature—exists in many research bases (Dwan et al., 2013; Ferguson & Heene, 2012; Vera-Badillo & Riechelmann, 2018), including in education and special education (Chow & Ekholm, 2018; Gage et al., 2017; Polanin et al., 2016). Accordingly, null findings in the published research base are exceedingly rare (Fanelli, 2012). At the study level, non-transparent researcher flexibility may promote unreliable, irreproducible results (Simmons et al., 2011). To obtain desired findings, researchers have the flexibility to engage in, but not fully report, a variety of questionable practices. For example, researchers have reported engaging in data peeking (i.e., checking findings throughout the study and stopping data collection when desired results are attained), p-hacking (e.g., running different analyses until desired results are attained), selective outcome reporting (i.e., selectively reporting statistically significant results), and hypothesizing after results are known (i.e., HARKing; Fraser et al., 2018; John et al., 2012; Makel et al., in press). Engaging in these types of practices markedly increases the likelihood of false positive results (Simmons et al., 2011). To be clear, there is nothing intrinsically wrong with exploring multiple approaches to analyzing data or generating a hypothesis from one’s data—what Nosek et al. (2018) termed postdictive research. Such practices become questionable when they are not made transparent.

A related issue is the relative inaccessibility of most published research, which typically lies behind paywalls and is accessible only to those willing and able to pay (Piwowar et al., 2018). The inaccessibility of research to practitioners is especially problematic in applied fields, like special education, and likely contributes to the long-standing gap between research and practice. Simply put, practitioners cannot be expected to inform their practice with research they cannot access.

Open-Science Practices

Open-science reforms have been proposed to help address irreproducible research findings, publication bias, undisclosed questionable research practices, and inaccessibility of research findings by making research more open, transparent, and reproducible—with the ultimate goal of improving the credibility and impact of scientific research. In this section, we briefly discuss four prominent open-science practices: preregistration, registered reports, open data and materials, and open access.

Preregistration (Gehlbach & Robinson, 2018; Nosek et al., 2018) involves researchers publicly registering key study elements—such as research questions and hypotheses, variables, design, and analyses—before conducting the study. Preregistration allows for a clear distinction between (a) predictive or confirmatory research (i.e., using predetermined procedures to test a priori hypotheses) and (b) postdictive or exploratory research (i.e., analyzing a data set in multiple ways to explain latent variability). Preregistration does not preclude exploratory analyses, but rather provides a transparent basis for determining when researchers are conducting them. Preregistration enables reviewers and other research consumers to examine whether researchers may have engaged in questionable research practices such as HARKing, selective outcome reporting, and p-hacking by identifying unexplained discrepancies between posted study plans and research reports. Perhaps by making questionable research practice discoverable, preregistration is associated with smaller—and potentially more credible—study effects (Kaplan & Irvin, 2015; Schäfer & Schwarz, 2019).

Registered reports (Chambers, 2019; Cook et al., in press) involve researchers submitting study plans for peer review before conducting the research. In Stage 1 review, researchers submit prospective introduction and method sections, and reviewers provide constructive feedback to improve the prospective study. Based on the importance and methodological rigor of the study, the Stage 1 manuscript may be granted in-principle acceptance for publication. Then, after conducting the study, researchers write their results and discussion sections and resubmit the final report for Stage 2 review. Reviewers then evaluate the paper for adherence to study plans (deviations should be identified and explained) and completeness of reporting. Importantly, the study cannot be rejected due to direction, significance, or magnitude of effects. Registered reports guard against undisclosed questionable research practices such as HARKing, p-hacking, and selective outcome reporting by removing the incentive of generating positive findings to increase the likelihood of publication. Indeed, the only way for a Stage 1 proposal with in-principle acceptance not to be accepted at Stage 2 is if the researchers do not report and justify modifications to the approved plans. Moreover, registered reports help address publication bias because publishability is based on methodological rigor and determined before results are known. Like preregistration, registered reports are associated with smaller, and perhaps more credible, study effects (Allen & Mehler, 2019; Scheel et al., 2020).

Data and materials sharing are at the heart of research transparency and open science. Researchers who make study data openly accessible (e.g., by posting on a repository) provide the opportunity for others to independently verify, replicate, and extend study findings (Adelson et al., 2019; Levenstein & Lyle, 2018). Thus, data sharing can enhance the credibility and reproducibility of findings by allowing others to check for errors in data entry, data analyses, and reporting of findings. In addition, open data are associated with greater citation rates and may therefore improve scholarly impact (McKiernan et al., 2016; Piwowar & Vision, 2013). Materials sharing involves publicly posting information and resources (e.g., survey items, details on intervention implementation) not included in published manuscripts due to length restrictions. Materials sharing can be critical for improving the ability of other researchers to replicate studies.

Open access subsumes multiple approaches for making research freely available to research consumers. For example, researchers can pay an article-processing charge to make articles freely accessible in gold- (journals in which all articles are open access) and hybrid- (traditional journals in which most articles are behind a paywall but specific articles are open to all) access journals. Researchers can also provide open access to their research by posting an author-formatted version of a paper before it is submitted for publication (i.e., preprint) to a preprint archive (e.g., www.edarxiv.org; Fleming & Cook, in press). Open-access and preprinted studies appear to increase the effects of research, as these papers receive more citations and social media attention than those behind paywalls (Fu & Hughey, 2019; Piwowar et al., 2018). Preprints may also help address publication bias because researchers can post and make discoverable any study, regardless of direction of findings.

Concerns about the reproducibility and transparency of research findings, as well as the promotion of open-science reforms to address these concerns, have focused primarily on group-design research, with relatively little attention paid to whether and how such issues may apply in SCD (Kratochwill et al., 2018). However, as discussed in the following section, these issues may have important implications for and applications in SCD.

Issues in Contemporary SCD Research

Although the reproducibility crisis has received relatively little attention in the SCD research community, SCD researchers may also experience contingencies reinforcing positive effects. As Kratochwill et al. (2018) noted,

Some part of the SCD research community may have an explicit ethos that an intervention must produce a functional relationship . . . for a study to proceed through the publication process. If so, publication bias in SCD research is typically driven by finding/producing effects rather than by the common “statistical significance” criterion that is characteristic of conventional “group” research. (p. 69)

In other words, although SCD researchers may not have a bright-line test of statistical significance (p < .05) to evaluate the effects of their research, they may nonetheless perceive studies demonstrating functional relations as more publishable.

Indeed, Shadish et al. (2016) reported that SCD researchers are significantly less likely to recommend a study for publication if it has small as opposed to large effects. Although Shadish and colleagues’ survey described a “very well done” A-B-A-B study involving three cases (p. 660), when the effect was small, data overlap high, and data variability low, the average rating for recommending the paper for publication was between somewhat and very unlikely. In contrast, when the effect was large, data overlap low, and data variability low, the average rating was between somewhat and very likely to recommend publication (see also Mahoney, 1977). Strong intervention effects enhancing the likelihood of publication among SCD researchers raise the possibility that publication bias and undisclosed questionable research practices may exist in and undermine the credibility of claims based on SCD research.

When Sham and Smith (2014) compared the effects of pivotal response therapy for children with autism reported in unpublished dissertations and published studies, average percentage of nonoverlapping data (PND) for the dissertations (k = 10) was 47%, compared with 69% for published studies (k = 21)—a large (d = 3.93) and statistically significant difference. Both published and unpublished studies were generally high quality and involved similar participants and interventionists. The authors concluded disparity in effects may, then, “be indicative of publication bias such that studies with favorable outcomes are more likely to be published than studies that are less propitious” (p. 674). See Kratochwill et al. (2018) for further discussion of publication bias in SCD research. Consistent with findings suggestive of publication bias, non-effects are rarely found in the published SCD literature. For example, Therrien and Cook (2018) reviewed intervention studies published between 2012 and 2017 in three research journals in the field of learning disabilities; none of the 23 SCD studies reported non-effects across all cases (see also Kittelman et al., 2018).

SCD researchers traditionally make important decisions after examining the data, such as how many data points to collect in each phase (i.e., response-guided methods). These decisions may be influenced, perhaps unintentionally, by aspirations to demonstrate positive effects and may not be reported transparently. Selective reporting of these post hoc decisions might be considered a questionable research practice, especially when SCD researchers are testing a priori hypotheses using a deductive-static approach. Some SCD researchers may change the hypotheses, independent variable, number of participants, dependent variables, tools for visually analyzing data, number and type of effect sizes, and other study elements during the course of a study in ways affecting study outcomes. If such changes are not fully reported, then readers may be left with the impression the intervention was more effective than it actually was. Indeed, in their survey of SCD researchers, Shadish et al. (2016) found “a non-trivial minority” (p. 656; 5%–14% depending on the condition) of respondents reported they would drop one or two cases with small effects before submitting a study for publication. Some participants noted they perceived results needing to be positive to warrant publication, and although they felt dropping a case was unethical, one participant noted “it is done quite frequently” (p. 666).

Published SCD studies, like other types of research, are stored behind paywalls and are largely inaccessible to many potential research consumers. Thus, it appears issues related to publication bias, undisclosed questionable research practices, and lack of accessibility may also exist in the SCD literature (Johnson & Cook, 2019; Kratochwill et al., 2018). To the degree these issues are common, they may pose threats to the credibility and impact of SCD research. Open science—and specifically preregistration, registered reports, data and materials sharing, and open access—may help address these issues in SCD research. As summarized in Table 1, we briefly describe how and why SCD researchers can engage in these key open practices.

OPEN IN VIEWER

Table 1.

Action Steps Needed to Engage in Open-Science Practices in Single-Case Design Research.

Practice	Researcher actions	External resources
Preregistration	• Prior to conducting study, publish research questions, hypotheses, target participant criteria and recruitment strategies, independent and dependent variable details, plan for executing phase changes, and plans for examining a functional relation and estimating effect sizes (as relevant) • Conduct study, updating preregistration with any modifications (and rationales for those changes) made prior to study completion • Write manuscript, citing preregistration and detailing any changes made from what was described in the preregistration and reasons for those changes	Registry of Efficacy and Effectiveness Studies https://sreereg.icpsr.umich.edu/sreereg/
Registered Reports	• Submit study introduction and prospective method sections to journal for peer review prior to conducting the research • Once Registered Report receives in-principle acceptance from journal, conduct study as proposed • Add results and discussion to accepted Stage 1 manuscript and resubmit the complete paper to journal • If authors closely adhere to approved methods, paper guaranteed journal acceptance	Center for Open Science Registered Report Primer https://www.cos.io/initiatives/registered-reports Journals that accept Registered Reports https://www.cos.io/initiatives/registered-reports
Data and Materials Sharing	• Write study IRB protocol to allow raw data to be shared • Once study is complete, upload data, code book, and study materials (e.g., intervention protocols) to a data repository	Data Repositories: Open Science Framework https://osf.io/dashboard Inter-University Consortium for Political and Social Research (ICPSR) https://www.icpsr.umich.edu/web/pages/
Open Access and Preprints	• Publish your paper in a gold or hybrid open-access journal, which involves paying a processing fee, and/or publish a preprint of your manuscript by posting manuscript to a preprint server prior to submission to journal	Directory of Open Access Journals https://doaj.org/ Education preprint archive https://edarxiv.org/

Note. IRB = Internal Review Board.

Preregistration

Preregistration in SCD is predicated on the assumption that transparently delineating research questions, design, and procedures before conducting a study increases credibility in the resulting findings by making questionable research practices more readily discoverable (Nosek et al., 2018). Johnson and Cook (2019) provided initial recommendations for specific components of SCD studies to preregister, with differential guidance for some points depending on the theoretical approach of the research team. The Registry of Efficacy and Effectiveness Studies (Society for Research on Educational Effectiveness, 2020) also provides a protocol for preregistering SCD studies. In the absence of consensus regarding how to preregister SCD studies, we provide a brief overview of some critical considerations.

Research questions should be preregistered, regardless of one’s theoretical viewpoint, although the presence of accompanying hypotheses is subject to the researcher’s view of their necessity and utility. The form of these hypotheses may be informed by the suggestions of Shadish and colleagues (2015), who argued that SCD is a process of complex hypothesis testing. Relevant participant characteristics should also be described to the extent possible. Because preregistration typically occurs before participant recruitment, specific characteristics may not yet be known. Thus, researchers might initially preregister the intended number of participants and their age ranges, and then update the preregistration with specific information as soon as participants are recruited. This provides protection against manipulation of study participants and corresponding results by a researcher. Unfortunately, removing cases with non-effects appears to be a practice engaged in by some SCD researchers and recommended by some reviewers (Shadish et al., 2016).

SCD researchers will likely be able to preregister information about research questions and participants regardless of their theoretical orientation. However, other components of a study’s preregistration will vary depending on researchers’ positions along the deductive-static to inductive-dynamic continuum. In general, researchers who are closer to the deductive-static end of the continuum should be able to preregister a majority of their design’s components, whereas researchers on the inductive-dynamic end will likely preregister their intended procedures for determining these components (Johnson & Cook, 2019). Preregistrations can then be updated once these decisions are made.

If a researcher designs a deductive-static study to examine the effect of a specific intervention on a specific outcome, then they will likely be able to preregister the core components, or active ingredients, of the intervention they plan to implement, as well as their measurement methods, operational definitions for the outcome variable, and expected effects of the intervention (e.g., will effects be immediate and large, or gradual and moderate?). On the contrary, a researcher may partner with a school to work with a student with aggressive behaviors, operationally define a target behavior after observations, identify the behavior function using a functional analysis, and then implement a function-based intervention based on the results of the functional analysis. In such an instance, rather than preregister specific independent and dependent variables, the researcher could preregister the decision-making procedures used for determining them, and then update the preregistration as soon as those decisions are made.

Finally, the timing of phase changes is one of the most fundamental decisions made by researchers during an SCD study. Response-guided decision making for phase changes is likely the norm for most SCD researchers, wherein a steady state should be observed before implementing an intervention. However, other more transparent methods are available for determining phase changes, including randomization, pre-determining a number of data points prior to implementation, and masked visual analysis (e.g., Kratochwill & Levin, 2014). Therefore, preregistration of phase-change rules should reflect the information available to the researcher prior to implementation. If, for example, a specific number of data points will be used in each phase, then that number should be preregistered. If a process for identifying a steady state is used, then this process should be carefully described in the preregistration.

Registered Reports

Although registered reports are becoming an increasingly common publication format (Chambers, 2019; Cook et al., in press), we are not aware of any published examples of SCD registered reports. As in group research, registered reports can help guard against undisclosed questionable research practices and publication bias in SCD by initially evaluating and conditionally accepting manuscripts prior to data collection. Thus, likelihood of publication is not enhanced by demonstrating positive results (e.g., a strong functional relation) but rather depends on conducting a well-designed, internally valid study as proposed. As such, SCD researchers taking a deductive-static perspective in their research should specify in Stage 1 manuscripts what design features and procedures are used to enhance confidence in the internal validity of a study in the absence of a functional relation (see Tincani & Travers, 2018).

Norms and practices for Stage 1 SCD manuscripts will likely vary along the deductive-static to inductive-dynamic continuum. Manuscripts for SCD studies using a more deductive-static approach can be written and reviewed similarly to group-research registered reports because researchers aim to adhere to a specific protocol as closely as possible in both. Thus, Stage 1 manuscripts should be as specific as possible, and Stage 2 manuscripts should fully describe and justify any changes that occurred. However, for registered reports of more inductive-dynamic SCD studies, Stage 1 manuscripts will need to be written and reviewed based on transparent decision-making protocols for aspects of the study not fully determined a priori. For example, some researchers may wish to manipulate the independent variable actively, depending on the responsiveness of individual participants. This will require researchers to develop and describe a clear decision-making protocol of when and how to modify the independent variable. Adherence to this decision-making framework, in turn, will be a determining factor for final acceptance and publication of the Stage 2 manuscript. Whereas studies from a more deductive-static perspective are likely always suitable for registered reports, researchers may be unable to describe the decision-making procedures thoroughly for some inductive-dynamic SCD studies a priori, making meaningful Stage 1 evaluation by reviewers difficult, if not impossible.

Data and Materials Sharing

Data from published SCD research can be electronically extracted from graphs with high reliability (Drevon et al., 2017), assuming one has access to the publication, which can then be used by other researchers to compute effect sizes, synthesize, and meta-analyze (Maggin et al., 2011). However, sharing raw data saves time associated with data extraction, eliminates occasional errors in extraction, and provides study data to those who cannot access the publication. Given the evolving and sometimes complex estimation of SCD effect sizes, we also recommend sharing analytic code used to calculate effect sizes in the published article. In addition, investigators can share other types of non-graphed data summarized in SCD studies (e.g., interobserver agreement data, social validity ratings, fidelity data). Sharing materials, including detailed intervention protocols and methods of data collection, is critical for enabling meaningful replication of SCD studies by independent researchers, and for applying the intervention with fidelity in practice. SCD researchers using more inductive-dynamic perspectives might share detailed information about why and how response-guided decisions were made, making this aspect of SCD research more transparent and replicable.

Open Access and Preprints

Removing paywalls that block access to most published SCD research—whether by publishing in a gold-access journal, paying an article-processing charge in a hybrid journal, or posting a preprint—enables practitioners and other stakeholders who do not otherwise have access to a publication to read and apply its findings. Providing open access to SCD research to broaden its impact and application is consistent with the strong emphasis on social validity in SCD research. Similarly, posting SCD research as a preprint allows research consumers access to research findings immediately, without having to wait for potentially multiple rounds of peer review and formatting by a journal. Although the lack of peer review for preprints is an important concern, open access may help address publication bias. Given potential bias toward publication of positive results in SCD (Shadish et al., 2016), researchers may not consistently write up internally valid SCD studies without a functional relationship because such studies may be perceived as unlikely to be published. Posting internally valid SCD research with small and non-effects as preprints, rather than relegating it to the file drawer, enables SCD researchers to openly disseminate their work so it can (a) inform research, policy, and practice; (b) help combat publication bias (by making these studies discoverable to research consumers and meta-analysts); and (c) garner citations.