Abstract
Current federal legislation emphasizes the use of programs, interventions, strategies, and activities that have been demonstrated through research to be effective. One way to increase the quantity and quality of research that guides practice is to conduct replication research. The purpose of this study was to undertake a systematic review of the replication research focused on self-determination conducted between 2007 and 2017. Using methods used by Cook and colleagues, we identified 80 intervention studies on topics related to self-determination, of which 31 were coded as replications. Intervention study trends, rate of replication studies, percentage of agreements between findings of original and replication studies, amount of author overlap, and types of research designs used are reported along with recommendations for future research.
Keywords
Following the lead of the fields of medicine, psychology, speech-language pathology, and social work as well as many others, the field of education has incorporated an approach in which high-quality research evidence is integrated with professional expertise as well as student/family preferences and values into the process of making educational decisions (American Speech-Language-Hearing Association, 2005). Simultaneously, education professionals are required to make decisions about what to teach students and to determine how effective the teaching and learning process has been based on the data they collect.
Both the Individuals with Disabilities Education Act (IDEA; P.L. 105–17) and the Every Student Succeeds Act (ESSA; P.L. 114–95) emphasize the use of programs, interventions, strategies, and activities that have been demonstrated through research to be effective. Identification of evidence-based interventions (EBI) tends to focus on four issues: research design, quality of research, quantity of research, and magnitude of effect size supporting the studies (Cook & Cook, 2011). EBI must show a clear cause and effect relationship with improved student outcomes using group comparison or single-case design research that is evaluated by high-quality experimental tests and published in high-quality peer-reviewed journals (Travers, 2016). One way to increase the quantity and quality of EBI is through the process of conducting replication research. As noted by National Science Foundation and the Institute of Education Sciences (2018), “Efforts to reproduce and replicate research findings are central to the accumulation of scientific knowledge that helps inform evidence-based decision making and policies” (p. 1).
Replication
Replication is “the process of repeating a research study with a different group of research participants using the same or similar conditions, for the purpose of increasing confidence in the original study’s findings” (Gall, Gall, & Borg, 2003, p. 635). Replication research has long been considered to be one of the fundamental building blocks of experimental research design (Fisher, 1935) and has even been referred to as “the Supreme Court of the scientific system” (Collins, 1985, p. 19).
Replication is considered to be at the heart of scientific inquiry (Schmidt, 2009) and critical to the scientific process because it attempts to demonstrate that the same findings can be obtained at another time, in a different place by other researchers, and as such, the results can be separated from the specific circumstances of the initial study, thus increasing the strength of the evidence of the effectiveness of the intervention (Cook, Collins, Cook, & Cook, 2016). Furthermore, replication is a means of controlling for threats to both internal and external validity that are inherent in any single study by allowing for a systematic examination of the impact of those threats across studies (Hosp, Ford, Huddle, & Hensley, 2018). Consequently, national funding agencies such as the Institute of Education Sciences and the National Science Foundation have included replication components in their guidelines. Concomitantly, in a recent document, they stated, “Given the central role of replication research in the progress of science, it is important that the education field promotes the conduct and dissemination of reproducibility and replication studies” (National Science Foundation and the Institute of Education Sciences, 2018, p. 3).
A review of the research across disciplines suggests that researchers generally divide replications into two main categories—direct and conceptual (Schmidt, 2009). Direct replication uses the exact study design, sampling, and methods as the original research, with no purposeful changes from those procedures (Makel et al., 2016). As explained by Mathews, Hirsch, and Therrien (2018), “direct replications serve as a check on the strength of an intervention as it was originally conducted and tested with the same population under the same circumstances” (p. 269). Direct replications are considered the strictest type of replication. However, the difficulty in duplicating the conditions and methods in experimental situations that involve complex human behavior can be a challenge in educational settings. Consequently, they are the least common in educational research (Schmidt, 2009).
Conceptual replications, in contrast, examine the same intervention as the original study, but purposefully adjust some aspects of the original study such as the design, sampling, methods, and/or analysis to examine the conditions under which the results occur. Conceptual replications help provide a way to answer the questions (a) for whom has the intervention been effective, (b) in which settings has the intervention been effective, (c) which components of the interventions are essential for it to be effective, and (d) which assessments have been used to demonstrate that the intervention is effective (Mathews et al., 2018). Conceptual replications are more common in educational research because they are easier to conduct and they provide researchers and educators with a better understanding of what interventions are effective, with which individuals, in which situations, and under what circumstances.
Self-Determination
Research indicates that developing the skills associated with self-determination positively impacts the achievement of education-related goals, employment, community participation, and quality of life outcomes for students with disabilities (e.g., Burke et al., 2018). As noted by Wehmeyer, Field, and Thoma (2012), “Over the past two decades, promoting the self-determination of students with disabilities has become a best practice in secondary education and transition services” (p. 171). Despite the benefits, students with disabilities typically have insufficient self-determination skills, demonstrating the need for interventions to support the development of self-determination (Shogren, Kennedy, Dowsett, Garnier Villarreal, & Little, 2014).
Self-determination became a major component of legislated transition services when the IDEA reauthorization in 1990 mandated that the needs, interests, and preferences of students with disabilities be taken into account when planning for their transition from school to adult life (Bremer, Kachgal, & Schoeller, 2003). Since then, IDEA of 2004 and a large number of research studies and meta-analyses (e.g., Algozzine, Browder, Karvonen, Test, & Wood, 2001; Burke et al., 2018; Fowler, Konrad, Walker, Test, & Wood, 2007; Test, Fowler, et al., 2009; Wehmeyer et al., 2012 Wood, Fowler, Uphold, & Test, 2005) have emphasized the importance of promoting the self-determination of students with disabilities.
Over the years, the conceptualization and definition of self-determination have undergone multiple iterations. The components of self-determination most commonly identified in the literature include (a) self-awareness/self-knowledge, (b) choice making, (c) decision making, (d) goal setting/attainment, (e) problem solving, (f) self-regulation/self-management, and (g) self-advocacy (Shogren, 2013). For practitioners, it has been defined as “the ability of individuals to live their lives as they choose, consistent with their own values, preferences and abilities” (Turnbull, Turnbull, & Wehmeyer, 2010, p. 500). For scholars, it has been defined as the dispositional characteristic manifested as acting as the causal agent in one’s life. Self-determined people (i.e., causal agents) act in service to freely chosen goals. Self-determined actions function to enable a person to be the causal agent in his or her life. (Shogren et al., 2015, p. 258)
Central to both definitions are the concepts of volition—purposeful activities and events, and causality—trying to develop and implement a plan that includes the hopes and dreams that individuals have for themselves.
The attitudes, knowledge, and skills to be self-determined are learned over the life span and need to be developed and nurtured over time (Wehmeyer & Field, 2007). Instruction, repeated opportunities, and appropriate support are essential for developing self-determination skills (Wehmeyer et al., 2012). A meta-analysis of interventions to promote self-determination for students with disabilities revealed that various interventions promoted self-determination in students across grade levels, disability labels, and settings (Burke et al., 2018). Although the effect sizes were positive, ranging from a small to a very large effect, the authors noted a need for increased rigor in the quality of research.
Purpose of the Study
Over the past 20 years, a variety of interventions including curricular programs and instructional models have been developed to promote self-determination. The National Secondary Transition Technical Assistance Center (NSTTAC) conducted a review of EBI in transition and established a moderate level of evidence for teaching skills related to self-determination and identified self-determination as one of the 16 evidence-based predicators of postschool employment, education, and independent living success (Test, Mazzotti, et al., 2009). Although systematic reviews have been conducted to examine literature on transition, the reviews were limited to studies published prior to May 2010 (Haber et al., 2016; Test, Fowler, et al., 2009). Burke and colleagues (2018) conducted a meta-analysis of interventions to promote self-determination for students with disabilities in which they studied the types of interventions, populations of participants, outcomes, and research rigor. Our study systematically reviewed literature published in 2007–2017 to examine current interventions focused on self-determination, specifically through the lens of identifying conceptual and direct replication studies. We decided to examine the self-determination intervention studies to answer the following questions adapted from Cook and colleagues (2016):
Method
Search Procedures for Self-Determination Intervention Studies
For the purpose of this investigation, we defined self-determination as . . . dispositional characteristics manifested as acting as the causal agent in one’s life. Self-determined people (i.e., causal agents) act in service to freely chosen goals. Self-determined actions function to enable a person to be the causal agent in his or her life. (Shogren et al., 2015, p. 258)
Before beginning the investigation, two national experts, who are at the forefront of self-determination research, were consulted for feedback on the identified research questions and the definition and attributes of self-determination. While there were no changes to the research questions, the list of search terms that would be used to identify studies that address the traits of self-determination was revised. Based on the experts’ feedback and a comprehensive literature review, the following keywords were used to identify self-determination studies: (a) self-determination, (b) choice making, (c) decision making, (d) problem solving, (e) goal setting, (f) goal attainment, (g) self-monitoring, (h) self-advocacy, (i) internal locus of control, (j) self-awareness, (k) self-knowledge, and (i) causal agency. The search procedure followed a three-step process (see Figure 1).
Search and coding procedures.
In Step 1, we conducted systematic searches of peer-reviewed publications in three electronic databases: ERIC, PsycINFO, and Education Source. The terms “disabilit* or exceptional *” were combined with the terms “study or investigation or research,” “student or child* or youth,” and “self-determination” or “choice making” or “decision making” or “problem solving” or “goal setting” or “goal attainment” or “self-monitoring” or “self-advocacy” or “internal locus of control” or “self-awareness” or “self-knowledge” or “causal agency” in the keywords field. This keyword search was limited to title and abstract only. This search yielded 702 articles from the three databases. We further screened the results to exclude articles that appeared in more than one database. This resulted in 468 unique articles published on the topic of self-determination between 2007 and 2017. The two reasons we chose to limit our search to the years 2007–2017 were to (a) capture the significant evolution of research on self-determination that has taken place in the past 10 years, and (b) build on other systematic reviews that had been conducted on self-determination studies up until 2007 (e.g., Cobb, Lehmann, Newman-Gonchar, & Alwell, 2009).
In Step 2, three researchers read the title and abstract of each of the 468 articles to identify articles that described intervention research on self-determination. Interventions were defined as “original, empirical research that investigated the effect or interaction of an independent variable that was implemented under the control of researchers on a specified outcome” (Cook et al., 2016, p. 225). Correlational and qualitative studies that described interventions or programs without investigating the impact or that only reported participants’ perceptions were excluded. This review process resulted in 102 articles that met the inclusion criteria for self-determination intervention studies.
During title and abstract review in Step 2, we noticed some studies used one of the 12 keywords for self-determination, but addressed the keyword in an academic area or did not investigate the concept of self-determination as defined for the purpose of this investigation. An example is the article titled “The Effects of Dynamic Strategic Math on English Language Learners’ Word Problem Solving.” Thus, in Step 3, a secondary review process was conducted on the initial 102 identified self-determination studies using the following procedures: First, intervention studies that included any of the keywords used in the search, but the specific focus of the study was on academic areas such as math or reading, rather than the components of self-determination, were excluded. An example is the study by Shin and Bryant (2017) titled “Improving the Fraction Word Problem Solving of Students With Mathematics Learning Disabilities: Interactive Computer Application.” Second, studies that addressed academic outcomes using an instructional strategy, such as self-monitoring, that could be generalized to other settings and impact behaviors associated with self-determination were included. One example of such a study was conducted by Soares, Vannest, and Harrison (2009) titled “Computer Aided Self-Monitoring to Increase Academic Production and Reduce Self-Injurious Behavior in a Child With Autism.”
This review yielded a total of 80 intervention studies. Interrater agreement was conducted on randomly assigned 20% of the reviewed interventions using a percentage of agreement estimates method. The calculation used for interrater agreement was (Agreements / [Agreements + Disagreements] × 100), which resulted in 94% agreement.
To answer RQ1, we additionally reviewed the methods in the 80 intervention studies to describe (a) design of the study (i.e., group, single case, mixed methods); (b) setting in which the study was conducted (i.e., general education classroom, special education classroom, resource room, or other); (c) disability or risk area (13 categories of IDEA or mixed categories/unspecified categories); (d) if the study was specifically about professional development for families or educators; (e) grade level studied (early childhood, elementary, middle school, high school, or mixed grades); (f) number of participants; (g) outcome area; and (h) the self-determination attributes addressed in the study.
Identification and Coding Procedures for Replication Studies
We took the following steps to determine how many of the 80 self-determination intervention studies were replications. Each step was recorded in the slightly modified coding sheet developed by Cook and colleagues (2016). The coding criteria included two initial questions: (a) Is there a replication statement in the Introduction or Method section that cites one or more studies or refers to study/studies/literature base/research base/research? and (b) Are findings from previous research compared with findings from current study in either the “Results” or “Discussion” section? Specific to RQ1, we added study characteristics described above. We had used this method and coding criteria for another published study, and thus were very familiar with this protocol.
We used the following broad, functional definition for replication provided by Cook and colleagues (2016): A replication statement indicates that the study purposefully replicates, extends, further investigates, or uses one or more previously conducted studies as its basis. Indicating that one or more gaps exist in one or more previous studies (e.g., not addressing a particular outcome area or learner population) and then noting in the same paragraph or in consecutive sentences in separate paragraphs that the present study addresses the gap(s) constitute a replication statement. The source for a replicated study must be a previously conducted study (i.e., a review or meta-analysis cannot be replicated). (p. 226)
Step a: Replication statement in Introduction or Method
To be identified as a replication study, the first step was to reference the past research in the Introduction or Method section and to include a replication statement in which the authors stated that the purpose of their study was to replicate or extend or further investigate previously conducted research. Following the procedures suggested by Cook and colleagues (2016), studies that noted the purpose was to extend previous research but did not cite those studies were not coded as replications. While this decision may have limited studies indicated as replications, we believed this process was necessary because without explicit connection and citation to the parent study, the scientific review would be compromised and may have led to confusion among researchers resulting in lower interrater agreement.
Step b: Comparison of findings in Results or Discussion
If a study was coded “yes” for a replication statement in Introduction or Method section, the second step was to analyze the Results and Discussion sections to see whether the findings of the current study (child study) were compared with the results of the original study(ies) that were cited in the Introduction or Method section. We used the following criterion suggested by Cook and colleagues (2016) as our guide for coding: Findings from previous research are considered to be compared to findings from the current study when the authors use language that indicates comparing or contrasting findings (e.g., as in, similar to, in contrast to) or when findings from previously conducted research and the current study are noted in the same or consecutive sentences (e.g., Author A (2000) reported positive effects of the intervention. In this study, we found no effects.). Noting findings from the current study and a previous study in different places (e.g., in different sections of the manuscript, in different paragraphs) does not constitute a comparison of findings. (p. 226)
Step c: Identification of replication study
A study was coded as a replication if it met both criteria explained in Steps a and b above. Furthermore, when coding intervention studies for replication statements, we reviewed all references in potential replication statements. A study was included as a “parent study” only if there was a clearly identifiable intervention with a dependent variable and results. Studies that met both criteria and referenced one or more “parent” study(ies) in the Introduction or Method section and Results or Discussion section were coded as replication studies. If the parent study(ies) referenced did not include at least one intervention study under each of the two criteria, the study was not coded as a replication. Also, the study was excluded if it only met one of the criteria. Interrater agreement was conducted on 20% of the replications using the same procedures previously discussed. Interrater agreement for replication identification was 88%.
Step d: Identification procedure for direct or conceptual replication studies
After a study was identified as a replication, the fourth step was to code the study as either a direct or conceptual replication. Studies that included statements of extending, further investigating, or adding to previous research were coded as conceptual replication. One example of such a statement was included in a study conducted by Valentini, Pierosan, Rudisill, and Hastie (2017): “The current study aims to extend the previous research by comparing two different intervention designs that have in common the autonomously engagement of children in intense levels of activity” (p. 351). Studies were coded as a direct replication if all aspects of the child study (study under review) were the same as the parent study (original study cited). These aspects included participants, setting, intervention, and outcome measures. Studies that differed in one or more aspects from the original study were coded as conceptual replications. Interrater agreement for coding of direct or conceptual studies was conducted on 20% of the studies and was 94%.
Step e: Identification procedure for research design, author overlap, and agreement in finding
The fifth step included coding the self-determination intervention studies for (a) research design, (b) author overlap, and (c) agreement in findings. For research design, the studies were coded as single-case, group, or other. A study was coded for author overlap when one or more of the authors of the child study also authored one or more of the parent studies. Interrater agreement for author overlap was 100%.
The last step was to determine agreement in findings between the child study and the parent studies. The four categories of agreement in findings as defined by Cook and colleagues (2016) included the following: (a) findings of parent studies are not explicitly compared with findings from the child study OR not clear whether comparison(s) indicate(s) agreement or disagreement between the parent and child studies, (b) findings of the parent studies are explicitly compared and noted only to agree, (c) findings of the parent studies are explicitly compared and noted only to disagree, and (d) findings of the parent studies are explicitly compared and noted to have mixed agreement/disagreement. The interrater agreement for concurrence in findings was 100%.
Results
Trends in Peer-Reviewed, Published, Self-Determination Intervention Research Between 2007 and 2017
To answer RQ1, we coded intervention studies that used one of the 12 terms frequently used in the literature for self-determination in their abstract or the title (i.e., self-determination, choice making, decision making, problem solving, goal setting, goal attainment, self-monitoring, self-advocacy, internal locus of control, self-awareness, self-knowledge, and causal agency). Eighty studies (out of 702) published over 11 years were identified and coded as intervention research. See Table 1 for a list of journals that published at least three or more intervention studies on the topic of self-determination during this time period. The majority of the studies were conducted in school settings. The remaining studies were conducted in home settings, clinical and hospital settings, juvenile center, or unspecified settings. Of those studies conducted in school settings, most studies were conducted in general education inclusive classrooms (n = 40, 62%), some in special education classrooms (n = 22, 34%), and a few in resource rooms (n = 3, 4%).
Journals That Published Three or More Intervention Studies on Self-Determination Between 2007 and 2017.
In terms of student characteristics, 4,151 children and youth participated in the studies with a range of one participant to 374 participants in a single study. Most studies reported that participants were in high school, followed by elementary school and middle school. In addition, we examined the data based on participants’ disability categories. We used the 13 disability categories described in IDEA (2004). See Figure 2 for the different disability categories represented in the intervention research on self-determination.
Study participants’ identified disability category.
Seven studies focused primarily on adult intervention such as outcomes related to professional development for families or educators to support self-determination in young children and youth. The targeted student outcomes included academics, self-advocacy, social communication/behavior, self-determination, vocational training or transition, or other miscellaneous. See Table 2 for the trends in published self-determination research.
Trends in Peer-Reviewed, Self-Determination Intervention Research Between 2007 and 2017.
Total percentage may not be exactly 100 due to rounding.
Of the 12 self-determination search terms that we used, the most reported attribute investigated was the general term “self-determination” (n = 24, 30%), rather than one of the traits of self-determination (e.g., choice making). See Table 3 for the different self-determination attributes investigated by the selected articles.
Self-Determination Attributes Addressed in the Studies.
Total is more than the total number of intervention or replication studies as some studies addressed more than one attribute.
Rate of Replication in Self-Determination Intervention Research
Eighty intervention studies on topics directly related to self-determination were coded to ascertain whether they were replication studies. Of the 80 intervention studies, 34 (42%) indicated that the purpose of the study was to replicate, expand, extend, add to, or use as their basis one or more previous intervention studies in the Introduction or Method section. Similarly, 34 of the intervention studies included a comparison of the findings of previously conducted research with the results of the current study in the Results or Discussion section. Of these studies, we identified 31 studies (39%) that included statements both in the Introduction or Method section and in the Results or Discussion section with at least one common parent study and thus met the criteria for replication studies. Four of these studies (13%) met the criteria for direct replication discussed above. The authors of the remaining studies (n = 27; 87%) did not make this claim and were therefore coded as conceptual replications.
Rate of Agreement in Findings
Of the 31 replication intervention studies, authors of 16 studies (52%) agreed with the findings of the parent studies. Authors of one study (3%) reported that their findings did not agree with that of the parent study, and authors of five studies (16%) reported that their findings both agreed and disagreed with the parent studies. Authors of the remaining studies (n = 9; 29%) did not explicitly compare the findings with the parent study, or their findings were not clearly presented to gauge whether they agreed or disagreed with the parent study.
Rate of Author Overlap
For the replication studies, we also calculated author overlap and the percentage of overlap between the “child” study and the “parent” study. Author overlap occurred for nine studies out of 31 replications (29%). Chi-square analysis for whether the intervention study was a replication or a nonreplication did not differ as a function of author overlap, χ2(1) = 4.64, p > .05. Furthermore, whether the agreements for studies differed as a function of author overlap also suggested a nonsignificant relationship, χ2(1) = 1.56, p >.05. Five of the nine studies with author overlap agreed with the previous results. The remaining four studies did not explicitly compare the findings, or it was not clear whether comparisons indicated agreement or disagreement between the parent and child studies. Researchers (e.g., Makel, Plucker, & Hegarty, 2012) caution that a high rate of author overlap between replication studies is problematic and should be interpreted with caution due to the possible inherent author bias.
Research Design
Most authors of the coded intervention studies reported using single-case designs (n = 41, 51%), 36 used group designs, and three studies used case study methods. Among the single-case designs, multiple baseline design was reported as the most frequently used design; pre-/posttest design was the most frequently reported group design. Of the 31 replication studies, a large proportion (n = 23, 74%) used single-case designs and 19% (n = 6) of the studies used group designs. Two studies used case study designs. Of the remaining 49 nonreplication studies, 35% (n = 17) used single-case designs and 65% (n = 32) used group designs. One nonreplication study used case study as the research design. Chi-square analysis for whether the agreements for studies differed as a function of study design was significant, χ2(1) = 12.69, p < .05. We did not include studies using “other” designs in this analysis because of the small n.
Discussion
Self-determination is a combination of attitudes, knowledge, and skills that enable individuals to engage in goal-directed, self-regulated, autonomous behavior. It develops over a life span when appropriate instruction and supports, along with repeated opportunities to learn and practice the behaviors of self-determination, are provided (Field, Martin, Miller, Ward, & Wehmeyer, 1998). Self-determination is considered an important factor in the transition from school to adult life (Test, Fowler, et al., 2009; Test, Mazzotti, et al., 2009).
As noted previously, replication is vital to cumulative science and the identification of EBI. Research must be repeated before the findings can be accepted as well established. As explained by Hunter and Schmidt (2004), Scientists have known for centuries that a single study will not resolve a major issue. Indeed, a small sample study will not even resolve a minor issue. Thus, the foundation of science is the culmination of knowledge from the results of many studies. (p. 13)
Given the importance of self-determination and replication research in building evidence, we conducted a review of the replication research focused on the topic of self-determination conducted over an 11-year period, ranging from 2007 to 2017. We identified 80 intervention studies on topics related to self-determination, of which 31 studies (39%) were identified as replications. This rate is slightly higher than that reported by Cook and colleagues (31%) in noncategorical special education research, but lower than the rate of replication reported by Banerjee, Movahedazarhouligh, Millen, and Luckner (2018) in early childhood special education intervention research (56%). Four studies were coded as direct replications and the remaining 27 studies were coded as conceptual replications. Author overlap occurred infrequently, and researchers predominately used both single-case and group designs. Finally, the majority of the studies indicated that the results of the replication study agreed with the findings of the parent studies. However, one study indicated that the findings did not agree with the parent study and five studies indicated that the findings both agreed and disagreed with the parent studies. The lack of concurrence between studies is an issue of concern for researchers and practitioners who seek to generalize the results across populations and settings.
A central tenet of EBI is that educators use the best current research to meet the needs of the students they serve. A challenge for many educators of students with disabilities is sorting through the research to determine whether the existent evidence can be generalized to the students with whom they work. The fact that some of the studies reported that the results only partially agreed (n = 9) or disagreed (n = 1) with the results of the parent study raises questions about whether or not the intervention outcomes of the parent study may be generalized to other students with different disabilities or in other settings. The caution is magnified when we examine the results of Figure 2. Specifically, there have been no intervention studies focused on self-determination for six IDEA categories of students and only one study conducted with students who have a speech and language impairment. In addition, Table 3 highlights the discrepancy in the self-determination attributes being studied by researchers. A large number of studies (30%) investigated self-determination as an unspecified dependent or independent variable. However, at least seven of the other 12 attributes reviewed had three or less studies examining the efficacy of that attribute of self-determination, suggesting that some components of self-determination have not been as well studied.
Limitations
Potential limitations of the study include the following: First, self-determination is an elusive concept (Shogren, 2013). The fact that we used 12 different terms to search the literature related to interventions focusing on self-determination suggests that the construct is composed of multiple, overlapping components. Similarly, as noted above, multiple definitions of self- determination exist. Our review indicates that researchers used a variety of definitions as well as assessment instruments. Consequently, it is likely that self-determination was measured differently using different instruments across studies and that some of the intervention studies reviewed may only tangentially directly relate to promoting self-determination. Second, there is no consensus about what exactly connotes a replication. We used the two-part definition of replication suggested by Cook and colleagues (2016). Other researchers have used a more rigid definition of replication (e.g., Lemons et al., 2016; Makel et al., 2016), and still others have used a more generous definition of replication (e.g., Therrien, Mathews, Hirsch, & Solis, 2016). These differences could result in dissimilar results. Third, although we completed a title and reference analysis of the parent studies to determine whether they were intervention studies, we did not conduct a thorough review to verify that the findings of the parent studies agreed or differed from what was reported in the child study.
Recommendations
As noted previously, there were no intervention studies undertaken with individuals from six of the IDEA categories. Research about how to promote the self-determination of individuals in each of these disability areas needs to be undertaken. Concomitantly, more research needs to be conducted on the component elements of self-determination (i.e., self-awareness, self-knowledge, choice making, decision making, goal setting, goal attainment, problem solving, self-regulation, self-management, and self-advocacy) with children and youth across different disability categories, to ascertain their relative importance in promoting self-determination. In addition, more research focusing on professional development needs to occur. For professionals to promote the self-determination of students with disabilities, they need to be knowledgeable and skilled at implementing EBI. Careful documentation of the accommodations and/or modifications to the interventions needs to be reported so that other researchers can undertake replications and practitioners can provide instruction and establish experiences geared to the strengths and areas of need specific to the student population they serve.
The overall lack of more direct replication studies published in special education on the topic of self-determination is concerning. Authors seldom explicitly state their intent to replicate due to some of the biases toward replication. Cook and colleagues (2016) observe that researchers often “look down upon” replication research and find them “difficult to publish” or “not prestigious” because journal editors and reviewers may think they do not provide a unique contribution; thus, authors may not explicitly report an investigation as a replication. Thus, we recommend that researchers, journal editors and reviewers, grant funders, and university policies intentionally and actively support a culture that promotes and values replication research using both single-case and group designs. For example, journal editors and reviewers will need to be more open to publishing replication studies when a clear goal of replication to build evidence for specific strategies to promote self-determination is provided. Similarly, authors must be “transparent” and “open” when reporting the origins of their research and data presented so that authors may easily replicate the methods with fidelity and ensure trustworthiness and efficiency (Cook, Lloyd, Mellor, Nosek, & Therrien, 2018) in self-determination research. Cook et al. recommend multiple open-science practices, such as preprints, data and materials sharing, preregistration of studies and analysis plans, and registered reports, to advance the quality of research and, consequently, policy and practice in special education. Future researchers should consider these reforms when developing research studies.
Conclusion
Current legislation and decades of research indicate that self-determination is an important factor for fostering positive adult outcomes for youth with disabilities. Consequently, it is imperative that educators, whenever possible, in collaboration with families and community agencies, provide instruction as well as learning opportunities that help students identify and communicate interests and preferences, establish and pursue goals, identify and solve problems, self-advocate, and access resources they will need as adults. Increasing the research base, including the undertaking of direct and conceptual replications, is an important step in improving educator effectiveness as well as the outcomes for young adults with disabilities.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
