The Impact of Teacher Supports for Implementing the Self-Determined Learning Model of Instruction on Student Outcomes in Inclusive General Education Classes

Abstract

People with disabilities consistently advocate for their right to be self-determined. Decades of research highlight the positive impacts of self-determination intervention on in-school and post-school outcomes of secondary students with disabilities. Increasingly, self-determination interventions are being implemented in inclusive contexts for all students, including students with and without disabilities. To enable implementation with high fidelity, further examination of the supports needed by teachers is required. In this study, the authors examine the impacts of differing intensities of teacher implementation supports (online modules only vs. online modules + coaching) on the initiation of the self-determined learning model of instruction (SDLMI) by schools and teachers and outcomes for students that arise from more intensive implementation when schools and teachers do initiate. Results suggest that intensifying teacher implementation supports has a positive impact on a cascading series of outcomes for schools, teachers, and students. In this study, the authors discuss the implications of the results for future research and practice.

Keywords

self-determination teacher supports goal attainment intervention research inclusive classrooms

One of the most consistently identified in-school predictors of positive in-school and post-school outcomes is self-determination (Burke et al., 2020; Mazzotti et al., 2021; Rowe et al., 2021), defined as students having the skills and supports to act as causal agents over their lives (Shogren & Raley, 2022). While early research focused on using self-determination interventions to support transition-age students with disabilities, researchers are increasingly highlighting the importance of tiered approaches to supporting self-determination and transition outcomes for all students with and without disabilities (Raley et al., 2022; Shogren et al., 2016). To provide universal instruction for self-determination in schools, schools must initiate as well as sustain effective supports for teachers.

Yet, there are multiple challenges to initiating and sustaining large-scale implementation of evidence-based self-determination interventions in inclusive education settings. These challenges include the need to (a) build administrative support, (b) plan for needed and effective teacher training and coaching, and (c) enable teachers to personalize interventions to students’ support needs, personal and family values, cultural identities, and post-school life goals (Shogren et al., 2021). All of this must be considered alongside competing demands for teacher and instructional time and the varying resources available in schools. These challenges necessitate a greater focus on studying different ways to support the initiation and implementation of evidence-based self-determination interventions to advance both uptake and sustained use to impact student outcomes. The intensification of implementation supports for teachers has the potential to encourage schools to initiate interventions by providing critical resources. The most effective intensities of supports to encourage the uptake of universal self-determination interventions in general education classes have not yet been robustly examined despite the potential importance of offering such supports to ensure initiation and sustained use of evidence-based self-determination interventions at scale (Hagiwara et al., 2020).

The self-determined learning model of instruction (SDLMI; Shogren & Raley, 2023) is an evidence-based practice (EBP) that enables teachers to flexibly support students in setting and attaining goals while building abilities and skills associated with self-determination. The SDLMI aligns with Causal Agency Theory (Shogren & Raley, 2022), a validated theoretical framework for the development of self-determination. The SDLMI can be overlaid on any content area (e.g., academic, social-emotional learning, and transition planning) with varying intensities of supports based on student needs. When the right implementation supports are in place, research has consistently shown teachers can implement the SDLMI with high fidelity across content areas, enhancing student outcomes (Hagiwara et al., 2017).

The Self-Determined Learning Model of Instruction

The SDLMI is a semester-long intervention that can be repeated over time. It consists of three phases: (a) set a goal, (b) take action, and (c) adjust goal or plan. The SDLMI enables students to identify and pursue goals while developing abilities, skills, and attitudes associated with self-determination, including choice-making, decision-making, goal-setting and attainment, problem-solving, planning, self-management, self-advocacy, self-awareness, and self-knowledge. The intended outcome of the SDLMI is not achieving a targeted goal, but instead, students learning the process of goal-setting, planning, and evaluating progress. By design, the SDLMI is a cyclical, structured process that students can use repeatedly throughout their academic careers and post-school life, identifying and taking steps toward new goals and building self-determination. Each SDLMI phase is driven by three core components: Student Questions, Teacher Objectives, and Educational Supports. Students answer four Student Questions per phase (12 total) to learn to self-regulate goal-setting, create and execute action plans, and evaluate progress. Teacher Objectives are linked to each Student Question and serve as a “roadmap” for teachers when supporting students to answer the Student Questions during SDLMI sessions. Furthermore, each phase includes Educational Supports (e.g., choice-making, decision-making, and goal attainment instruction) that teachers can utilize and intensify to meet Teacher Objectives based on students’ support needs (Burke et al., 2019; Raley et al., 2022).

SDLMI Coaching Model

To facilitate the initiation of the SDLMI by schools and to support teachers in their sustained implementation, standardized training and coaching procedures have been developed for teachers to enable them to use the SDLMI in inclusive general education classrooms. Researchers have documented the impacts of such training on teachers’ self-reported knowledge and skills post-training (Bojanek et al., 2021). There has been less focus on the impacts on ongoing supports for teachers as they are implementing the SDLMI. In fact, a recent literature review found high variability in the description of supports provided to teachers during SDLMI implementation, ranging from in-person coaching to regular emails to no reported supports beyond initial training (Kiblen et al., 2023). To address the lack of standardized procedures for SDLMI teacher implementation supports, the SDLMI coaching model was developed based on a comprehensive review of the coaching literature (Hagiwara et al., 2020). Like the SDLMI, the SDLMI coaching model is grounded in Causal Agency Theory, assuming goal-directed actions and self-determination are essential to teachers implementing the SDLMI with high fidelity.

There are six components of the SDLMI coaching model. The first component is clear principles (application, empowerment, equality, reflective dialogue, shared vision, and trust) that guide all supports coaches provide for teachers. The second component is four coaching stages: Plan, Observe, Reflect, and Share. The Plan stage involves coaches establishing a relationship with teachers, establishing times for coaching (at least one coaching visit during each phase of the SDLMI), reviewing the SDLMI coaching process, and building a shared vision and trust. The Observe stage involves coaches observing teachers’ delivery of the SDLMI, completing a fidelity observation to guide the next stage, Reflect. In the Reflect stage, coaches and teachers meet to review implementation and identify goals and action plans for ongoing enhancement of SDLMI delivery. The final stage, Share, involves follow-up communication, resource sharing, and circling back to planning for the next observation. To implement these stages, the third component of the SDLMI coaching model is the SDLMI coaching procedural checklist that coaches use to ensure the completion of all tasks associated with the SDLMI coaching model. The fourth component is the SDLMI fidelity measure (Shogren & Raley, 2018), which guides coaches in strategically observing how the SDLMI is implemented and structures feedback that coaches provide teachers. The fifth component is SDLMI coaching conversation notes, which document the coach–teacher relationship, how implementation was discussed during each vision, and shared goals for implementation and future coaching sessions. Finally, the coaching feedback survey is the sixth element of the SDLMI coaching model and allows teachers to provide anonymous feedback to improve future coaching practices and identify additional supports they may need.

While coaching is often recognized as a best practice to support the initiation and sustained implementation of EBPs (Snyder et al., 2015), there are other approaches to sharing implementation resources and supporting teachers (e.g., access to online materials and emails). Coaching is a more time- and resource-intensive approach to SDLMI implementation (Rifenbark et al., 2023). Research is needed to explore if the intensity of implementation supports (e.g., coaching vs. less intensive emails or access to online materials) impacts school and teacher initiation and fidelity of implementation of EBPs, like the SDLMI. For example, emerging research has shown that teachers report variability in their own fidelity of implementation of the SDLMI based on supports available (Shogren, Burke et al., 2020). Limited research has examined how available resources impact initiation of the SDLMI by schools and teachers, and implementation over time. We conducted this study to address this gap. We first (Research Question 1) examined the impacts of offering differing intensities of SDLMI teacher implementation supports in inclusive general education classes on the initiation of the SDLMI intervention by schools and second, we examined the impacts—when schools initiate—on student goal attainment and self-determination outcomes. We examined the following questions:

Research Question 1: Does varying the intensity of offered teacher implementation supports impact school decisions to initiate implementation of the SDLMI, after schools have signed a memorandum of understanding (MOU) to implement the SDLMI as part of a research project?

Research Question 2: If schools do initiate implementation of the SDLMI, does varying the intensity of teacher implementation supports impact the goal attainment outcomes of students with and without disabilities learning in inclusive classes?

Research Question 3: If schools do initiate implementation of the SDLMI, does varying the intensity of teacher implementation supports impact the self-determination outcomes of students with and without disabilities learning in inclusive classes?

Method

Setting and Sample

Fifteen high schools across three mid-Atlantic states signed MOUs to participate in a three-year cluster randomized control trial (C-RCT) investigating the impact of varying intensities of SDLMI teacher implementation supports on student outcomes. Simple random assignment was used to assign schools to one of two conditions: (a) online modules only or (b) online + coaching. For teachers in schools randomly assigned to the online modules only condition, teachers were emailed online modules to support SDLMI implementation once every two weeks via email after training. The modules provided additional instructional strategies, video examples, and materials to supplement implementation aligned with the three SDLMI phases. There was no additional interaction with the research team or an SDLMI coach. In the online + coaching condition, in addition to receiving the online modules, teachers received monthly, in-person coaching using the SDLMI coaching model.

Eight (53.3%) high schools were randomly assigned to the online condition and seven (46.7%) high schools were randomly assigned to the online + coaching condition. Of these 15 high schools, only eight (53.3%) initiated SDLMI implementation, which we defined as at least one teacher in the school completing training, delivering the SDLMI to students, and receiving teacher implementation supports for at least one year. Supplemental Table 1 provides school demographic information and implementation patterns across years. Ten (66.7%) of the 15 schools qualified for Title I services (five online schools and five online + coaching schools). Thus, the 15 schools were the focus of Research Question 1, on initiation of the SDLMI, and the eight schools that did initiate implementation were the focus of Research Questions 2 and 3. This allowed us to explore what impacted initiation, as well as how initiation then impacted student outcomes while maintaining an intent-to-treat approach to data analysis.

Across the eight schools initiating implementation (four online and four online + coaching), school administrators selected a total of 38 educators (19 online condition teachers vs. 19 online + coaching teachers) to participate. Consistent with Institutional Review Board (IRB) protocols, the selection of teacher and student participants was based on content area, school resources, priorities, and teacher consent and student assent. We recruited teachers of inclusive, general education core content classes that included students with and without disabilities in natural proportions. Specific co-teaching or collaboration models between general and special education were not included in the selection criteria. As such, there was a range in the engagement of special educators. When a collaborative model (e.g., co-teaching) was in place in a school, special and general educators were trained together as an instructional unit (see Table 1 for teacher demographics). Trained educators included 31 general and 7 special educators. General educators primarily identified as White/European American (85.7%) and female (78.6%), while 66.7% of special educators identified as Black/African American and half (50.0%) identified as female. Four (10.5%) educators withdrew after training and never initiated the SDLMI (two online teachers, two online + coaching teachers; 12.9% overall teacher attrition).

Table 1.

Teacher Demographic Information.

Characteristic	Overall		Group
	Overall		Online		Coaching
	N	(%)	N	(%)	N	(%)
Total sample	38	100	19	100	19	100
Teacher role	31	81.58	16	84.21	15	78.95
General education	31	81.58	16	84.21	15	78.95
Special education	7	18.42	3	15.79	4	21.05
Highest degree	18	47.37	10	52.63	8	42.11
Bachelor	18	47.37	10	52.63	8	42.11
Masters	12	31.58	2	10.53	10	52.63
Masters +	7	18.42	6	31.58	1	5.26
Missing	1	2.63	1	5.26	0	0.00
Gender identity	28	73.68	14	73.68	14	73.68
Female	28	73.68	14	73.68	14	73.68
Male	9	23.68	4	21.05	5	26.32
Missing	1	2.63	1	5.26	0	0.00
Hispanic	33	86.84	15	78.95	18	94.74
No	33	86.84	15	78.95	18	94.74
Yes	3	7.89	3	15.79	0	0.00
Missing	2	5.26	1	5.26	1	5.26
Race	6	15.79	3	15.79	3	15.79
Black/African American	6	15.79	3	15.79	3	15.79
White/European American	31	81.58	15	78.95	16	84.21
Missing	1	2.63	1	5.26	0	0.00
Subject area*
English language arts	16	43.0	8	44.0	8	42.0
Mathematics	5	14.0	0	0.00	5	26.0
Science	15	41.0	10	56.0	5	26.0
Electives	1	3.0	1	6.0	0	0.00
Grades taught
Grade 9	28	76.0	13	72.0	15	79.0
Grade 10	19	51.0	10	56.0	9	47.0
Grade 11	12	32.0	7	39.0	5	26.0
Grade 12	12	32.0	7	39.0	5	26.0
	M	SD	M	SD	M	SD
Teaching experience (in years)	10.7	7.0	9.4	7.8	11.9	6.2
Age (in years)	37.2	8.7	35.4	8.4	38.9	8.8

Teachers supported 3,214 students (1,631 students in online condition vs. 1,583 students in online + coaching), with 2,803 (87.2%) students contributing outcome data in at least one data wave on a primary measure. Among this subset of students, 98.7% (n = 2,766) contributed self-determination data and 78.3% (n = 2,195) contributed goal attainment data. As described in the data analysis plan, we adjusted inferences for student attrition, which was largely due to the COVID-19 pandemic in Year 2 and Year 3, with a tenfold increase in student attrition during the pandemic (see Supplemental Table 2 for additional details). The average age among the student sample was 15.0 (SD = 0.95) and the sample was split relatively evenly between students who identified as male (48.5%) and female (51.1%); 43.9% (n = 1,229) of students identified as Black and 11.4% (n = 320) as Hispanic. Approximately 37.5% (n = 1,051) of students qualified for free and reduced lunch, and 16.2% (n = 451) had an identified disability, with 10.0% (n = 280) of students having learning disabilities and 3.0% of students having other health disabilities (n = 85). See Table 2 for additional information, including demographics by conditions.

Table 2.

Student Demographic Information.

Demographic information	Overall		Group
	Overall		Online		Coaching
	N	%	N	%	N	%
Total sample	2,803	100.00	1,476	100.00	1,327	100.00
Gender identity
Female	1,431	51.05	735	49.80	696	52.45
Male	1,359	48.48	736	49.86	623	46.95
Missing	13	0.46	5	0.34	8	0.60
Race/ethnicity
Asian American	80	2.85	37	2.51	43	3.24
Black/African American	1,229	43.85	643	43.56	586	44.16
Hawaiian or Pacific Islander	4	0.14	4	0.27	0	0
Hispanic	320	11.42	153	10.37	167	12.58
Native American	13	0.46	8	0.54	5	0.38
White/European American	1,019	36.35	570	38.62	449	33.84
Multiracial	122	4.35	56	3.79	66	4.97
Missing	16	0.57	5	0.34	11	0.83
Multilingual learner	2,584	92.19	1,378	93.36	1,206	90.88
No	2,584	92.19	1,378	93.36	1,206	90.88
Yes	106	3.78	56	3.79	50	3.77
Missing	113	4.03	42	2.85	71	5.35
Free and reduced lunch status	1,572	56.08	735	49.80	837	63.07
No	1,572	56.08	735	49.80	837	63.07
Yes	1,051	37.50	661	44.78	390	29.39
Missing	180	6.42	80	5.42	100	7.54
Disability status	2,331	83.16	1,167	79.07	1,164	87.72
No
Yes	455	16.23	305	20.66	150	11.30
Missing	17	0.61	4	0.27	13	0.98
Disability label
Autism	21	0.75	16	1.08	5	0.38
Emotional disability	28	1.00	24	1.63	4	0.30
Hearing loss or deafness	4	0.14	2	0.14	2	0.15
Intellectual disability	15	0.54	9	0.61	6	0.45
Learning disabilities	280	9.99	190	12.87	90	6.78
Multiple disabilities	2	0.07	2	0.14	0	0
Other health disability	85	3.03	49	3.32	36	2.71
Physical disability	2	0.07	2	0.14	0	0
Speech/language disability	8	0.29	7	0.47	1	0.08
Traumatic brain injury	3	0.11	1	0.07	2	0.15
Vision loss or blindness	1	0.04	0	0	1	0.08
Other	2	0.07	1	0.07	1	0.08
N/A (no disability)	2,331	83.16	1,167	79.07	1,164	87.72
Missing	21	0.75	6	0.41	15	1.13
Age M, SD	15.0	0.95	14.9	0.91	15.2	0.97

Note. Demographic information was collected from administrative records.

Study Context and Timeline

In Year 1 (2018-2019), a cohort of Grade 9 teachers was recruited to initiate class-wide SDLMI implementation in inclusive core content classes (i.e., English Language Arts, Mathematics, Science). Outcome data on student goal attainment (fall, spring) and self-determination (fall, winter, spring) were collected each year for all students with and without disabilities. In Year 2 (2019-2020), a new cohort of Grade 10 teachers in the same schools were trained to continue implementation with students, while the Grade 9 teachers initiated the SDLMI with the next cohort of Grade 9 students. In Year 3 (2020-2021), a new cohort of Grade 11 teachers in the same schools were added. The original C-RCT design called for all schools to be recruited during the first year of the project. Despite expressed interest and signing MOUs during the recruitment year that preceded implementation, several teams removed their schools from the C-RCT prior to implementation. Research Question 1 seeks to understand the reasons why schools did not initiate, after committing to participate in the C-RCT and receive training and implementation supports. To compensate for these challenges, school enrollment was extended into subsequent project years and research questions were adjusted to focus on outcomes during the first full year of exposure to SDLMI implementation to eliminate the potential of unplanned school attrition in subsequent years biasing results. The COVID-19 pandemic, which started in Spring 2020, also impacted trial implementation, hindering student and teacher data collection as well as school recruitment and retention. As such, the research team pivoted to supporting virtual and then hybrid instruction during the last year and a half of the project and did not recruit new schools during the last year of the project.

Procedures

Teachers participated in a standardized SDLMI professional development training in the summer before each project year. The training was consistent across trial conditions. Teachers who participated for more than one project year (n = 13) attended training each year. The SDLMI professional development training aligns with key indicators of high-quality professional development and has been shown to impact teachers’ reported knowledge and skills of self-determination instruction. For example, to address the active quality indicator (Darling-Hammond et al., 2009), SDLMI trainers integrate hands-on activities, such as small-group discussions and role-playing, to promote SDLMI fidelity. Teachers, with support from SDLMI trainers, also developed SDLMI implementation schedules that incorporated delivery of two weekly SDLMI lessons (i.e., 15-minute instructional sessions) aligned with their class content and priorities as well as other district-wide initiatives to ensure their students cycled through the three SDLMI phases twice an academic year, once per semester. As the SDLMI is intended to be an individualized process, SDLMI trainers provided teachers with frequent opportunities to reflect on potential modifications to implementation based on their students’ support needs and cultural identities, as well as the content area being targeted. Teachers were empowered to modify the SDLMI lesson materials as necessary, so long as they adhered to implementation standards for the three core components.

The Year 1 (Summer 2018) training was in-person over two days and the Year 2 (Summer 2019) training was in-person over three days to allow for more sharing of implementation examples by Grade 9 teachers who had implemented the previous year. Because of the COVID-19 pandemic, Year 3 (Summer 2020 and Winter 2020) professional development was provided virtually via Zoom, across three days to stay consistent with previous years’ training procedures. During Year 3, two professional development opportunities were provided to accommodate schools and teachers as two schools deferred implementation during Fall 2020 and resumed implementation during Spring 2021 because of the pandemic. To address quality indicators using online modalities, the virtual training design included videos, online discussion boards, and case studies of different implementation scenarios. To support teachers during COVID-19, SDLMI trainers created online versions of all materials in Google Classroom and Schoology, two online learning management systems used by teachers.

Fidelity

Across both conditions (described subsequently), fidelity data was collected by external observers three times a semester once during each SDLMI phase, resulting in six observations per academic year. Observations were conducted in-person or virtually after the onset of COVID-19 using the SDLMI fidelity measure (Shogren & Raley, 2018). As reported in other sources, almost all teachers (93%) initially met standards for sufficient adherence to the core SDLMI components. Smaller numbers initially met standards for sufficient quality of delivery (64%) and student responsiveness (69%), but this number grew over time with more intensive implementation supports (i.e., online + coaching) predicting growth (Shogren et al., 2024).

Teacher Implementation Support Conditions

As described previously, in the online-only condition, teachers were emailed online modules every two weeks to support SDLMI implementation. While all teachers opened the emailed resources, there was limited duration of engagement (e.g., less than 5 minutes) and interaction (e.g., click-throughs to all resources) with the online resources over the life of the project. In the online + coaching condition, in addition to receiving the online modules, teachers also received coaching using the SDLMI coaching model. Coaches had previous experience as teachers, administrators, and/or instructional coaches and completed a standardized two-day training (Hagiwara et al., 2020). An area of emphasis during the SDLMI coach training was how to support teachers in proactively modifying their SDLMI implementation to address the needs of students with disabilities as well as students from diverse racial and ethnic backgrounds, consistent with SDLMI coaching model protocols (Hagiwara et al., 2020). Coaching sessions involved 30 minutes. SDLMI lesson observation using the procedures learned during training and a 30-minute coaching conversation in which coaches prompted teachers to reflect on their implementation and provided feedback and resources to address teachers’ implementation needs. Every coaching session ended with coaches and teachers collaboratively setting a goal and action plan for implementation prior to the next coaching session. Prior to the onset of the COVID-19 pandemic, all coaching sessions were in-person, however, after the onset of the pandemic, all coaching sessions were virtual.

Measures

Goal Attainment Scaling

Goal attainment scaling (GAS; Kiresuk et al., 1994) is a measure of the attainment of individualized goals. GAS has been frequently used in education and disability research to establish a range of personalized and differentiated levels of goal attainment, using standardized procedures that allow for comparable ratings across different types of goals. GAS rubrics include five levels of possible attainment: much less than expected (−2), somewhat less than expected (−1), expected outcome (0), somewhat more than expected (1), and much more than expected (2). These levels of goal attainment are directly linked to the goal and reasonable expectations of attainment within a specific timeframe, in the case of this project, an academic semester. Examples of student goals that were used to establish GAS rubrics included: “My goal is to get a good course grade in math this semester,” “My goal is to turn in my science homework on time this semester,” and “My goal is ace my report card this semester.”

Researchers have developed specific procedures to promote the reliability and validity of GAS ratings (Krasny-Pacini et al., 2016; Shogren, et al., 2021) that were used in this study. Specifically, students set their own GAS goals and developed levels of possible attainment after receiving instruction integrated into SDLMI instruction. After SDLMI Phase 1, students entered their goal and GAS rubric in a customized online platform with support from teachers. Approximately eight weeks after establishing their GAS goals and rubric, students completed SDLMI Phase 3 focused on self-evaluating goal attainment and logged their goal attainment outcomes using the same online platform. When rating goal attainment, students were first asked if they decided not to complete their goal for any reason. If they responded “yes,” their goal attainment rating was coded as not completed, irrespective of the reason. If students reported they completed their goal, they self-rated attainment on the GAS rubric (−2 to +2). GAS data were collected once each academic semester (twice a year).

Self-Determination Inventory: Student Report

The self-determination inventory: student report (SDI: SR; Shogren & Wehmeyer, 2017) is a standardized self-report measure of self-determination that has been validated with adolescents ages 13 to 22 with and without disabilities. The SDI: SR aligns with Causal Agency Theory, and research has suggested that a single general self-determination factor fits SDI: SR data (Shogren et al., 2020). During the development of the SDI: SR, data were collected from over 4,000 adolescents, including students from racially and ethnically marginalized backgrounds with a variety of disability labels, and SDI: SR scores varied in expected ways based on race/ethnicity and disability labels and had satisfactory internal consistency (Shogren et al., 2018). The SDI: SR consists of 21 slider-scaled items (computer score to range from 0 to 99). The SDI: SR is administered online, enabling precision in scoring and integration of accessibility features (e.g., audio playback, in-text definitions). Students completed the SDI: SR at baseline (Fall) prior to SDLMI implementation, after the first cycle through the SDLMI (mid-year; Winter), and at the end of the second cycle and academic year (Spring).

Data Analysis

Multilevel Hurdle Model

We conducted analyses to assess whether intensified SDLMI teacher implementation supports (i.e., online vs. online + coaching) offered through the study design impacted an unfolding series of outcomes, including school initiation of the SDLMI (Research Question 1) and student outcomes when teachers implemented the SDLMI (Research Question 2). All scripts and data needed to replicate these analyses can be accessed on the Open Science Foundation (OSF) at https://osf.io/m2djw/?view_only=4bdb35ce08214064abbb50f682cbe371.

We hypothesized at the start of this trial that offering intensified implementation supports would impact the decision of schools to initiate SDLMI and that this was a substantive question of interest as understanding attrition and developing strategies to increase the uptake of EPBs is an important area of research. Essentially, after signing MOUs committing to participation in the project, because of the C-RCT design, schools were randomly assigned and offered online only or online + coaching accordingly. After random assignment, schools and the teachers then had to take additional steps to initiate and sustain implementation.

To model this hypothesized two-stage process, where we hypothesized that offering more intensive implementation support would increase SDLMI initiation rates and, contingent on the decision to initiate SDLMI, student outcomes would subsequently improve, we chose hurdle modeling analysis. This modeling strategy allows the school’s decision to initiate to be recognized as a necessary condition for subsequent enhanced student self-determination and goal attainment outcomes. It also allows us to first examine factors that impact initiation in the larger sample (Research Question 1) and then examine outcomes in the sample that did initiate based on the intensity of implementation supports (Research Questions 2 and 3). We also incorporated multilevel modeling (MLM) into these hurdle modeling analyses because student outcome data were derived from sets of nested units with measurement occasions (Level-1) nested in students (Level-2), students nested in educators (Level-3), and educators nested in schools (Level-4). MLM corrects hurdle modeling analyses for data dependencies in hierarchically structured data (Gelman & Hill, 2006).

Bayesian Model Averaging Analysis

We selected Bayesian model averaging (BMA; Raftery et al., 1997) analysis to evaluate models instead of null hypothesis significance testing (NHST), given the sample of 15 schools available for the first research question. NHST aims to reject the null model in a single decision, which can be challenging with smaller sample sizes due to power concerns. In contrast, BMA analyses can compare probabilities of different models without rejecting any of them, while continuously updating probabilities as more data is collected. Even if the null model is not rejected in NHST, BMA analysis can still clarify if its relative probability is greater next to an alternative. BMA analysis eliminates the assumed need in NHST to categorically reject the null (Howson & Urbach, 2006), with the uncertainty of smaller sample sizes reflected in the results.

Computation Methods for BMA Analysis

The technical details of Bayesian computation, including the Markov chain Monte Carlo (MCMC) simulation and posterior predictive modeling checks (PPMC), can be accessed on OSF. We conducted the BMA analysis in three steps. First, we used Bayesian methods with diffuse priors to estimate models. We employed the MCMC simulation procedure in SAS 9.4 software (PROC MCMC; SAS Institute, 2015). We separately modeled components of each stage in the hurdle model within a generalized linear mixed modeling (GLMM) framework to accommodate differently scaled outcomes (e.g., binary, ordinal, metric). We checked the accuracy of our model specifications using posterior predictive model checks. Second, we used the deviance information criterion (DIC; Ando, 2007) to assess the fit of candidate models. The DIC value (smaller is better) quantifies the fit by balancing the benefits of model simplicity and explanatory power. Third, we converted DIC values into model weights, which indicate the probability that each model has the most predictive power. Thus, using BMA analysis, we were able to retain all available models without rejecting any and synthesize their predictions when drawing inferences based on the corresponding weight of each model.

Application of BMA Analysis to Research Questions

We applied BMA analysis to each research question. To address our first question (Does varying the intensity of offered teacher implementation supports impact school decisions to initiate implementation of the SDLMI, after schools have signed MOUs to implement the SDLMI as part of a research project?), we used Bayesian logistic analysis, regressing implementation outcomes (initiated vs. uninitiated) on teacher implementation support (online vs. online + coaching). We used BMA analysis to compare potential covariates: $(M_{0})$ null; $(M_{1})$ group only; $(M_{2})$ Title I status; and $(M_{3})$ group-by-Title I status. In BMA analysis, the sum of weights of all models with the same covariate denotes the probability that this covariate belongs in the most explanatory model. We used the odds ratio (OR) as our metric of effect size, with boundaries of 1.68, 3.47, and 6.71 denoting small, medium, and large effect sizes, respectively. Although it is well known that it is challenging to obtain the exact value of a true effect size with small sample sizes, we used Bayesian estimation to calculate the probability the true effect is indeed in the hypothesized direction and within a certain effect size range rather than interpret an exact estimate. Taking advantage of Bayesian estimation, we examined the full posterior of effect size parameters as opposed to only using point and interval estimates. With 15 schools, point and interval estimates of effect sizes are commonly recognized as being too noisy and imprecise to be informative unless more data is collected. Bayesian estimation still allows us to learn from the current sample data, though, by using the full posterior distribution to tentatively determine the relative probability that the true effect size is in the expected direction and effect size ranges.

To address our second research question (If schools initiate the SDLMI, does varying the intensity of teacher implementation supports impact the goal attainment outcomes of students with and without disabilities learning in inclusive classes?), we performed four-level categorical logistic regression analysis (occasions [Level-1] nested in students [Level-2] nested in teachers [Level-3] nested in schools [Level-4]), regressing goal attainment outcomes on the group. Goal attainment outcomes categories were: (a) did not continue working on goal (i.e., abandoned goal), (b) attained goal below expectations (rating of −2 or −1), and (c) attained goal at or above expectations (rating of 0 or above). We used BMA analysis to compare different configurations of covariates in the model: $(M_{0})$ null; $(M_{1})$ group only on goal attainment ratings; and $(M_{2})$ group on both goal attainment ratings and goal abandonment. Like the first research question, we also selected the OR as our measure of effect size for this question. Despite some teacher and student attrition, we used full information Bayesian estimation with diffuse priors, which coincides with full information maximum likelihood, to compensate for attrition.

To address our third research question (If schools initiate the SDLMI, does varying the intensity of teacher implementation supports impact self-determination outcomes of students with and without disabilities learning in inclusive classes?), we performed four-level beta regression analysis (occasions [Level-1] nested in students [Level-2] nested in teachers [Level-3] nested in schools [Level-4]), regressing self-determination outcomes on the group. Beta regression analysis is suitable for slider-scale data with hard boundaries in the distribution’s tails. Notably, we rescaled SDI: SR scores to fall between 0 and 1, with the small percentage of resulting 0 and 1 values (which are inadmissible in beta regression analysis) changed to 0.001 and 0.999, respectively. In addition, taking advantage of BMA analysis, we pooled predictions across separate models to predict the effect size, using Cohen’s d as our effect size metric. 0.3, 0.5, and 0.8 mark the borders of small, medium, and large effect sizes, respectively. For both goal attainment and self-determination outcomes, we controlled for COVID-19 pandemic status (pre-pandemic onset vs. post-pandemic onset) as an occasion-level covariate and disability status (no disability reported or self-reported disability) as a student-level covariate in all models, which necessitated discarding the 17 (<1%) students who had missing disability information.

Results

Research Question 1: School Initiation Outcomes

Table 3 presents the results of BMA analysis examining the impact of two covariates, Title I status (yes vs. no) and implement support group (online vs. online + coaching), on school initiation of the SDLMI. This BMA analysis indicated that, while the null model cannot be rejected at this time, there is a 79.2% probability that including at least one of these covariates contributes explanatory power. $M_{3}$ has the greatest probability (≈ 40.0%) of being the most explanatory model. This model predicts the direction of the impact of group assignment depending on Title I status. Supplemental Figure 1 presents a visualization of the posterior effect sizes. The model assigns an 83.5% probability that offering online + coaching improved the odds that Title I schools initiate the SDLMI compared to offering only online support, with the most probable effect size resting in the medium range. In contrast, for non-Title I schools, there is a 78.4% probability that offering online + coaching supports has a negative impact on the odds of initiating SDLMI as opposed to offering only online support. Although more data need to be collected to pinpoint the exact effect size with precision, these preliminary results provide initial evidence that offering online + coaching supports may incentivize participation for Title I schools while it may be potentially counterproductive for non-Title I schools.

Table 3.

BMA Analysis of Factors Predicting SDLMI Initiation.

Parameter	$M_{0}$		$M_{1}$		$M_{2}$		$M_{3}$
Parameter	Est.	SE	Est.	SE	Est.	SE	Est.	SE
Intercept	0.122	0.499	0.044	0.608	0.664	0.719	1.291	1.393
Online + coaching group status			0.209	0.838			−0.920	1.170
Title I					−0.902	0.843	−1.691	1.081
Online + group × Title I status							1.881	1.531
Dbar (posterior mean of deviance)	21.632		22.197		20.458		19.045
Dmean (deviance evaluated at posterior mean)	20.728		20.657		19.009		16.832
pD (effective number of parameters)	0.904		1.540		1.449		2.213
DIC (smaller is better)	22.537		23.738		21.906		21.258
Model weight	0.208		0.114		0.285		0.394

Note. BMA results compared the explanatory power of different combinations of covariates in a model of school outcomes (initiated SDLMI vs. not initiated SDLMI). We evaluated model fit with DIC analysis. Model weights, which are based on DIC values, denote the probability the model is the most explanatory. In this case, at almost 40.0% probability, $M_{3}$ has the highest probability, which suggests offering more intensive supports could increase the probability that Title I schools will implement SDLMI. More data needs to be collected to be certain of this interpretation. SE = standard error; DIC = deviance information criterion.

Research Question 2: Goal Attainment Outcomes

Table 4 presents the results of the BMA analysis of goal attainment outcomes for students in schools that initiated the SDLMI. Although it would be premature to reject the null, BMA analysis assigns over a 99.0% probability that group differences (online vs. online + coaching) belong in the most explanatory model. In this case, $M_{1}$ , which has group differences on only goal attainment outcomes, had the highest probability (91.4%). Interestingly, this model includes an interaction between semester (Fall vs. Spring) and group (online vs. online + coaching). In the first semester, this model predicts that the online + coaching condition leads to a very small increase in the odds of positive goal attainment (OR = 1.264, equivalent to a 26.4% increase in odds), whereas in the second semester, the model predicts that online + coaching leads to a very small decrease in the odds of positive goal attainment (OR = 0.778, equivalent to a 22.2% decrease in odds). In addition, although it would be premature to draw any strong inferences given the uncertainty of estimation, it is noteworthy that $M_{1}$ implies the odds students will abandon goals were slightly higher after the onset of the COVID-19 pandemic and virtual delivery (OR = 1.349, equivalent to a 34.9% increase in odds), while at the same time, the odds students attained goals at expected levels also became slightly higher after the onset of the COVID-19 pandemic and virtual delivery (OR = 1.416, equivalent to a 41.6% increase in odds). Also, compared to their peers without disabilities, students with disabilities had slightly lower odds of abandoning goals instead of not achieving at expected levels (OR = 0.720, equivalent to a 28% decrease in odds) and slightly lower odds of attaining goals at expected levels compared to not attaining at expected levels (OR = 0.790, equivalent to a 21% decrease in odds). We need to collect additional data, but one implication of both is that abandoning goals could be an effective strategy to increase the odds of goal attainment at expected or greater levels (compared to lower-than-expected levels). After the onset of the COVID-19 pandemic, students without disabilities increased the abandonment of goals, while students with disabilities did not and had greater odds of expected or greater than expected goal attainment.

Table 4.

BMA Analysis of Factors Predicting Goal Attainment Outcomes.

Parameter	$M_{0}$		$M_{1}$		$M_{2}$
Parameter	Est.	SE	Est.	SE	Est.	SE
Goal attainment
Fixed effects
Intercept	0.932	0.148	0.796	0.195	0.834	0.232
Post-pandemic onset	0.071	0.187	0.299	0.203	0.308	0.194
Disability (Yes)	−0.334	0.137	−0.329	0.134	−0.335	0.138
Online + coaching group			0.264	0.287	0.156	0.332
Spring semester	0.016	0.031	0.265	0.125	0.254	0.129
Online + coaching × Spring semester			−0.795	0.236	−0.781	0.236
Random effects
Student effects (variance)	0.636	0.195	0.733	0.163	0.629	0.165
School effects (variance)	0.116	0.092	0.141	0.065	0.145	0.095
Goal abandonment
Fixed effects
Intercept	−1.540	0.182	−1.593	0.144	−1.508	0.227
Post-pandemic onset	0.344	0.270	0.348	0.267	0.547	0.282
Disability (yes)	−0.245	0.228	−0.240	0.234	−0.239	0.234
Online + coaching					−0.157	0.298
Spring semester	−0.058	3.071	2.335	2.212	−0.114	0.215
Online + coaching × Spring semester					−0.139	0.372
Random effects
Student effects (variance)	1.305	0.381	1.442	0.240	1.458	0.299
School effects (variance)	0.064	0.073	0.044	0.068	0.079	0.087
Model fit
Dbar (posterior mean of deviance)	4,380.319		4,302.422		4,342.580
Dmean (deviance at posterior mean)	3,854.379		3,722.708		3,797.411
pD (effective number of parameters)	525.940		579.714		545.169
DIC (smaller is better)	4,906.259		4,882.136		4,887.749
Model weight	0.000		0.943		0.057

Note. BMA results compare the explanatory power of different combinations of covariates predicting goal outcomes (not attaining goals at expected levels, attaining goals at expected or greater levels, and goal abandonment). Not attaining goals at expected levels served as the reference category. The fit of each model was evaluated with DIC analysis. The model weights, based on DIC values, denote the probability that the model is the most explanatory. In this case, at over 90%, M₂ has the highest probability. SE = standard error; DIC = deviance information criterion.

Research Question 3: Self-Determination Outcomes

Table 5 presents BMA analysis results for self-determination outcomes. The BMA analysis found over a 40% probability that $M_{4}$ is the most explanatory model. Based on the model weights, it would be premature to accept or reject any model. Therefore, rather than selecting only one of these models as the final model and discarding the rest as incorrect, we instead synthesized the divergent predictions of these separate models based on model weight to generate one unified prediction of the impact of intensifying teacher implementation supports on student self-determination outcomes that take into account uncertainty of the correct covariates. As shown in Figure 1, BMA analysis predicts that intensifying teacher implementation support in the online + coaching condition will lead to a slight (albeit, steady) acceleration in growth in self-determination outcomes over time, without short-lived drips in self-determination as occurring in the online condition. Although pandemic and disability status remained in the model as covariates to be true to C-RCT design, the estimates of their impacts on self-determination outcomes are practically equivalent to the null regardless of the model, tentatively suggesting these two factors only have negligible impacts on self-determination outcomes. Future research should corroborate these covariates findings before strong inferences are drawn.

Table 5.

BMA Analysis of Factors Predicting Self-Determination Inventory Data.

Parameter	M ₀		M ₁		M ₂		M ₃		M ₄
Parameter	Est.	SE	Est.	SE	Est.	SE	Est.	SE	Est.	SE
Fixed effects
Intercept	1.419	0.056	1.426	0.051	1.387	0.081	1.389	0.078	1.408	0.092
Disability (yes)	−0.000	0.047	−0.005	0.047	0.001	0.046	−0.006	0.049	−0.010	0.050
Post-pandemic onset	−0.025	0.027	0.018	0.036	−0.026	0.027	0.003	0.037	−0.006	0.040
Mid-year (Winter)			−0.048	0.029			−0.069	0.032	−0.076	0.031
End of year (Spring)			0.020	0.034			0.025	0.036	0.012	0.032
Online + coaching group					0.052	0.132	0.061	0.109	0.027	0.110
Online + coaching × mid-year (Winter)							0.074	0.045	0.089	0.040
Online + coaching × end of year (Spring)							−0.061	0.081
Random effects
Occasion effects (variance)	0.116	0.013	0.116	0.012	0.117	0.012	0.116	0.013	0.116	0.012
Student effects (variance)	0.516	0.023	0.516	0.023	0.517	0.023	0.516	0.023	0.517	0.024
Teacher effects (variance)	0.012	0.007	0.012	0.007	0.012	0.007	0.012	0.008	0.013	0.009
School effects (variance)	0.014	0.014	0.012	0.013	0.016	0.016	0.016	0.015	0.014	0.015
Model fit
Dbar (posterior mean of deviance)	−10,307.6		−10,313.1		−10,308.5		−10,314.0		−10,316.1
Dmean (deviance at posterior mean)	−12,167.2		−12,173.6		−12,168.0		−12,175.9		−12,178.8
pD (effective number of parameters)	1,859.586		1,860.490		1,859.520		1,861.861		1,862.787
DIC (smaller is better)	−8,448.01		−8,452.63		−8,448.95		−8,452.18		−8,453.27
Model weight	0.029		0.291		0.046		0.233		0.401

Note. BMA results compare the explanatory power of different combinations of covariates predicting self-determination outcomes. We evaluated model fit with DIC analysis. Model weights, based on DIC values, denote the probability the model is the most explanatory. In this case, at over 40%, M₄ has the highest probability, which is consistent with the hypothesis that more intensive implementation supports (online + coaching) would enhance student self-determination outcomes. SE = standard error; DIC = deviance information criterion.

Figure 1.

BMA Predictions of Group Effects for Self-Determination Outcomes

Discussion

As explained in the introduction, we conducted this study to examine the impacts of offering different intensities of SDLMI teacher implementation supports in the context of a C-RCT on SDLMI initiation in schools, and subsequently in cases when schools did initiate, the impact of intensifying supports on student goal attainment and self-determination outcomes in general education classrooms. The findings highlight key considerations for advancing school and teacher capacity to sustain and bring SDLMI implementation to scale (Cook & Odom, 2013). Results provide support for our hypotheses that offering intensifying teacher implementation support (in this case, the possibility of having coaching + online supports as a result of random assignment after agreeing to participate in a C-RCT) has a positive impact on a cascading series of outcomes for schools, teachers, and students (relative to online supports only). This work builds on a line of research confirming students can benefit from self-determination interventions, and it is important to provide practical, effective approaches that empower schools and educators to teach all students self-determination abilities and skills within the general education classroom. It additionally advances our understanding of factors that may impact decisions to scale-up on EBPs (like the SDLMI) in practice. For example, having the opportunity for more intensive implementation supports appears to impact the initiation of an EBP and subsequently, if initiated, student outcomes over time. In short, the field needs information on how best to empower schools and educators within schools to commit to initiating evidence-based self-determination interventions at scale with the knowledge, confidence, and skills necessary to meet this charge (Hagiwara et al., 2020). This is particularly important as we continue to navigate the impacts of the COVID-19 pandemic on learning and supporting teachers and schools to overcome the new hurdles to implementing new practices (Toste et al., 2021). In recent years, researchers and practitioners alike have committed to exploring effective professional learning to facilitate initiation and sustained implementation efforts (Bojanek et al., 2021; Buckman et al., 2021; Common et al., 2021). Findings from the current study hold promise for informing implementation supports not only for the uptake and implementation of SDLMI but also for the implementation of a range of complex interventions. In the following section, we detail key educational implications gleaned from this complex longitudinal study

Implications for Future Research and Practice

School Initiation of Self-Determination Interventions

As described in the literature on implementation science, closing gaps between intervention research and real-world practices is challenging (Cook & Odom, 2013; Stahmer et al., 2018). In this C-RCT, we recruited 15 schools that committed to initiating SDLMI and signed detailed MOUs that provided clear expectations for schools, teachers, and the university partner. Only eight school administrative teams (52%) ultimately initiated the SDLMI in their schools, and Title I status predicted initiation outcomes. While there is likely a range of factors that impact initiation, it is important to note that the finding that Title 1 status predicts initiation provides direction for ongoing research that informs both research design as well as an understanding of how school-level resources impact the initiation of EBPs. For example, anecdotal data from field notes indicate that, regardless of group, the typical reason schools decided against initiating implementation after signing MOUs was concerns that taking up a new intervention would be too overwhelming for them at that moment. But only in Title I schools were they more likely to initiate the SDLMI if they were randomly assigned to the online + coaching group. This suggests that the opportunity for more intensive teacher supports may lead to greater initiation in financially under-resourced schools. While more work is needed to examine the replicability of this finding in new samples and further explore the range of factors that may impact initiation, this is a first step in exploring this issue directly rather than simply reporting on attrition of schools and further considering how school resources impact participation in research studies and in the implementation of EBPs. We encourage other researchers to repeat these types of analysis, using materials and resources provided as part of our commitment to open science practices, to test the generalizability of findings.

The findings also highlight important issues to consider in the implementation of C-RCTs in school contexts. Researchers have described the difficulties of such research, particularly when including students with disabilities (Odom, 2021). There is limited research on the impact of school-level factors on the initiation of EBPs and participation in C-RCTs. This is unfortunate as researchers can treat the failure to initiate an EBP in implementation as a natural learning opportunity to understand factors that contribute to actual initiation in authentic settings, following an implementation science framework (Odom et al., 2020). In this C-RCT, a two-stage hurdle analysis, wherein SDLMI initiation outcomes became the first outcome in a sequence of outcomes of substantive interest, allowed us to examine the impact of varying intensities of teacher implementation support offered as part of the C-RCT on school initiation as well as subsequent student outcomes. This same methodology could be of interest to other intervention researchers focused on inclusive contexts who face limited initiation in implementation in authentic conditions and want an alternative to dismissing these data as merely sample attrition.

Intensity of Teacher Implementation Support and Student Outcomes

For schools that did initiate SDLMI implementation, we examined the impact of varying intensities of teacher implementation support on student goal attainment and self-determination outcomes. Because of school initiation challenges and the onset of the COVID-19 pandemic in Spring 2020, the focus of the C-RCT shifted to the relationship between condition and student outcomes over a year (Fall and Spring) given student attrition during COVID-19 (see Supplemental Materials). Specifically, for analyses, we collapsed data across years, while controlling for pre- or post-pandemic onset. As we were interested in differences in outcomes of students with disabilities compared to their peers without disabilities learning in inclusive classes, we also included disability status in the models to inform ongoing research and practice on integrating tiered supports into self-determination instruction (Shogren et al., 2016).

In terms of goal attainment, an interesting pattern of findings emerged. There was an effect of teacher implementation support condition (online vs. online + coaching). Yet, this impact differed across the Fall and Spring semesters with the more intensive teacher implementation support condition (online + coaching) predicting expected or greater levels of goal attainment in the Fall semester, while the less intensive support condition (online only) predicting greater goal attainment in the Spring. There are several possible interpretations of this finding, each of which deserves attention in future research and should be considered in practice. It is possible teachers’ and students’ support needs changed as they had a second opportunity to use the SDLMI in the Spring to set and work toward goals. As described, the SDLMI is meant to be used repeatedly with teachers gaining implementation experience and students gaining abilities and skills to set and go after new goals. It is possible teachers may not need intensive support to enhance student outcomes after the first semester of the SDLMI, which would have implications for the long-term delivery of the more intensive and expensive SDLMI coaching model (Rifenbark et al., 2023). It is also possible that teachers may need different supports across years of implementation and based on the support needs of students in their classes. It is also possible that as students learned about goal setting and attainment in the Fall semester they recalibrated their goals and expectations, leading to a more accurate evaluation of their goal attainment in the Spring as has been seen in other self-determination outcome research (Raley et al., 2021). Furthermore, it is important to note the COVID-19 pandemic most significantly impacted Spring 2020 and students may have been less likely to make progress toward goals under any conditions due to the pandemic and the rapid shift to virtual instruction. While the research team provided access to all materials in an online format after the pivot to virtual learning, this abrupt disruption was unprecedented and the impacts on goal attainment in this study and in subsequent years are unknown.

The impacts of the COVID-19 pandemic are also potentially reflected in other findings. For example, students reported abandoning goals at a higher rate (35.9%) after the onset of the pandemic. Of students who did not abandon goals, the remaining students were more likely to be achieving goals at expected or greater than expected levels. This may suggest that when students were able to persist with goal-directed actions during the pandemic, they had better goal attainment outcomes, but larger numbers of students could not sustain their goal-directed actions. This finding was particularly pronounced for students with disabilities, with greater proportions of students with disabilities than their peers without disabilities who did not abandon goals reporting achievement at expected or greater than expected levels. Furthermore, this finding aligns with other research reporting teachers and students identified abilities and skills associated with self-determination as critically important for students to stay on track during the COVID-19 pandemic (Raley et al., 2023; Toste et al., 2021). These findings may generalize to other disruptive circumstances encountered by students, teachers, and schools. Overall, these findings suggest the need for (a) careful planning and monitoring over time to determine the most effective teacher implementation supports that enable student goal attainment and (b) more intensive teacher implementation supports to be provided initially, but potentially faded over time although more research is needed. Future research should also consider tiered coaching models that adjust the supports provided to teachers based on teacher fidelity and student outcomes.

In terms of student self-determination outcomes, Figure 1 shows different patterns across teacher implementation support groups. Previous research using Year 1 data from the C-RCT suggested a “dip” in self-determination at the middle of the year, possibly reflecting Grade 9 students over-reporting their self-determination at baseline and recalibrating as they learned during the first semester of SDLMI instruction (Raley et al., 2021). This pattern was only found in the online-only group in this study, suggesting perhaps this pattern may be shaped by teacher implementation supports, although more research is needed. The nuances in the factors that impact this “dip” are supported by other research that suggests that when controlling for student race and ethnicity this pattern disappears for certain groups of marginalized students (Shogren, Scott et al., 2021). This highlights the need to consider, in research and in practice, tiered approaches to teacher implementation supports (Snyder et al., 2015) as well as a greater focus on culturally responsive implementation (Scott et al., 2021) supports. Interestingly, we did not find an impact of disability or pre/post-pandemic status on self-determination outcomes, suggesting as have other sources (Toste et al., 2021), the importance of self-determination and goal-directed actions, even if goals were not attained, during the pandemic and beyond.

Limitations

We encourage readers to consider the following limitations when interpreting findings. First, as noted, the sample included a small number of schools for Research Question 1 (n = 15) and Research Questions 2 and 3 (n = 8). We used BMA analysis rather than NHST to weigh the probabilities of competing models rather than eliminating a null and selecting an alternative model. This means these probabilistic results await more data. It is important to acknowledge that, as is true for most components of a statistical model, different priors might yield different findings. We selected default priors to ensure Bayes estimates coincide with familiar maximum likelihood estimates. Our posterior predictive model check also confirms that our estimated model can reproduce the observed data. Without independent replication, we should continue to consider all models but weigh them by their probability when predicting new data and focus on the probable direction of effect sizes rather than generalize exact estimates. Furthermore, the impacts of the pandemic on implementation and student-level outcomes cannot be fully understood and remain an ongoing issue. Ongoing, longitudinal research is needed to examine the impacts on learning and engagement of students over time. In addition, the pandemic and resulting attrition of students within schools necessitated us to examine the impacts during a one-year implementation period, rather than across years. We could only look at disability status in our covariate analysis and not race/ethnicity, necessitating ongoing work given other research establishing the interaction of disability and race/ethnicity on student outcomes. Ongoing work is needed to examine the longitudinal impacts of the SDLMI when used as a universal support on teacher and student outcomes.

Finally, it must be acknowledged that multiple factors played a role in shaping a school’s decision to sign an MOU and then not initiate participation in the study, and more research is warranted. Our design still has high internal validity. Specifically, schools were randomly assigned to groups. Therefore, our design allowed us to test if group membership itself might influence the outcome of this initiation decision over and above other explanatory factors (measured and unmeasured). Thus, the present probabilistic findings suggest that group membership was one explanatory factor driving initiation outcomes, which warrants further investigation of ways to isolate and amplify factors that do predict initiation to eventually develop effective strategies for encouraging schools to take up EBPs in challenging times (e.g., pandemic conditions, budgetary challenges, and constraints). For example, we found that schools randomly assigned to the less intensive online-only condition were more prone to drop out and not initiate implementation, compared to schools in the more intensive online + coaching condition, which was further influenced by Title I status. This finding identifies a potential causal mechanism, given the robust internal validity of our study design, that must be further considered in research and in practice. It specifically highlights the critical need to match teacher implementation supports to the needs of schools and the teachers within them to advance initiation and sustained use of EBPs impacting student outcomes.

Conclusion

Despite limitations, this study provides critical information to consider in the design and analysis of C-RCTs and factors that impact both the initiation and implementation of EBPs. Ongoing research and practice-based evaluation are needed to determine the most effective ways to tailor supports for schools, teachers, and students to enable self-determined learning and advance sustained implementation and implementation supports that are aligned with school, teachers, and student needs and advance student outcomes.

Supplemental Material

sj-docx-1-tes-10.1177_08884064241255217 – Supplemental material for The Impact of Teacher Supports for Implementing the Self-Determined Learning Model of Instruction on Student Outcomes in Inclusive General Education Classes

Supplemental material, sj-docx-1-tes-10.1177_08884064241255217 for The Impact of Teacher Supports for Implementing the Self-Determined Learning Model of Instruction on Student Outcomes in Inclusive General Education Classes by Karrie A. Shogren, Tyler A. Hicks, Sheida K. Raley, Kathleen Lynne Lane, Carol Quirk, Hunter A. Matusevich, Dale W. Matusevich and Abdulaziz Alsaeed in Teacher Education and Special Education

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A170008 to University of Kansas. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

ORCID iDs

Karrie A. Shogren

Sheida K. Raley

Kathleen Lynne Lane

Abdulaziz Alsaeed

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Karrie A. Shogren is Director of the Kansas University Center on Developmental Disabilities (a University Center for Excellence in Developmental Disabilities), Senior Scientist at the Schiefelbusch Life Span Institute, and Distinguished Professor in the Department of Special Education all at the University of Kansas. Dr. Shogren’s research focuses on assessment and intervention in self-determination and supported decision making for people with disabilities.

Tyler A. Hicks is the Director of Quantitative Methodology at the Kansas University Center on Developmental Disabilities. He is a methodologist interested in advancing the application of statistical methods in special education research.

Sheida K. Raley was an Assistant Research Professor at the Kansas University Center on Developmental Disabilities. Her interests are in self-determination and transition. She is now an Assistant Professor at the University of Arkansas.

Kathleen Lynne Lane is a Roy A. Roberts Distinguished Professor in the Department of Special Education at the University of Kansas and Associate Vice Chancellor for Research. Dr. Lane’s research interests focus on designing, implementing, and evaluating Comprehensive, Integrated, Three-tiered (Ci3T) models of prevention to (a) prevent the development of learning, behavior, and social and emotional well-being challenges and (b) respond to existing instances, with an emphasis on systematic screening.

Carol Quirk is the Founder and Executive Director of the Maryland Coalition for Inclusive Education (MCIE). She leads professional development and technical assistance, including the MCIE systems change process for school transformation.

Hunter A. Matusevich was a doctoral student at the Kansas University Center on Developmental Disabilities. Her interests are in self-determination and transition. She is now at James Madison University.

Dale W. Matusevich is the Director of Exceptional Children Resources at the Delaware State Department of Education.

Abdulaziz Alsaeed is a doctoral student at the Kansas University Center on Developmental Disabilities. His interests are in self-determination and transition.

References

Ando

(2007). Bayesian model selection and statistical models. CPC Press.

Bojanek

E. K.

Raley

S. K.

Shogren

K. A.

Lane

K. L.

(2021). Examining the impact of professional development on the self-determined learning model of instruction for general and special educators. Inclusion, 9(2), 118–133. https://doi.org/10.1352/2326-6988-9.2.118

Buckman

M. M.

Lane

K. L.

Common

E. A.

Royer

D. J.

Oakes

W. P.

Allen

G. E.

Lane

K. S.

Brunsting

(2021). Treatment integrity of primary (Tier 1) prevention efforts in tiered systems: Mapping the literature. Education and Treatment of Children, 44, 145–168. https://doi.org/10.1007/s43494-021-00044-4

Burke

K. M.

Raley

S. K.

Shogren

K. A.

Hagiwara

Mumbardó-Adam

Uyanik

Behrens

(2020). A meta-analysis of interventions to promote self-determination for students with disabilities. Remedial and Special Education, 41(3), 176–188. https://doi.org/10.1177/0741932518802274

Burke

K. M.

Shogren

K. A.

Antosh

A. A.

LaPlante

Masterson

L. H.

(2019). Implementing the SDLMI with students with significant support needs during transition planning. Career Development and Transition for Exceptional Individuals, 43(2), 115–121. https://doi.org/10.1177/2165143419887858

Common

E. A.

Buckman

M. M.

Lane

K. L.

Oakes

W. P.

Royer

D. J.

Chafouleas

Briesch

Sherod

(2021). Project ENHANCE: Assessing professional learning needs for implementing comprehensive, integrated, three-tiered (Ci3T) models of prevention. Education and Treatment of Children, 44(3), 125–144. https://doi.org/10.1007/s43494-021-00049-z

Cook

B. G.

Odom

S. L.

(2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(3), 135–144. https://doi.org/10.1177/001440291307900201

Darling-Hammond

Wei

R. C.

Andree

Richardson

Orphanos

(2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. National Staff Development Council.

Gelman

Hill

(2006). Analytical methods for social research: Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.

10.

Hagiwara

Shogren

Leko

(2017). Reviewing research on the self-determined learning model of instruction: Mapping the terrain and charting a course to promote adoption and use. Advances in Neurodevelopmental Disorders, 1, 3–13. https://doi.org/10.1007/s41252-017-0007-7

11.

Hagiwara

Shogren

K. A.

Lane

K. L.

Raley

S. K.

Smith

S. A.

(2020). Development of the self-determined learning model of instruction coaching model: Implications for research and practice. Education and Training in Autism and Developmental Disabilities, 55(1), 17–27.

12.

Howson

Urbach

(2006). Scientific reasoning: The Bayesian approach (3rd ed.). Open Court.

13.

Kiblen

J. C.

Shogren

K. A.

Zimmerman

K. N.

Raley

S. K.

Mansouri

M. C.

Alsaaed

A. H.

(2023). A review of teacher fidelity measurement of the self-determined learning model of instruction. Education and Training in Autism and Developmental Disabilities, 58(4), 422–440

14.

Kiresuk

T. J.

Smith

Cardillo

J. E.

(1994). Goal attainment scaling: Applications, theory, and measurement. Lawrence Erlbaum Associates.

15.

Krasny-Pacini

Evans

Sohlberg

M. M.

Chevignard

(2016). Proposed criteria for appraising goal attainment scales used as outcome measures in rehabilitation research. Archives of Physical Medicine and Rehabilitation, 97(1), 157–170. https://doi.org/10.1016/j.apmr.2015.08.424

16.

Mazzotti

V. L.

Rowe

D. A.

Kwiatek

Voggt

Chang

W.-H.

Fowler

C. H.

Poppen

Sinclair

Test

D. W.

(2021). Secondary transition predictors of postschool success: An update to the research base. Career Development and Transition for Exceptional Individuals, 44(1), 47–64. https://doi.org/10.1177/2165143420959793

17.

Odom

S. L.

(2021). Education of students with disabilities, science, and randomized controlled trials. Research and Practice for Persons with Severe Disabilities, 46(3), 132–145. https://doi.org/10.1177/15407969211032341

18.

Odom

S. L.

Hall

L. J.

Steinbrenner

J. R.

(2020). Implementation science research and special education. Exceptional Children, 86(2), 117–119. https://doi.org/10.1177/0014402919889888

19.

Raftery

A. E.

Madigan

Hoeting

(1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92(437), 179–191. https://doi.org/10.1080/01621459.1997.10473615

20.

Raley

S. K.

Hagiwara

Burke

K. M.

Kiblen

J. C.

Shogren

K. A.

(2022). Supporting all students to be self-determined: Using the self-determined learning model of instruction within multi-tiered systems of supports. Inclusive Practices. 2(1), 3–12. https://doi.org/10.1177/27324745221132478

21.

Raley

S. K.

Kutscher

Shogren

K. A.

Kurth

J. A.

Matusevich

H. A.

(2023). Finding common ground: Tips to promote self-determination in inclusive contexts for high school general and special educators. Inclusive Practices. 2(1), 22–29.

22.

Raley

S. K.

Shogren

K. A.

Rifenbark

G. G.

Lane

K. L.

Pace

J. R.

(2021). The impact of the self-determined learning model of instruction on student self-determination in inclusive, secondary classrooms. Remedial and Special Education, 42(6), 363–373. https://doi.org/10.1177/0741932520984842

23.

Rifenbark

G. G.

Hicks

T. A.

Burke

K. M.

Swinburne Romine

R. E.

Fleming

K. K.

Shogren

K. A.

(2023). Evaluation of program costs for implementing a self-determination intervention using the ingredients method. In Mazzotti

Lombardi

(Eds.), Career development and transition for exceptional individuals (Vol. 47, p. 79). SAGE. https://doi.org/10.1177/21651434221130295

24.

Rowe

D. A.

Mazzotti

V. L.

Fowler

C. H.

Test

D. W.

Mitchell

V. J.

Clark

K. A.

Holzberg

Owens

T. L.

Rusher

Seaman-Tullis

R. L.

Gushanas

C. M.

Castle

Chang

W-. H.

Voggt

Kwiatek

Dean

(2021). Updating the secondary transition research base: Evidence- and research-based practices in functional skills. Career Development and Transition for Exceptional Individuals, 44(1), 28–46. https://doi.org/10.1177/2165143420958674

25.

SAS Institute. (2015). SAS/STAT^® 14.1 user’s guide.

26.

Scott

L. A.

Thoma

C. A.

Gokita

Bruno

Ruiz

Brendli

Taylor

Vitullo

(2021). I’m trying to make myself happy: Black students with IDD and families on promoting self-determination during transition. Inclusion, 9(3), 170–188. https://doi.org/10.1352/2326-6988-9.3.170

27.

Shogren

K. A.

Burke

K. M.

Anderson

M. H.

Antosh

A. A.

LaPlante

Hicks

T. A.

(2020). Examining the relationship between teacher perceptions of implementation of the SDLMI and student self-determination outcomes. Career Development and Transition for Exceptional Individuals, 43(1), 53–63. https://doi.org/10.1177/2165143419887855

28.

Shogren

K. A.

Dean

E. E.

Burke

K. M.

Raley

S. K.

Taylor

J. L.

(2021). Goal attainment scaling: A framework for research and practice in the intellectual and developmental disabilities field. Intellectual and Developmental Disabilities, 59(1), 7–21. https://doi.org/10.1352/1934-9556-59.1.7

29.

Shogren

K. A.

Little

T. D.

Grandfield

Raley

S. K.

Wehmeyer

M. L.

Lang

K. M.

Shaw

L. A.

(2020). The self-determination inventory-student report: Confirming the factor structure of a new measure. Assessment for Effective Intervention, 45(2), 110–120. https://doi.org/10.1177/1534508418788168

30.

Shogren

K. A.

Pace

J. R.

Hicks

T. A.

Raley

S. K.

Lane

K. L.

(2024). Setting standards for SDLMI fidelity: Promoting data-driven decision making to advance self-determination instruction. Psychology in the Schools, 61(2), 532–552. https://doi.org/https://doi.org/10.1002/pits.23068

31.

Shogren

K. A.

Raley

S. K.

(2018). SDLMI fidelity measure. Kansas University Center on Developmental Disabilities.

32.

Shogren

K. A.

Raley

S. K.

(2022). Self-determination and causal agency theory: Integrating research into practice. Springer.

33.

Shogren

K. A.

Raley

S. K.

(2023). Implementing self-determined learning: A guide for special education. Routledge.

34.

Shogren

K. A.

Scott

L. A.

Hicks

T. A.

Raley

S. K.

Hagiwara

Pace

J. R.

Gerasimova

Alsaeed

Kiblen

J. C.

(2021). Exploring self-determination outcomes of racially and ethnically marginalized students with disabilities in inclusive, general education classrooms. Inclusion, 9(3), 189–205. https://doi.org/10.1352/2326-6988-9.3.189

35.

Shogren

K. A.

Shaw

L. A.

Raley

S. K.

Wehmeyer

M. L.

(2018). The impact of personal characteristics on scores on the self-determination inventory: Student report in adolescents with and without disabilities. Psychology in the Schools, 55(9), 1013–1026. https://doi.org/10.1002/pits.22174

36.

Shogren

K. A.

Wehmeyer

M. L.

(2017). Self-determination inventory. Kansas University Center on Developmental Disabilities.

37.

Shogren

K. A.

Wehmeyer

M. L.

Lane

K. L.

(2016). Embedding interventions to promote self-determination within multi-tiered systems of supports. Exceptionality, 24(4), 213–224. https://doi.org/10.1080/09362835.2015.1064421

38.

Snyder

P. A.

Hemmeter

M. L.

Fox

(2015). Supporting implementation of evidence-based practices through practice-based coaching. Topics in Early Childhood Special Education, 35(3), 133–143. https://doi.org/10.1177/0271121415594925

39.

Stahmer

A. C.

Suhrheinrich

Schetter

P. L.

McGee Hassrick

(2018). Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): A study protocol. Implementation Science, 13(1), 3. https://doi.org/10.1186/s13012-017-0698-1

40.

Toste

J. R.

Raley

S. K.

Gross Toews

Shogren

K. A.

Coelho

(2021). “Eye opening and chaotic”: Resilience and self-determination of secondary students with disabilities amidst the COVID-19 pandemic. Journal of Education for Students Placed at Risk (JESPAR), 26(2), 157–183. https://doi.org/10.1080/10824669.2021.1906248

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB