Abstract
This study examined the differential impact of implementing the Self-Determined Learning Model of Instruction (SDLMI) alone with implementing the SDLMI combined with Whose Future Is It? with transition aged students with intellectual disability in a cluster randomized trial in the state of Rhode Island. The state of Rhode Island is implementing systemic change in transition services and supports under the auspices of a Consent Decree entered into by the state with the U.S. Department of Justice. One area of focus is promoting self-determination during transition planning in the school context as a means to affect employment trajectories. This study focused on the impact of self-determination instruction on self-determination outcomes while youth were still in school, given research establishing a relationship between self-determination and employment outcomes. Latent mediation models suggested that students in the SDLMI-only group reported significant increases in their self-determination scores from baseline to the end of the year, and teachers of students in the SDLMI-only group saw students’ goal attainment as predicting change in self-determination over the course of the year. Teachers reported significant changes in student self-determination in the SDLMI + Whose Future Is It? group. Implications for individualizing interventions to teach skills associated with self-determination in the context of planning and setting goals for the transition to integrated employment are discussed.
Despite a long-standing emphasis on the importance of supporting adolescents with intellectual disability to transition from school to meaningful postschool education and employment outcomes (Wehman, 2012), data continue to suggest that only approximately 10% of adults with intellectual and developmental disabilities in the United States are competitively employed in the community (Butterworth, Hiersteiner, Engler, Bershadsky, & Bradley, 2015). These rates have remained essentially unchanged over the last decade (Butterworth et al., 2014; Domin & Butterworth, 2013). Furthermore, segregated, noncommunity-based employment continues to be the default option for many adults with intellectual disability despite long-standing concerns and research demonstrating that such models do not lead to integrated, community-based employment (Cimera, Wehman, West, & Burgess, 2012). Researchers and policy makers have acknowledged the need for systemic change in policy and practice to change the options and outcomes for people with intellectual disability (Nord et al., 2015).
Recent policy initiatives to address these disparate outcomes have recognized the importance of planning for the transition from school to integrated employment. For example, the Workforce Innovation and Opportunities Act (WIOA) of 2014 requires pre-employment transition services for students with disabilities. In addition, the increased enforcement by the U.S. Department of Justice (DOJ) of the Americans with Disabilities Act (ADA; 1990) and the application of the Supreme Court decision in Olmstead v. L.C. (1999) to employment supports and services (U.S. Department of Justice, Civil Rights Division, n.d.) has emphasized the role of transition supports and services for students with disabilities in enabling integrated employment outcomes in adulthood.
In 2014, Rhode Island was the first state to enter into a statewide Consent Decree with the DOJ related to violations of Olmstead as applied to integrated employment options and outcomes. In 2015, the state of Oregon reached a similar settlement with the DOJ. Although the Olmstead decision focused on supports provided for community living, it is being applied by the DOJ to other domains of life, including employment supports (U.S. Department of Justice, Civil Rights Division, n.d.). The DOJ found “unnecessary and overreliance upon segregated sheltered workshops and facility-based day programs” in Rhode Island. Consistent with WIOA, the Consent Decree recognized that poor adult employment outcomes and the overreliance on segregated options did not simply emerge in adulthood. The decree highlighted the role that the school system played by recognizing transition-age youth (adolescents 14 and older in Rhode Island) as a target population and mandating changes in the school-based transition planning process with a focus on promoting the movement of students with intellectual disability from school to integrated employment outcomes. A major impetus for the inclusion of transition-age youth as a class in the Consent Decree was a finding from the DOJ investigation that only 5% of transition-age youth with intellectual disability in the state had integrated employment goals in their Individualized Education Programs (IEPs) or transition plans.
To address the needed change in the state to improve options for integrated employment, the Consent Decree established the “Conversion Institute” at the Paul V. Sherlock Center on Disabilities at Rhode Island College, and leaders in Rhode Island identified multiple strategies designed to promote systematic changes in employment outcomes. One area of emphasis identified by the Conversion Institute was enhancing the capacity of secondary special education teachers to support students with intellectual disability in developing skills associated with self-determination (i.e., choice making, decision making, problem solving, planning, goal setting and attainment, self-management, self-advocacy, self-awareness, and self-knowledge) to enable students to set and attain transition goals related to integrated employment. This area of emphasis was targeted because of research establishing a relationship between enhanced self-determination and postschool employment outcomes in adolescents with disabilities, specifically findings that youth with disabilities, including those with intellectual disability, who exit school with higher levels of self-determination experience more positive integrated employment outcomes (Shogren & Shaw, 2016; Shogren, Wehmeyer, Palmer, Rifenbark, & Little, 2015; Wehmeyer & Palmer, 2003; Wehmeyer & Schwartz, 1997). As such, enhancing self-determination was recognized as a research-based practice that could be utilized in school-based transition services and supports to affect not only self-determination outcomes in school but also postschool integrated employment outcomes.
Self-Determination and Transition to Adulthood
Self-determination is defined as a “dispositional characteristic manifested as acting as the causal agent in one’s life. Self-determined people (i.e., causal agents) act in service to freely chosen goals” (Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015, p. 258). Researchers suggest that self-determination develops over the lifespan as young people are taught and have opportunities to practice skills leading to greater self-determination across life domains, including academic, social, home, and career development (Wehmeyer, Shogren, Little, & Lopez, 2017). Three essential characteristics define self-determined action: volitional action (initiating goal setting by making conscious choices based on personal preferences), agentic action (acting intentionally to make or cause something to happen when working toward goals), and action-control beliefs (believing one has what it takes to achieve their goals; Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015). In the context of school-based transition planning, educators can use interventions, like the Self-Determined Learning Model of Instruction (SDLMI) and Whose Future Is It? (WF) to target self-determination skills by supporting students to set and go after transition goals related to integrated employment (Shogren, 2013).
Self-Determined Learning Model of Instruction
The Self-Determined Learning Model of Instruction (SDLMI; Shogren, Wehmeyer, Burke, & Palmer, 2017; Wehmeyer, Palmer, Agran, Mithaug, & Martin, 2000) is a teaching model based on theory in self-determination (Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015) that can be used by educators (or others trained in its use) to organize instruction and supports to enhance self-regulated goal setting and attainment of students with and without disabilities. The SDLMI, as a model of instruction, differs from curricula designed to teach specific transition planning and self-determination-related skills in that it is not designed to deliver standardized content related to transition planning. It was, instead, developed to be utilized by educators to shape their instruction to be student-directed rather than teacher-directed through supporting students to self-regulate problem solving leading to goal setting and attainment in any content area, including planning for the transition to employment. Teachers individualize the implementation of the SDLMI for students and their specific learning goals. When targeting goals related to transitioning to employment, teachers use the SDLMI to identify curricular and instructional resources necessary to enable students to identify career options, interests, and resources. As such, the SDLMI can be utilized by teachers with students with a range of support needs, including students with severe disabilities, by individualizing the supports provided based on communication and comprehension-related needs.
Over a dozen quasi-experimental and single-case design studies (see S. H. Lee, Wehmeyer, & Shogren, 2015) and large-scale, randomized control trials (Shogren, Palmer, Wehmeyer, Williams-Diehm, & Little, 2012; Wehmeyer et al., 2012) have been conducted using the SDLMI. Researchers have established the causal impact of the SDLMI on transition goal attainment in youth with intellectual disability (Shogren et al., 2012) and overall self-determination when used to instruct youth with intellectual disability in middle and high school (Wehmeyer et al., 2012). Researchers have also linked enhanced postschool employment and community participation outcomes with enhanced self-determination resulting from interventions delivered to students while in school (Shogren, Wehmeyer, Palmer, Rifenbark, et al., 2015).
Whose Future Is It?
WF (Wehmeyer & Palmer, 2011) was developed to provide teachers a curriculum to guide the delivery of instruction on specific self-determination skills associated with the transition planning process. Curricula such as WF differ from models of instruction like the SDLMI in that they provide content relevant for a specific topic, in this case transition planning, rather than a model through which teachers organize their delivery of curricular content, as does the SDLMI. As such WF requires dedicated time for instruction and engaging in the curriculum, while the SDLMI is overlaid on existing and ongoing curricular activities. WF is technology-based and utilizes features of universal design (e.g., multiple means of representation, action and expression, engagement) to deliver content. It includes read-aloud supports, questions to check for understanding embedded in lessons, picture and graphical representations of concepts, and opportunities for repetition of content. It is an adaption of Whose Future Is It Anyway? (Wehmeyer et al., 2004). Both WF and Whose Future Is It Anyway? have been demonstrated to positively affect transition knowledge and skills (Y. Lee et al., 2011; Wehmeyer, Palmer, Lee, Williams-Diehm, & Shogren, 2011; Wehmeyer, Palmer, Shogren, Williams-Diehm, & Soukup, 2013).
Purpose of the Present Study
The SDLMI and WF are evidence-based practices that affect transition knowledge and skills, goal attainment, and self-determination while youth are in school, and postschool employment outcomes resulting from enhanced self-determination during school (Shogren, Wehmeyer, Palmer, Rifenbark, et al., 2015). However, the combined impact of the SDLMI and WF has not been examined. As mentioned, in Rhode Island, enhancing student self-determination skills in the context of planning for the transition from school to the adult world was a targeted area based on the assumption that this would lead to long-term changes in integrated employment outcomes. The SDLMI and WF were identified as interventions that could be utilized in the state through a partnership between researchers at the Conversion Institute and researchers at the University of Kansas. However, questions about the degree to which teachers would be able to implement the SDLMI in flexible and individualized ways to promote change in self-determination and transition outcomes emerged. It was determined that during the first year of implementing systematic efforts (2015-2016), the target would be to train teachers to implement the SDLMI with a focus on exploring fidelity of implementation during instruction with students with a range of support needs. The following year (2016-2017) would be spent exploring the impact of adding WF and exploring the in-school and postschool outcomes of young adults targeted under the decree. Questions about the need for both interventions, particularly given the demands on instructional time, drove the questions of the differential impact of utilizing both the SDLMI and WF.
Shogren, Burke, et al. (2017) reported on the outcomes of the initial year of implementation of the SDLMI only with special educators working with students with intellectual disability. They found that with a one-and-a-half-day initial training and ongoing coaching, special education teachers could implement the SDLMI with fidelity with students with intellectual disability with a range of support needs. Furthermore, teachers were able to flexibly use the SDLMI materials (e.g., using picture-based supports), in accordance with implementation protocols, for students with a range of communication and cognitive support needs. Teachers reported changes after a year of implementation in student volitional action and agentic action, and expected levels of the attainment of goals linked to transitioning to integrated employment.
As mentioned previously, the next step was to explore the combined impact of the SDLMI and WF on transition outcomes during the 2016-2017 year. The present study reports the findings from the examination of the differential impact of the SDLMI only versus the SDLMI + WF during the 2016-2017 year. In 2016-2017, participating schools were randomly assigned to continue implementing the SDLMI alone or to add WF to the instruction provided to students with intellectual disability. The purpose of this analysis was to explore whether adding WF to the SDLMI further enhanced self-determination or whether teachers were able to individualize instruction with the SDLMI to enhance self-determination without the need for an additional, scripted curriculum such as WF. This represented another step in ongoing research to determine the most effective ways to support self-determination in the context of planning for the transition to integrated employment while youth are still in school. The following research questions were addressed:
Method
Sample
The sample included 340 transition-age students served under the educational classification of intellectual disability from 17 school districts across the state of Rhode Island. There were 205 males and 99 females in the sample, with gender not reported for 36 participants. Participants ranged in age from 10 to 21 years (M = 16.52, SD = 2.13). The largest racial/ethnic group was White (n = 123), and the second largest racial/ethnic group was Hispanic/Latino (n = 41). In addition to a diagnosis of intellectual disability, 101 (30%) participants had a secondary disability classification. Specific information on the level of intellectual impairment (i.e., mild, moderate, severe, profound) of participants was requested from teachers but not consistently reported in students’ educational records. Level of intellectual impairment data was collected for 72 of 340 students in the sample (21%). Of these 72 students, 67 were identified with moderate, severe, or profound intellectual disability. Further information about student participants, including amounts of missing demographic data, are shown in Table 1. A total of 64 special education teachers (60 females and four males) implemented the intervention with students. The average age of teachers was 42.10 years (SD = 12.44), and the average time teachers knew the student was 2.98 years (SD = 2.48). Teachers worked with between one and 22 students.
Student Descriptive Statistics by Randomization Group.
Note. Total of percentages for each category may not be 100% due to rounding. SDLMI = Self-Determined Learning Model of Instruction; WF = Whose Future Is It? ADD = attention deficit disorder; ADHD = attention deficit hyperactivity disorder.
Procedures
As described in Shogren, Burke, et al. (2017), during the first year of implementation (2015-2016), 40 teachers received a one-and-a-half-day training on the SDLMI from University of Kansas researchers. They also received ongoing coaching to implement the SDLMI from trained district coaches. For the second year of implementation, districts were randomly assigned to the SDLMI-only or SDLMI + WF conditions; districts, teachers, and students were relatively evenly split between the SDLMI-only (n = 8 districts, 27 teachers, 173 students) and SDLMI + WF condition (n = 9 districts, 31 teachers, 167 students). At the beginning of the second year, new teachers receive the same one-and-a-half-day training from the first year, and all teachers in the SDLMI + WF condition received an additional one-half-day training on WF and all materials necessary to utilize the curriculum with students. Teachers in both conditions were supported by coaches identified in each district. Coaches made a minimum of three classroom visits (45 to 60 min each) per year. Coaching supports were differentiated based on treatment condition (e.g., teachers in the SDLMI + WF group received resources and supports on using WF). Coaches also participated in a monthly in-person problem-solving and professional development meeting with staff from the University of Kansas and the Conversion Institute to enhance their skills for supporting teachers. Coaches, after each of their three classroom visits, provided ratings of teacher fidelity of implementation. Fidelity was assessed using indicators of quality of implementation of the Teacher Objectives rated on a scale of 0 (not at all) to 3 (completely). The average fidelity rating was 1.43 for the SDLMI-only group and 1.61 for the SDLMI + WF group. These numbers represent average or acceptable implementation, based on the metric of the scale.
Intervention Components
As mentioned, the primary goal of the study was to examine the differential impacts of the SDLMI alone and the SDLMI combined with WF on overall self-determination and goal attainment for goals set by students while in school but connected to planning for the transition to integrated employment.
The Self-Determined Learning Model of Instruction
Teachers were trained to implement the SDLMI in the context of ongoing activities to enhance transition planning and employment outcomes. The SDLMI is designed to be used repeatedly during a school year; teachers in both conditions worked at an individualized pace using the SDLMI to enable students to set and work toward at least three goals (i.e., completing the SDLMI Phases 1, 2, and 3 at least three times during the year). Teacher implementation of the SDLMI involves three distinct phases of instruction (Phase 1: Set a Goal, Phase 2: Take Action, Phase 3: Adjust Goal or Plan). In ways directly relevant to the goals that students were targeting, which included goals related to career exploration, building skills necessary for certain career paths, creating job shadowing or internship opportunities, and a variety of other student-directed goals, students worked through the four Student Questions embedded in each phase. Students were supported to interact with the questions verbally, through pictures, and through behavioral indicators and observations with educational supports provided by teachers. Teachers delivered direct instruction on skills embedded in the SDLMI (i.e., goal setting, problem solving, decision making, planning, initiation) at least two times per week consistent with the SDLMI Teacher Objectives and Educational Supports, and integrated the goals and action plans set through the SDLMI into ongoing curricular activities consistent with SDLMI implementation protocols (Shogren, Wehmeyer, Burke, et al., 2017). Teachers were asked to use appropriate individualized modifications for students (such as alternate methods of content presentation and student communication) based on their instructional expertise and knowledge of the student. See Shogren, Burke, et al. (2017) and www.self-determination.org for more information on the SDLMI and its implementation.
Whose Future Is It?
The WF curriculum includes 15 chapters organized into three sections: Getting to Know Your IEP, Decisions and Goals, and Your IEP Meeting. Teachers utilize an Instructor’s Guide to engage students with the Student Reader that delivers the content. There is also a Student Workbook that provides for activities linked to the curriculum that are completed by students. All materials are available in both digital and print formats, and students utilized supports such as read-aloud narration and vocabulary definitions embedded in the digital format. WF instruction occurred one-on-one, in small groups, or with a whole class. Teachers were required to work through all 15 chapters over the school year, dedicating 45 min at least three times a week, and this instruction was in addition to the ongoing use of the SDLMI.
Measures
Self-determination
Student self-determination was assessed using the pilot version of the Self-Determination Inventory: Student-Report (SDI:SR; Shogren et al., 2014b) and Parent/Teacher-Report (SDI:PTR; Shogren et al., 2014a). The SDI:SR and SDI:PTR are newly developed measures of self-determination based on Causal Agency Theory (Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015) that were, at the time of the implementation of the research, being validated in a large-scale standardization sample. The final measures have 21 items, while the pilot version had those 21 items and 30 additional items from the original pool of items. Researchers established that the pilot version of the SDI:SR has satisfactory reliability indices (i.e., Cronbach, Alpha, and Omega) for youth aged 13 to 22 with and without disabilities, including for youth with intellectual disability (Shogren, Wehmeyer, Palmer, et al., 2017). The SDI:SR and SDI:PTR are identical, except for minor wording differences to make items appropriate for self-report (“I have what it takes to reach my goals”) or parent/teacher report (“This student has what it takes to reach his or her goals”). The SDI:SR and SDI:PTR were designed to be implemented in an online format, and almost all respondents (i.e., students and teachers) completed the 51-item pilot version using the online platform, with a small subset of respondents completing the measure using paper and pencil (n = 3 students). Students and teachers responded to each statement using a slider scale with a range of responses from 0 to 99 for disagreement or agreement. An overall self-determination score, and scores for the three essential characteristics defined in Causal Agency Theory (volitional action, agentic action, and action-control beliefs), was calculated. Consistent with administration guidelines, students were supported to complete the SDI:SR at the beginning and end of the year if their teachers determined that self-report was appropriate for the student. For 43 students, teachers reported that the self-report was not appropriate for the student because of significant support needs. Teachers completed the SDI:PTR for all students in the sample at the beginning and end of the year, irrespective of whether the student completed the SDI:SR.
Goal attainment
Goal Attainment Scaling (GAS) was used as the measure of student progress on transition-related goals. The GAS is a measure of goal attainment (Kiresuk, Smith, & Cardillo, 1994) which has been extended to special education (Carr, 1979). The teacher records a student’s goal and establishes a corresponding range of outcomes that might occur from the student working on that goal, which can be described in both quantified (e.g., percentage correct responses) or in less quantified (e.g., appropriate peer interactions) terms. In the context of using the SDLMI, GAS goals and outcomes were specified by the teacher based on the goal set in Phase 1 of the model. The teacher then provided a rating of the student’s outcome when the student completed Phase 3 of the model. As such, teachers submitted GAS data at three time points during the year. The measure includes a five-point scale for outcomes, wherein −2 is least favorable (much less than expected), −1 is less favorable (somewhat less than expected), 0 is acceptable (expected), +1 is favorable (somewhat more than expected), and +2 is most favorable (much more than expected). Raw GAS scores were compiled and converted to standardized T-scores with a mean of 50 and a standard deviation of 10. A mean of 50 reflects an acceptable outcome score, while scores of 40 or less indicate less favorable outcomes, and scores of 60 and above indicate more favorable outcomes (Kiresuk et al., 1994). As mentioned, GAS rubrics were created for each goal set through the SDLMI. There was some variability in GAS ratings made by teachers over the course of the school year; for the first goal, teachers rated 201 student goals; 214 for the second goal; and 196 for the third.
Analysis
Preprocessing
As described previously and shown in Table 1, there was missing demographic information for the control and treatment groups. In addition, there was missing information present on the outcome measures. With the exception of the missing data for the 43 students who were not able to complete the SDI:SR whose data was excluded from analyses of this scale, the rest of the missing data was treated as missing at random (Graham, 2009). To address the missing data, we used multiple imputation; the multivariate imputation by chained equations (mice) package, version 2.46 (van, Buuren, & Groothuis-Oudshoorn, 2011) in R (R Core Team, 2013) was used. Specifically, 100 data sets with 20 iterations were imputed, consistent with current best practices in imputing missing data (Enders, 2010). Quality of the imputations was examined through convergence diagnostic plots.
Research questions
To address our research questions, we engaged in a series of steps that enabled us to examine change over time in SDI:SR and SDI:PTR scores from baseline to the end of the year, and goal attainment across the three data-collection points. Furthermore, we wanted to examine the relation between goal attainment and self-determination over time. As such, our ultimate goal was to construct latent mediation models to examine change in the SDI:SR and SDI:PTR, and the relationship of GAS scores over time to change in the self-determination scores. The first step was to establish invariance of the self-determination measures over time, and across the control and treatment group to ensure that the scale was measuring the same constructs across groups.
Invariance testing
Establishing cross-group (control vs. treatment) and longitudinal measurement invariance confirms that the target measures, the SDI:SR and SDI:PTR, have equivalent structures, factor loadings, and indicator intercepts for configural, weak, and strong invariance over time. To test for invariance, confirmatory factor analysis (CFA) models consisting of the SDI:SR and SDI:PTR subscales were created and evaluated. Mplus 7.3 (Muthén & Muthén, 2012) with robust maximum likelihood (MLR) estimation was used for all analyses. The latent variances of the construct are fixed to one during this process, also known as the fixed factor method, to satisfy scale identification requirements.
Measurement differences in the SDI:SR and SDI:PTR were inspected through model fit and model comparisons, when appropriate, by inspecting the comparative fit index (CFI), the root mean squared error of approximation (RMSEA), and the standardized root mean squared residual (SRMR). A change in CFI that is less than 0.01 is deemed acceptable for establishing invariance (Cheung & Rensvold, 2002). The RMSEA and SRMR reflect model fit where values closer to the lower bound of zero suggest a properly fitting model. The CFI statistic compares the null model with the current model. A better fitting model will possess a CFI value closer to 1.0 (Raykov & Marcoulides, 2006).
Latent mediation model
To examine self-determination and goal attainment concurrently, a latent mediation model was created. To begin, the mediation portion of the latent mediation model consisted of a latent growth curve (LGC). LGC models are a powerful and flexible tool for assessing change over time and, in our case, for GAS scores. Within the LGC portion of the mediation model, both a latent slope and intercept factor were specified to assess each participant’s initial GAS level and how it varied over time. The intercept of this model is interpreted as the average starting value of GAS for that specific group, and the slope is the average amount of change over time. The latent variable associated with the first self-determination time point was specified with its three observed subscale variables (volitional action, agentic action, and action-control beliefs). This latent construct then predicted both the latent intercept and slope of the LGC with the GAS scores. Next, the intercept factor predicted the second self-determination time point with its associated indicators. Finally, a direct path from the first self-determination time point to the second was specified, and an observed covariate associated with time predicted the second time point. This covariate was included to account for variation in when implementation started and ended, and the completion of each assessment. This process was conducted separately for the SDI:SR and SDI:PTR. This latent mediation model was replicated for the treatment and the control group to allow for comparisons based on assignment to the SDLMI-only versus the SDLM + WF group.
Results
As described previously, we engaged in a series of steps to determine the impact of exposure to the SDLMI and the SDLMI + WF on self-determination and goal attainment outcomes, and the relationship between change in self-determination and goal attainment. Table 2 provides the raw scores on the SDI:SR and SDI:PTR from baseline to end of the year, showing growth across all domains from the student and teacher perspective. Due to constraints resulting from the size of the sample and the complexity of the analyses, the nested properties of the data were not addressed in the analyses (see “Limitations”).
Mean and Confidence Interval of Student Outcome Measures.
Note. CI = Confidence Interval; SDI:SR = Self-Determination Inventory: Self-Report; SDI:PTR = Self-Determination Inventory: Parent/Teacher-Report.
Invariance Testing
Our first step was to establish invariance across time, and across the control and treatment group on the SDI:SR and SDI:PTR to ensure the same set of items could be meaningfully used across groups and over time. First, longitudinal null models were generated for the SDI:SR and the SDI:PTR. This specification was required due to the default null model traditionally provided by SEM software that is not appropriate for longitudinal invariance testing. The correct null model is defined by no latent or residual covariances across time and no mean differences. After incorporating this distinction, only item intercepts (means) and variances are reproduced (Widaman & Thompson, 2003). Next, we tested configural, weak, and strong invariance as described in the “Method” section. Model fit information for the correctly specified null model and each subsequent step of the longitudinal invariance testing process is reported in Table 3, separately for the SDI:SR and the SDI:PTR. Results supported a factor structure with a single factor at each time point suggesting no differences in structure between the two groups at baseline, and correlations between the subscales across time and across the treatment and control group. It should be noted that full factorial invariance was not met for the SDI:SR weak longitudinal invariance model, as the factor loading associated with volitional action needed to be freed across time. Within longitudinal structural equation models, this type of partial invariance is not unexpected and did not affect further model testing (Widaman, Ferrer, & Conger, 2010). The partial invariance model was then used for further SDI:SR analysis, as the change in CFI going from the configural specification to the weak partial model satisfied model fit requirements.
Fit Indices for the SDI:SR and SDI:PTR: Measurement Invariance and Model Fit.
Note. Each invariance model contains its constraints, plus the constraints of all previous models. SDI:SR = Self-Determination Inventory: Self-Report; SDI:PTR = Self-Determination Inventory: Parent/Teacher-Report; RMSEA = root mean square error of approximation; CFI = comparative fit index; |ΔCFI| = absolute value of change in comparative fit index; SRMR = standardized root mean square residual.
Latent Mediation Model
After testing measurement invariance, the full latent mediation model (with LGC as the mediator) was specified. This model allowed us to explore (a) change in self-determination as rated on the SDI:SR and SDI:PTR, and (b) the impact of GAS scores on change in SDI scores. First, as shown in Table 4, the two sets of GAS results show change over time. In the “control” group that received the SDLMI only, there was a significant positive slope, indicating change in GAS scores from the beginning to end of the year. No significant changes were observed in the SDLMI + WF. The expected value of 50 was within the standard deviation of the observed scores for all three goal attainment periods.
Parameter Estimates for GAS Latent Growth Curve and SDI Latent Mediation Model.
Note. GAS = Goal Attainment Scaling; SDLMI = Self-Determined Learning Model of Instruction; WF = Whose Future Is It? SDI:SR = Self-Determination Inventory: Self-Report; SDI:PTR = Self-Determination Inventory: Parent/Teacher-Report.
p < .05.
When looking at the mediation results, the results reported in the bottom half of Table 4 represent the effect of GAS scores on change in SDI scores. The first parameter determines whether a participant’s score on the SDI at Time Point 1 can predict their average initial GAS score, seen as the intercept in the LGC portion of the model. Similarly, the second reported parameter quantifies the effect of the participant’s initial SDI score and in turn predicts the slope factor (average change in GAS) of the LGC. The third parameter of interest shows the degree to which the intercept factor of the LGC predicts the participant’s SDI score at the second observed time point. The fourth reported parameter estimate examines the direct effect of SDI at Time Point 1 to Time Point 2. As shown in Table 4, for student self-reported self-determination, the only significant path was the direct effect of the SDI:SR from baseline to the end of the year in the SDLMI-only group. There was a similar effect in the SDLMI + WF group for the SDI:PTR. Also, for the SDI:PTR but in the SDLMI-only group, there was a significant impact of the GAS intercept on teacher-reported self-determination at the end of the intervention year in the control group, but no other significant effects.
Discussion
The purpose of this study was to examine the differential impact of implementing the SDLMI only compared with the SDLMI with WF on self-determination and goal attainment outcomes of adolescents with intellectual disability over a 1-year period in the state of Rhode Island. The context for this evaluation was the ongoing implementation of efforts in the state mandated by a Consent Decree with the DOJ to address poor integrated employment outcomes of adults with intellectual disability, and the recognized need to utilize the secondary transition planning process and specifically the promotion of self-determination as a part of the change process given the identified linkages between increased self-determination while youth are in school with postschool employment outcomes (Shogren, Wehmeyer, Palmer, Rifenbark, et al., 2015).
The findings suggest that changes in self-determination and its essential characteristics were reported by students and teachers over the 1-year period (see Table 2). Interestingly, when exploring the differential impacts of promoting these skills through the SDLMI only versus the SDLMI + WF, most of the changes were focused in the group of students that experienced the SDLMI only. As shown in Table 4, students in this group had greater growth in GAS and self-determination scores over time than students in the SDLMI + WF group. And teachers of students in the SDLMI-only group saw students’ goal attainment as predicting change in self-determination over the course of the year. The one differential finding between teachers and students was that teachers in the SDLMI + WF group reported significant change in student’s self-determination scores over time.
The results provide useful information for schools, districts, and states to consider when making decisions about expanding the intensity and type of instruction (e.g., model of instruction vs. curriculum vs. both) focused on self-determination in the context of planning for the transition from school to integrated employment outcomes. Overall, it appears that the focus of the SDLMI on enhancing goal attainment when planning for the transition from school to integrated employment influences self-determination, from the student’s perspective, and actual goal attainment from the teacher’s perspective. Furthermore, teachers see a strong link between goal attainment and self-determination, consistent with the theoretical perspectives that guide the SDLMI (Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015).
The fact there were fewer significant changes in the SDLMI + WF group could be interpreted in several ways. First, it may be that the addition of a standardized curriculum required more time and diffused the focus on goal setting and attainment through the SDLMI. It is possible that using the SDLMI only allowed students and teachers to focus more on individualized, meaningful goals, and the additional time required by a standardized transition planning curriculum may not promote the same outcomes. Also, students may have been learning transition planning skills in the WF process (which were not directly measured in this study) but not yet applying these skills to goal attainment using the SDLMI. The fact that teachers reported significant change in self-determination scores in the SDLMI + WF group may suggest that teachers saw a stronger impact of these learning activities. Previous randomized control trial studies of the WF and Whose Future Is It Anyway? processes show the interventions’ positive impact on a variety of outcomes, including transition knowledge and self-determination skills (Y. Lee et al., 2011; Wehmeyer et al., 2011; Wehmeyer et al., 2013). So, it is possible that, over time, differential impacts may be seen, particularly if specific student profiles are analyzed.
Limitations and Future Research, Policy, and Practice Directions
In interpreting the results, several limitations must be considered in the context of the present study, providing direction for future research, policy, and practice. First, the implementation and data-collection efforts occurred in a real-world context, with multiple competing demands and concurrent change initiatives. For that reason, capturing and analyzing data on the multiple factors that affect variability in outcomes was challenging. For example, while the goal of the Conversion Institute was to implement intervention statewide, targeting all schools in the state of Rhode Island and all adolescents with intellectual disability who were of transition-age when the Consent Decree was signed (approximately 700), multiple factors influenced the buy-in and participation of districts and the ability to promote rapid change. Some teachers and schools chose to participate in trainings but not to collect and report data or to participate in ongoing coaching activities; others felt that they were already engaged in initiatives to support change and chose to continue their own initiatives. More work is needed to explore the best ways to promote top–down and bottom–up change, and engage all members of the stakeholder community in planning for systematic change and scaling-up (Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010a). As other researchers have noted, it can take several years before change initiatives reach large-scale implementation and adoption with different supports needed at different implementation stages (Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010b). Ongoing work is needed to determine whether the process adopted in transition education for students with intellectual disability in Rhode Island is sustainable, and whether the efforts affect integrated employment outcomes and adult services and supports provided.
Relatedly, the data collection and implementation efforts emerged quickly in direct response to mandated change emerging from the Consent Decree. As such, there were limitations in the data available and the integration of data across sources across systems for the evaluation. Integrating data on school and postschool outcomes to analyze the impact of student, school, and state-level interventions is an area in need of ongoing attention as initiatives are scaled-up, as linking student school-based experiences and outcomes with adult services and supports and postschool transition outcomes remains challenging. There is a need for better communication and integration across systems during not only the transition planning process but also the integrated use of data to make decisions about effectiveness of change efforts and cross-system initiatives. In the present study, there were missing data resulting from difficulties with data being entered into and accessed across the various electronic systems used to collect and manage data. Furthermore, while we had hoped to easily track data on the small subset of youth who transitioned from the school system to adult services and supports during the intervention period, this proved more challenging than anticipated, and ongoing work is being undertaken to explore ways to track longitudinal outcomes of the transition from school-based supports and services. This will be a critical area for ongoing focus, as the goal of the Consent Decree and other policy initiatives is making changes in adult employment outcomes. Research suggests that enhancing self-determination in school affects postschool employment outcomes (Shogren, Wehmeyer, Palmer, Rifenbark, et al., 2015), but that needs to be confirmed in these types of large-scale change initiatives particularly as implemented in concert with interventions specifically targeting employment skills and opportunities.
In addition, we had a relatively small sample size in our two conditions, particularly at the school level, and missing data on several outcome variables. As a result, restrictions in our ability to account for the multiple nested factors present in the data were encountered. However, these influences could have explained variability in outcomes across groups and at the individual level. While these results that did not account for nesting provide guidance for the future, there is a need for ongoing work that builds larger sample sizes and integrates data across multiple sources to enable a more nuanced understanding of outcomes at the individual and systems level, and allows for multilevel analyses. In addition, given the data-collection issues, we were not able to link the data collected in 2016-2017 to data collected in 2015-2016 on self-determination, and examine cross-year change. As a result, many of the students already had exposure to self-determination interventions, and the results may not generalize to other settings where interventions to promote self-determination are first being implemented. Furthermore, we were not able to control for or analyze differences based on specific student characteristics, such as severity of disability.
Although fidelity of implementation data were collected that showed adequate (or average) implementation fidelity, there are likely multiple ways that ongoing supports could be utilized to further enhance implementation leading to higher levels of fidelity. A coaching model was utilized, which provided useful ongoing support, but ongoing research is needed to examine the impact of utilizing a coaching model to promote self-determination and specific issues that should be considered in coaching teachers in this context (Fixsen, Blase, Naoom, & Wallace, 2009). Accounting for variability in coaching, training, and implementation, and contextual factors and barriers in future analyses will assist in identifying the best combination of supports that promote self-determination and longer-term outcomes, particularly as complex, long-term, cross-system initiatives are implemented to address the disparities in integrated employment outcomes (Burke et al., in press).
Finally, researchers have established that variations of the SDLMI, such as the Self-Determined Career Design Model (SDCDM; Wehmeyer et al., 2003), which adopts the same self-regulated problem solving process with adults to focus on career design and development goals, can be meaningfully implemented to support adults with intellectual disability (Shogren et al., 2016; Shogren & Wehmeyer, 2016). It is possible that the utilization of flexible models of instruction, such as the SDLMI and SDCDM may provide a way to promote collaboration between school and adult supports in the way that instruction and supports are conceptualized, with a focus on self-determination as the base. Essentially, collaborations that target seamless supports for goal setting and attainment, particularly during the transition process, using models such as the SDLMI and SDCDM could be developed. However, achieving this will necessitate the use of interagency collaboration practices during the transition process (Noonan, Morningstar, & Erickson, 2008; Test et al., 2009), and researchers, policy makers, and practitioners must consider ways to enhance opportunities for communication, collaboration, and joint adoption of initiatives and interventions necessary to facilitate this outcome particularly as integrated employment outcomes will be influenced by the resources and supports available in-school and postschool.
Conclusion
There is an increased focus on enforcing the right of adults with intellectual disability to integrated employment options and outcomes. There is also a growing focus in policy on supporting the use of strategies such as transition and pre-employment supports and services to enable postschool integrated employment outcomes (e.g., 2014 WIOA). This necessitates ongoing work to ensure that young people with disabilities are at the center of the process of identifying, setting, and working toward meaningful goals that lead to integrated employment. A wide body of research suggests that self-determination skills are central to promoting goal-directed behavior and outcomes related to integrated employment (Shogren, Wehmeyer, Palmer, Rifenbark, et al., 2015). The results of this study suggest that adolescents with intellectual disability show enhanced self-determination skills (self- and teacher-reported) when using the SDLMI in secondary school to focus on setting and attaining goals related to the transition to integrated employment. Longer-term research, policy, and practice initiatives must continue to include and evaluate the impact of transition planning and the enhancement of self-determination skills as components of efforts to reduce the disparities in integrated employment outcomes.
Footnotes
Authors’ Note
The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324L160002 to the University of Kansas.
Editor in Charge:
Paul Wehman