The Effects of Working at Gaining Employment Skills on the Social and Vocational Skills of Adolescents With Disabilities

Abstract

The current investigation was designed to evaluate the effects of the Working at Gaining Employment Skills (WAGES) curriculum on the social and occupational skills of adolescents with disabilities. Adolescents with disabilities were assigned to either an intervention or control condition. Youth in the intervention group were exposed to the WAGES curriculum for approximately 4.5 months, whereas students in the control group received “business-as-usual” within special education settings. Students and teachers completed brief measures pertaining to prevocational/occupational skills as well as measures pertaining to students’ social skills prior to and following the intervention. Results indicated that after controlling for pretest differences on outcome variables, students participating in the intervention had greater vocational outcome expectations, greater occupational skills, and greater social skills (i.e., empathy, cooperation, and assertiveness) than did students with disabilities in the control condition following the intervention. These findings provide preliminary evidence of the effectiveness of the WAGES curriculum.

Keywords

adolescents with disabilities transition social skills instruction intervention research

For the past 25 years, a large and diverse body of research has focused on developing further understanding about the long-term outcomes of adolescents and young adults with disabilities. Within the context of special education, early state-level follow-along studies of youth with disabilities raised serious concerns about the long-term employability of these youth and drew attention to corollary challenges pertaining to earnings, benefits, and independent living within communities (Affleck, Edgar, Levine, & Kortering, 1990; Hasazi et al., 1985; Mithaug, Horiuchi, & Fanning, 1985). Although many of these concerns were not new to educators (Halpern, 1992), the systematic documentation of poor outcomes among youth with disabilities contributed to the incorporation of specific language regarding the importance of “transition” into the Individuals With Disabilities Education Act (1990) including specific requirements to implement transition planning into the special education programming process. The Rehabilitation Act (1973) and its amendments (1992, 1998) provided similar emphasis on outcome-focused planning and interagency collaboration.

The poor outcomes of youth with disabilities also prompted increased national funding for educational research focused on development and testing practices to support the school-to-work transition needs of adolescents with disabilities in public schools.

These policy, research, and practice initiatives appear to be having the desired effect. Recent findings from the first and only national follow-along study of students with disabilities indicates that the postschool outcomes of youth with disabilities are improving (Newman, Wagner, Cameto, Knokey, & Shaver, 2010; Sanford et al., 2011). However, although gaps in postschool outcomes have narrowed, students with disabilities still lag significantly behind their peers in access to and participation in postsecondary education (55% vs. 62%), hourly earnings (US$9.40 vs. US$13.20), and engagement in either postsecondary education or employment (84% vs. 95%) up to 6 years following high school (Sanford et al., 2011). Moreover, despite heightened awareness among educators regarding the importance of transition among students with disabilities, educational efforts during the past 25 years have yielded surprisingly few empirically validated practices for improving the post–high school outcomes among this population of youth (Test, Fowler, et al., 2009). For example, a recent review of evidence within transition conducted by researchers at the National Secondary Transition Technical Assistance Center (Test, Fowler, et al., 2009) found that only 2 of 25 teaching practices reviewed had “strong” levels of evidence (i.e., teaching life skills and teaching purchasing skills).

This is not to say that nothing has been learned. In a separate review, these same researchers reported that transferable skills such as social skills, self-determination, and self-advocacy had “potential” levels of evidence based on correlational findings (Test, Mazzotti, et al., 2009). Indeed, within the field of special education generally, and within transition specifically, there has been a slow but steady increase in understanding of correlates of the postschool outcomes of youth with disabilities. This work is providing an increasingly solid foundation for the development and evaluation of school-based interventions. Some of the more salient predictors of the postschool outcomes of students with disabilities include employment experiences during high school, self-determination skills, and social skills. The current study was undertaken to develop further understanding about the effects of a school-based job-related social skills curriculum (Working at Gaining Employment Skills [WAGES]) on the prevocational and social skills of adolescents with disabilities. The WAGES curriculum is designed to be implemented in high school classrooms and it specifically targets the transition-related social skill needs of adolescents with disabilities.

Vocational Expectations and Occupational Skills

One challenge that educators and other school-based professionals face within the context of transition-related instruction is that they may not have an opportunity to observe the long-term employment or postsecondary educational outcomes that they are attempting to influence. Therefore, high school personnel must rely on more proximal “indicators” of student performance to evaluate the extent to which they are developing critical career-related skills. Two such indicators are students’ vocational outcome expectations and their occupational skills. Social Cognitive Career Theory (SCCT) provides a framework for understanding the development and pursuit of career-related outcomes, including vocational outcome expectations (Betz, 2007). Within SCCT, vocational outcome expectations are the anticipated result of goal pursuits (Fouad & Guillen, 2006). These expectations are positively associated with career and vocational self-efficacy (McWhirter, Rasheed, & Crothers, 2000), career interests (Lent, Brown, & Hackett, 1994), and goal pursuits (Lent et al., 2008). Moreover, positive expectations about future career outcomes are negatively correlated with perceptions of barriers to postsecondary education and training (McWhirter et al., 2000). Several researchers have shown that vocational outcome expectations are malleable through intervention (Diegelman & Subich, 2001; McWhirter et al., 2000), and Fouad and Guillen (2006) argued that adolescence may be a particularly critical developmental period for shaping vocational outcome expectations through learning experiences.

Whereas vocational outcome expectations represent cognitive expectancies about the future, occupational skills represent basic functional skills that are valued in occupational settings. Occupational skills include time management, teamwork skills, and basic-problem-solving skills (Hamzah & Abdullah, 2009). Included within the scope of occupational skills are job-related social skills, along with other basic skills that are important for maintaining employment (L. K. Elksnin & Elksnin, 1996). Basic occupational skills are a foundation for successful employment, and some researchers have estimated that as much as 90% of job loss is due to poor occupational skills including job-related social skills (N. Elksnin & Elksnin, 2001).

Social Skills

In addition to investigating the effects of WAGES on students’ vocational expectations and occupational skills, we were also interested in evaluating the effects of the curriculum on students’ social skills. Social skills have been defined as “socially acceptable learned behaviors that enable a person to interact effectively with others and avoid socially unacceptable responses” (Gresham & Elliott, 1990, p. 1). Students’ social skills are predictive of adjustment across a broad developmental spectrum and have been linked to grade promotion and retention (Agostin & Bain, 1997), academic achievement (Konold, Jamison, Stanton-Chapman, & Rimm-Kaufman, 2010; Miles & Stipek, 2006), and emotional and behavioral adjustment (Segrin & Flora, 2000; Strahan, 2002). Social skills have also been shown to be associated with transition-related skills such as adolescents’ capacity and opportunities to use self-determined behaviors (Pierson, Carter, Lane, & Glaeser, 2008). Benz, Yovanoff, and Doren (1997) reported that parent- and teacher-rated social skills during high school were predictive of postschool employment among young adults with disabilities 1 year after graduation. Despite increased recognition of the importance of social skills, recent findings from the National Longitudinal Transition Study–2 (NLTS-2) indicate that adolescents with disabilities exhibit fewer social skills than their same-age peers in the general population (Marder, Wagner, & Sumi, 2003). These same researchers reported that levels of social skills among students with disabilities are positively associated with adjustment during high school (Marder et al., 2003), postschool employment (Cameto, 2005), and postschool community participation (Wagner, 2005).

In an effort to quantify the effects of social skills interventions within the context of special education and transition, Alwell and Cobb (2009) conducted a meta-analysis of 30 interventions designed to improve the augmentative communication, conversational, and social skills of adolescents with disabilities. One important finding that emerged from this study was that none of the social skills training interventions reviewed included randomized control group designs (all were single subject, within group, or nonequivalent control group designs). A second important finding from this review was that social skills training held more promise than either augmentative communication or conversation skills interventions with effect sizes ranging from 0.42 and 1.10 for the social skills interventions reviewed.

Together, these findings suggest three things. First, there is a need for additional research on the effects of school-based social skills interventions among adolescents with disabilities. The Alwell and Cobb (2009) study was devoted to evaluating interventions designed to improve social skills but yielded only three group designs, none of which were randomized. Second, despite these limitations in the literature, social skills do appear to be associated with adolescents’ academic, behavioral, and emotional adjustment (Konold et al., 2010; Miles & Stipek, 2006; Segrin & Flora, 2000); transition-related skills (Pierson et al., 2008); and postschool outcomes among adolescents with disabilities (Benz et al., 1997, Cameto, 2005; Wagner, 2005). Thus, these skills appear to have broad benefits and may affect student adjustment across multiple developmental domains and contexts. Third, despite the apparent importance of social skills among adolescents with disabilities, these skills may be underdeveloped among students with disabilities (Marder et al., 2003).

Current Study

The current study was undertaken in light of the need for additional, rigorous research on interventions that may improve transition-related skills of adolescents with disabilities. In this study, adolescents with disabilities were randomly assigned (i.e., at the classroom level) to participate in the WAGES curriculum or a business-as-usual control group. We anticipated that adolescents with disabilities exposed to the WAGES curriculum would demonstrate greater gains in vocational outcome expectations, occupational skills, and social skills than would adolescents with disabilities in control classrooms. We further anticipated that fidelity of implementation of the intervention (level of exposure) would affect the magnitude of these effects such that students receiving greater exposure to WAGES would make stronger gains than would students receiving lesser exposure.

Method

Participants

Participating students (N = 222) were recruited from three high schools and 18 classrooms (Grades 9–12). Two of the high schools were located in a large urban environment and the third was located in a small city. All schools were located in the pacific northwestern region of the United States. Students were selected in roughly proportionate numbers from each of these schools (i.e., School 1 = 31%, School 2 = 38%, and School 3 = 31%). All classroom settings were 100% students with disabilities and included “specially designed instruction” (SDI) settings. These classrooms ranged in size between 8 and 21 students and averaged approximately 12 students per class. Overall, 63% of the participants were White and 37% were students of color including African American (10%), Latino (10%), multiracial (8%), Native American (5%), and Asian American (4%). All students had public school identified disabilities including learning disabilities (69%), serious emotional disturbance (8%), autism (7%), intellectual disability (6%), other health impairment (6%), traumatic brain injury (1%), and for 3% of the sample a specific label was not available. Overall, 61% of participating students were males and the average age of the participants was 16.4 years (SD = 1.3).

In Table 1, we provide an overview of the demographic characteristics of students in the intervention and control groups. As shown in the table, these groups were similar to one another on gender, disability, age, and grade characteristics. There was a slightly higher proportion of 11th graders in the intervention group and a slightly higher proportion of 12th graders in the control group. A slightly larger proportion of students in the intervention group was drawn from School 1, and a slightly larger proportion of students in the control group was drawn from School 2.

Table 1.

Select Demographic Characteristics of Youth in Intervention and Control Groups

	Intervention group (n = 122)	Control group (n = 100)
	F (%)	F (%)
Female	48 (39)	39 (39)
Male	74 (61)	61 (61)
High incidence	115 (94)	90 (90)
Low incidence	7 (6)	10 (10)
Age^a	16.37 (1.29)	16.53 (1.34)
9th grade	28 (23)	24 (23)
10th grade	34 (28)	27 (27)
11th grade	28 (23)	17 (17)
12th grade	32 (26)	32 (32)
School 1	43 (35)	25 (25)
School 2	42 (34)	42 (42)
School 3	37 (30)	33 (33)

Means and standard deviations are reported.

Measures

Data were gathered from students and their teachers and consisted of rating scales that evaluated perceptions of vocational outcome expectations, occupational skills, and social skills.

Student Report

Vocational outcome expectations (VOCs)

VOCs (McWhirter et al., 2000) were measured by a six-item scale that assesses students’ general outcome expectations (e.g., “I will be successful in my chosen career”). Responses are provided on a 4-point scale ranging from 1 = strongly disagree to 4 = strongly agree. McWhirter et al. (2000) reported a 9-week test–retest of reliability of .59 and a Cronbach’s alpha of .83 with a sample of high school sophomores. In a separate study of ninth graders, Ali, McWhirter, and Chronister (2005) reported a Cronbach’s alpha of .92. On the current sample, the internal consistency reliability on this scale was .75 at T1 and .85 at T2.

Social Skills Rating System (SSRS)

Student perceptions of social skills were measured by the 39-item self-report form for secondary students. This widely used instrument has excellent psychometric properties (Pierson et al., 2008; Walker et al., 2009) and includes four factors pertaining to cooperation (e.g., “I ask before using other people’s things”), assertion (e.g., “I make friends easily”), empathy (e.g., “I feel sorry for others when bad things happen to them”), and self-control (e.g., “I ignore other children when they tease or call me names”). Student ratings were provided on a 3-point scale ranging from 1 = never to 3 = very often. This widely used instrument has strong psychometric properties (Gresham & Elliott, 1990), and the alpha coefficient on the current sample for the four factors ranged between .77 and .83 at T1, and between .77 and .84 at T2.

Teacher Report

Occupational skills

Students’ occupational skills were assessed by teacher ratings on the 28-item Occupational Skills subscale on the Adaptive Behavior Inventory (Brown & Leigh, 1986). Items on this subscale are rated on a 4-point scale ranging from 0 = student does not perform skill to 3 = student has mastered skill. Items are designed to assess students’ general occupational skills (e.g., “Performs assigned work accurately” and “Continues working until assigned tasks are complete”). The overall instrument contains norms for students with and without disabilities and has strong psychometric properties based on the standardization sample that included approximately 2,300 students with and without disabilities (Brown & Leigh, 1986). In the current study, the alpha coefficients for teacher ratings on the Occupational Skills subscale were .95 at T1 and .97 at T2.

SSRS

Teachers rated students’ social skills with the 30-item teacher report form of the SSRS for secondary students. This measure includes three latent factors (10 items each) pertaining to cooperation (e.g., “Attends to your instructions”), assertion (e.g., “Initiates conversations with peers”), and self-control (e.g., “Receives criticism well”). Teachers provided ratings on a 3-point scale ranging from 0 = never to 2 = very often. This widely used instrument has strong psychometric properties (Gresham & Elliott, 1990; Ogden, 2003), and the alpha coefficients on the current sample ranged from .91 to .92 at T1, and between .92 and .93 at T2.

Fidelity Measures

Absences

Data pertaining to student absences were gathered from a teacher rating of student absences during the project period. In the current study, these data were utilized to evaluate level of exposure to the intervention.

Teacher-rated school engagement, Research Assessment Package for Schools (RAPS)

Teacher perceptions of each student’s school engagement were measured using a brief 3-item teacher rating (Institute for Research and Reform, 1998) of school engagement (i.e., “In my class, this student seems tuned in,” “This student comes to class unprepared [reversed],” and “This student does more than required”). Ratings are provided on a 4-point scale ranging from 1 = not at all true to 4 = very true. The alpha coefficient on the current sample on this measure was .85. In the current study, data pertaining to student engagement were utilized to estimate level of exposure (receipt) to the intervention.

Teacher coverage of the WAGES curriculum

To estimate curriculum coverage, we created a 33-item rating scale that included the title of each lesson plan as each item prompt (e.g., teamwork, communication breakdown) and a response scale for each item that ranged from 0% to 100%. Therefore, teacher ratings on this measure provided an estimate of the extent to which each of the 33 lessons in the WAGES curriculum was taught by each teacher. These data were summed and averaged across all items to provide a “total coverage score” for each classroom. Teachers in the SDI intervention and control conditions completed these ratings. Teachers in the control condition reported an average of 0% coverage, whereas coverage among teachers in the intervention condition averaged 71% (range = 53%–90%).

Procedures

Each school was contacted and the project was discussed with special education case managers at each site. Case managers then discussed the proposed project with special education teachers who volunteered to participate in the study. All participating classrooms were SDI settings. These classrooms were devoted mainly to study skills (i.e., providing students with direct assistance completing work from other classes) and included instruction ranging from individualized support to whole group instruction. Following initial recruitment, baseline data were gathered from students and their teachers. Project personnel attended each classroom and administered student assessments in a group format during one 50-min class period. During the remainder of the same week, additional data were gathered from students who were absent during the initial testing day. Approximately 20% of data were gathered through follow-up testing. Moreover, during the same week, all teachers completed rating scales on students. Teachers were paid US$5 per completed rating.

Once baseline data were gathered, classrooms within each school were randomly assigned to participate in the intervention or control group using the random assignment procedure in SPSS. There were a total of five classrooms in School 1, six classrooms in School 2, and seven classrooms in School 3. For each school, each classroom was assigned a case number and then a random sample of cases was selected using the “select cases” procedure in SPSS. For this procedure, the sample size was specified as “approximately 50%.” For each school, case numbers (classrooms) selected through this procedure were assigned to the intervention condition (n = 10) and unselected cases (classrooms) served as controls (n = 7). All teachers in the intervention condition were provided a free copy of the WAGES curriculum and participated in a paid (US$35 per hour) 2-day training about WAGES. During this workshop, teachers were provided with an overview of curriculum’s conceptual framework, an overview of the broad curriculum goals, and participated in practice sessions implementing specific lesson plans. All the teachers who attended the initial workshop agreed to continue participation in the study.

WAGES (Johnson, Bullis, Benz, & Hollenbeck, 2004) is a job-related social skills curriculum consisting of 33 comprehensive lesson plans in four domains: (a) self-regulation, (b) teamwork, (c) communication, and (d) problem solving. According to the WAGES manual, the curriculum is guided by a cognitive-behavioral orientation and seeks to explicitly and systematically develop students’ skills in identifying situations and problems, generating solutions to problems, and acting on problems through contextually appropriate social skills. In WAGES, these skills are taught through activities that focus on “real-life” social interactions in competitive work settings. Each lesson plan includes (a) an overview of the purpose of the lesson, (b) expected learning outcomes, (c) recommendations for review of prior knowledge, (d) required materials, (e) lesson-specific vocabulary, (f) a detailed activity, (g) a recommended assessment, and (h) a wrap-up/homework recommendation.

Activities within each lesson encourage students to participate cooperatively with other class members by utilizing effective systems of communication and problem solving. Teachers, vocational rehabilitation (VR) counselors, or other school personnel assume a facilitative role during instructional activities spending most of their time interacting with students and providing feedback. For instance, a lesson on self-control places students in either a “venting” or “empathy chair” as they work through a job-related “argument cue card.” The teacher facilitates this role-play and interaction between students, so they learn the importance of managing their emotions with empathy rather than rage. Another facilitated activity involves teamwork and group cooperation where a bean can is filled with pinto beans and maneuvered by a group of six students, each holding a different length of string. The group’s performance is measured by the number of beans that are spilled when they have to maneuver the can so as to fill a small bowl at the halfway point. All activities are generalized to apply to events, attitudes, and interactions that may be encountered in employment settings.

Students in the intervention group received instruction in the WAGES curriculum for between 3 and 4 days per week for approximately 4.5 months (January–May). At the end of May, follow-up data were gathered using procedures that were similar to those used at pretest. Project personnel attended each classroom and administered the student assessments in a group format. Additional data (approximately 30%) were gathered from students who were absent during initial posttesting during the 2 weeks directly following posttesting. Teachers completed posttest ratings for students during the same 2-week period and were compensated (US$7 per measure) for the time they spent completing the ratings.

Analyses

The primary analysis strategy employed in the current study was a MANCOVA. A multivariate procedure was selected because we were interested in evaluating the effects of the intervention on multiple dependent variables including child and teacher reports of career related and social skills adjustment. A covariance model was selected to evaluate these effects after adjusting for potential baseline differences on the dependent variables. Homogeneity of variance was tested with Levene’s tests prior to conducting the MANCOVA, and none of the nine tests were significant.

In addition to the primary analysis, secondary analyses were conducted to evaluate the extent to which fidelity of implementation was associated with changes in student performance on unadjusted outcome variables. To evaluate students’ level of exposure to the intervention, we restricted our sample to students in the intervention condition and conducted two different analyses. First, we used multiple regression analyses to evaluate the extent to which student engagement and student attendance were associated with growth on the target outcomes. Second, to evaluate the effects of teacher-reported coverage of the curriculum on student growth on dependent variables, we conducted a MANOVA.

Results

For descriptive purposes, intercorrelations between all study variables and overall means and standard deviations are provided in Table 2. As would be expected, data gathered from the same source (student or teacher) showed modest correlations particularly within the same rating period (pre or post). Lower associations were observed for cross-rater associations regardless of time point.

Table 2.

Correlation Matrix for All Variables Used in These Analyses

Variable	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
Student ratings (pretest)
1. T1 vocational outcome	—
2. T1 cooperation	.22**	—
3. T1 assertion	.26**	.33**	—
4. T1 empathy	.29**	.50**	.61**	—
5. T1 self-control	.13	.65**	.29**	.42**	—
Teacher ratings (pretest)
6. T1 occupational skills	−.02	.15*	−.04	.02	.19**	—
7. T1 cooperation	−.07	.22**	−.14*	.03	.18**	.75**
8. T1 assertion	.08	.06	.15*	.11	.11	.55**	.49**	—
9. T1 self-control	.04	.10	−.11	−.05	.13	.72**	.75**	.71**	—
Student ratings (posttest)
10. T2 vocational outcome	.25**	.14*	.19**	.15*	.05	−.03	.04	−.01	.03	—
11. T2 cooperation	.06	.26**	.06	.13*	.20**	.02	.14*	−.06	.03	.43**	—
12. T2 assertion	.16*	.16*	.41**	.25**	.09	−.05	−.02	.10	.02	.44**	.44**	—
13. T2 empathy	.12	.20**	.26**	.39**	.14*	.02	.10	.11	.09	.44**	.64**	.60**	—
14. T2 self-control	−.02	.18**	.03	.07	.29**	−.02	.05	−.07	.02	.34**	.70**	.38**	.45**	—
Teacher ratings (posttest)
15. T2 occupational skills	.14*	.16*	.02	.10	.14*	.30**	.29**	.13	.24**	.14*	.12	.08	.17*	.18**	—
16. T2 cooperation	.11	.17**	−.04	.12	.17*	.37**	.41**	.22**	.32**	.10	.21**	.06	.22**	.18**	.76**	—
17. T2 assertion	.14*	.07	.23**	.27**	.15*	.25**	.22**	.46**	.24**	.12	.00	.21**	.26**	.08	.55**	.64**	—
18. T2 self-control	.10	.10	.03	.13*	.13*	.37**	.35**	.34**	.39**	.17*	.16*	.18**	.30**	.16**	.70**	.82**	.74**	—
M	18.81	22.61	21.04	23.45	20.75	42.93	13.58	11.04	12.21	18.87	22.37	21.42	23.72	21.13	49.50	14.03	12.10	13.12
SD	2.66	3.07	3.57	3.54	3.38	14.05	3.98	4.66	4.27	2.73	2.82	3.46	3.35	2.90	16.38	3.81	4.37	3.98

p < .05. **p < .01.

In Table 3, unadjusted means and standard deviation for each group (i.e., intervention and control) at pre- and posttest are provided. The results of the overall MANCOVA for group status (intervention vs. control) were significant, Wilks’s Lambda = 4.70 (9, 203). Results of the univariate tests indicated that after controlling for pretest differences, students in the intervention group reported greater levels of vocational outcome expectations, F = 4.01 (1, 211), p < .05; estimated marginal M^intervention = 19.19 versus M^control = 18.47. The partial η² for this comparison was .02 indicating that the effect was small. On the SSRS, students in the intervention condition reported greater empathy than did students in the control, F = 4.12 (1, 211), p < .05; estimated marginal M^intervention = 24.11 versus M^control = 23.25. The effect size for this comparison was also small (partial η² = .02).

Table 3.

Unadjusted Means and Standard Deviations on Outcome Variables

	Intervention group (n = 122)		Control group (n = 100)
	Pre	Post	Pre	Post
	M	SD	M	SD
Child report
Vocational outcome expectations	18.79 (2.82)	19.17 (2.51)	18.83 (2.46)	18.49 (2.94)
Cooperation	22.70 (3.05)	22.50 (2.56)	22.50 (3.09)	22.22 (3.13)
Assertion	21.12 (3.88)	21.45 (3.68)	20.95 (3.16)	21.39 (3.20)
Empathy	23.79 (3.62)	24.25 (3.22)	23.03 (3.41)	23.07 (3.41)
Self-control	20.95 (3.68)	21.28 (2.76)	20.51 (2.97)	20.94 (3.06)
Teacher report
Occupational skills	43.81 (14.08)	54.08 (14.54)	41.85 (14.01)	43.92 (16.83)
Cooperation skills	13.77 (4.13)	14.75 (3.59)	13.35 (3.80)	13.14 (3.90)
Assertion	11.44 (4.82)	12.96 (4.19)	10.54 (4.42)	11.05 (4.39)
Self-control	12.67 (4.39)	13.68 (3.91)	11.67 (4.07)	12.41 (3.98)

On teacher-rated outcomes, students in the intervention group had greater occupational skills scores, F = 22.88 (1, 211), p < .001; estimated marginal M^intervention = 53.89 versus M^control = 44.15. The effect size for this comparison was in the medium range (partial η² = .10). On the SSRS, teachers rated students in the intervention group as higher in cooperation, F = 8.97 (1, 211), p < .01; estimated marginal M^intervention = 14.66 versus M^control = 13.26 (partial η² = .04). Teachers also rated students in the intervention group as higher in assertiveness, F = 8.93 (1, 211), p < .01; estimated marginal M^intervention = 12.78 than students in the control group, M^control = 11.17 (partial η² = .04).

Fidelity Analyses

Two analyses of fidelity were conducted to evaluate the extent to which fidelity of implementation may have affected treatment outcomes. First, student exposure to the curriculum was evaluated by calculating gain scores on each of the outcomes and then regressing these scores on student absences and teacher ratings of student engagement. For these analyses, absences and engagement were entered on the same block to evaluate the cumulative variance attributable to these predictors. Results indicated that exposure was not predictive of any of the student-rated outcomes (vocational outcome expectations, cooperation, assertion, empathy, or self-control). However, exposure did account for a small but significant portion of the variance in change scores on teacher-rated occupational skills, R² = .07, F(2, 119) = 4.23, p < .05; assertion, R² = .05, F(2, 119) = 3.19, p < .05; and self-control, R² = .06, F(2, 119) = 3.65, p < .05. In all cases, teacher ratings of student engagement were the only predictor that made a unique contribution to these equations (βs = .26, .23, .23 and all ps < .01, respectively).

To evaluate the extent to which teacher coverage was associated with student gains, we created three categories based on overall teacher-reported coverage: Group 1 included students from classrooms wherein teachers reported 70% or greater coverage (n = 50), Group 2 included students in classrooms where between 50% and 69% coverage (n = 56) was reported. The third group (Group 3) included the control condition which averaged 0% coverage (n = 100). One teacher in the intervention group did not provide ratings of coverage so the total number of participants in the intervention group was 106 for this analysis. Gain scores of these groups on all dependent variables were analyzed in a MANOVA using group membership as the fixed factor. Although the overall MANOVA was significant, Wilks’s Lambda = 1.74 (18, 390), p < .05, only one of the univariate tests (occupational skills) was significant, F = 7.56 (2, 203), p < .001. Results of a Bonferroni-adjusted post hoc analysis indicated that this effect was due mainly to differences in gains between students in classrooms that reported greater than 70% coverage (M gain = 14.12) and students in the control condition (M gain = −0.21).

Discussion

The current study was designed to evaluate the effects of the WAGES curriculum on the prevocational and social skills of adolescents with disabilities. Our findings are promising and suggest that WAGES can improve these skills among adolescents with disabilities. The findings contribute to a growing number of correlational (Benz et al., 1997; Cameto, 2005; Pierson et al., 2008) and quasiexperimental (Alwell & Cobb, 2009) studies by showing that transition-related skills can be improved through intervention efforts within the context of schools. As adolescents with disabilities often have poorer social skills than do their nondisabled peers (Marder et al., 2003), and because these skills appear to be meaningfully related to important transition-related skills (Cameto, 2005), finding ways to systematically implement curricula to improve these skills is an important educational objective. Our specific study findings indicated that exposure to the WAGES curriculum improved students’ occupational skills and we observed the largest effect on this outcome. In an effort to more fully understand the practical significance of this effect, we compared the means of students in the intervention and control groups with the original norm sample on this scale (ages 16–18 years old). At pretest, the mean for both groups (M = 43.81 and 41.85, respectively) was at the 50th percentile on the original norm. At posttest, however, the mean score for the intervention group (M = 54.08) had improved to 75th percentile, whereas the mean for students in the control condition (M = 43.92) remained in the 50th percentile on the original norm. Thus, although all the students participating in this investigation were still in high school and had not yet embarked on careers, our findings suggest that exposure to WAGES can have a meaningful impact on students’ occupational skills. Future efforts that examine the relationship between adolescents’ occupational skills and their long-term outcomes are needed to evaluate the extent to which changes in occupational skills during high school affect the long-term employment-related outcomes of youth. Moreover, evaluating the long-term effects of WAGES or other transition-related curricula on employment and other occupation-related outcomes could potentially provide insights regarding effective employment-related prevention and intervention efforts.

In addition to observing a positive effect on occupational skills, a small effect was observed on students’ vocational outcome expectations. As noted in the introduction, most classroom teachers do not have an opportunity to observe students as they make the postschool transition so they must rely on proximal indicators of students’ vocational adjustment, such as vocational outcome expectations and occupational skills. Prior research on SCCT indicates that vocational outcome expectations are positively associated with vocational self-efficacy, career interests, and goal pursuits (Lent et al., 1994; Lent et al., 2008; McWhirter et al., 2000). Therefore, vocational outcome expectations are an important, proximal, skill that can be measured during high school. Our findings are important because this is the first study to investigate an intervention designed to change outcome expectations among students with disabilities. Although our findings in this area were modest, we did observe some growth on this variable among students in the intervention group. This finding is promising and suggests that vocational outcome expectations among students with disabilities can be improved through exposure to the WAGES curriculum.

In addition to the effects observed on students’ prevocational skills, students exposed to WAGES had greater empathy, cooperation, and assertion skills following the intervention. Although the effect sizes for these comparisons were small, our findings contribute new knowledge regarding the effectiveness of WAGES for improving the social skills of adolescents with disabilities within the context of transition. Some researchers have estimated that 90% of job loss is due to poor job-related social skills (N. Elksnin & Elksnin, 2001), and other research has shown that social skills are associated with other transition-related skills (Pierson et al., 2008) as well as post–high school outcomes (Benz et al., 1997; Cameto, 2005). Moreover, some evidence suggests that adolescents with disabilities have poorer social skills than do their nondisabled peers (Marder et al., 2003). Therefore, our findings are important because they suggest that social skills can be improved over a relatively short period of time through the systematic implementation of curricula to teach these skills.

Results of our fidelity analyses suggested two things. First, our findings indicated that students with higher levels of teacher-rated engagement had greater gains on occupational, assertion, and self-control skills according to teacher, but not student ratings. Although this finding may have been due in part to the fact that these data were gathered from the same source (i.e., method variance), the findings also suggest that student engagement in learning activities can affect their growth on those activities. This positive relationship between active engagement and consequent performance makes sense conceptually and similar relationships have been demonstrated in research focusing on the acquisition of academic skills (Greenwood, 1991; Singh, Granville, & Dike, 2002). Therefore, future efforts targeting the acquisition of transition-related skills should investigate strategies for maximizing student engagement in the learning process.

A second important finding that emerged from our fidelity analyses was that there were minimal differences in student gains in classrooms where teachers covered between 50% and 69% of the content of the curriculum and classrooms where teacher-reported coverage was greater than 70%. Although there are a number of possible explanations for this finding including issues related to the outcome variables measured and potential bias on the part of teachers in self-reporting content covered, one possible explanation of this finding is that it may be possible to deliver the WAGES content more efficiently. That is, moderate exposure to the concepts covered in the curriculum may produce effects that are similar to more extensive coverage. Given the increasing demands placed on teachers and other school-based personnel to focus exclusively on academic skill development in core content areas such as reading, writing, and mathematics, it is important to further investigate the possibility that these transition-related skills can potentially be delivered more efficiently because such findings could potentially be useful for arguing for the integration of this and similar curricula within schools.

Implications for VR Counselors

The research conducted here has several implications for VR counselors. The field of special education has long recognized the importance of transition services and supports for adolescents with disabilities but the current evidence base for such supports is limited at best (Test, Fowler, et al., 2009). Furthermore, preparation of special education teachers to deliver such services is uneven with large segments of secondary special education teachers reporting that they are not qualified and/or do not regularly deliver transition-related supports to students (Benitez, Morningstar, & Frey, 2009). VR counselors are well positioned to collaborate with special education teachers in ways that help to fill this void. VR professionals value the importance of providing youth opportunities to develop transition competencies (e.g., providing career planning and counseling, providing career preparation experiences) but may not have the opportunities to develop such skills among their clients (Plotner, Trach, & Strauser, 2012). Therefore, VR counselors should continue to work toward developing and maintaining collaborative relationships with teachers and other school-based personnel because these partnerships can provide VR counselors with opportunities to recommend strategies and curricula that will improve the transition-related experiences of adolescents with disabilities. Utilizing teachers as partners in this process and addressing transition-related competencies early (during high school) have the potential to improve the delivery of VR services because students with stronger skills in areas such as those studied here would likely be more capable of developing initial employment plans, acquiring jobs, and maintaining employment.

In addition to offering suggestions to school professionals, VR counselors are also in the best position to assist youth with disabilities in making meaningful linkages between skills learned in educational settings and real-world experiences in communities. VR professionals can provide youth with access to structured exploration and work experiences that allow them to practice the knowledge and skills learned in curricula, such as WAGES. It is important that youth have successful work experiences and VR counselors can assist youth with disabilities in obtaining job-related experiences that match students’ interests and capitalize on their strengths. To help facilitate collaborative partnerships, school-based personnel need to ensure that VR providers are invited to participate in meetings where transition planning takes place (e.g., Individual Education Program [IEP] team meetings) as well as in the delivery of transition services. Prior research indicates that active collaboration in transition planning and service provision between VR and schools occurs less often than should (Baer, Daviso, Queen, & Flexer, 2011). The implementation of and outcomes gained by the WAGES curriculum is an example of an intervention that would benefit from interagency transition planning and services. The combined resources and expertise of VR and school personnel have the potential to amplify the outcomes of the curriculum summarized in this study.

Limitations

It is important to recognize that this study has several important limitations. First, the sample for this study was relatively small and focused exclusively on classrooms for students with disabilities. Future efforts that include larger and more diverse population of students and efforts in other types of instructional settings should be conducted prior to making broad generalizations about the findings. Second, the current study relied on student and teacher reports to evaluate intervention effects, and some of the instruments used for this purpose were somewhat dated. Although using multisource data is a recommended and valued practice (Kratochwill, Sheridan, Carlson, & Lasecki, 1999), we did not incorporate any direct assessments of student performance in the current study. Future efforts that incorporate direct observations of student behavior or direct measures of student performance are needed. A third limitation of the study is that the unit of analysis in this study was students but classrooms, not students, were assigned to each condition. Because we were working with a relatively small number of classrooms (i.e., 18) and because we had no basis for anticipating large effects, we did not utilize hierarchical linear modeling to evaluate intervention effects. Future efforts that (a) assign students to condition at the individual level or (b) include a greater number of classrooms to evaluate the effectiveness of WAGES using multilevel models with adequate power are needed. A final limitation is that each of the participating schools contained intervention and control classrooms and this is potentially problematic for two reasons. First, teachers were aware of student condition as both conditions were being implemented in the same settings (schools), and we relied on teacher reports for many of the outcomes studied. In an effort to reduce the potential for rater bias on these measures, we gathered pretest data prior to making classroom assignments to study conditions. In addition, we gathered data directly from students because they were a more independent source for data. However, teacher ratings on the posttest (outcome) data may have been affected by their awareness of condition, a problem that could be resolved in future efforts through the use of direct assessments of student knowledge or skills and/or by assigning schools, rather than classrooms within schools, to experimental conditions. A second risk associated with assigning classrooms to different conditions within the same setting is related to spillover or contamination effects. In the current study, we attempted to reduce this possibility by talking directly to teachers about the importance of limiting the activities to participating classrooms, by providing intervention teachers with training, and by restricting the distribution of free copies of WAGES to teachers in the intervention condition. Moreover, we asked teachers to rate the extent to which the curriculum was covered in the intervention and control conditions and based on teacher self-reports, the amount of coverage in the control classrooms averaged 0%. Despite these cautions, future efforts that assign students to condition at the school level and efforts that incorporate classroom observations are warranted.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was made possible by financial support from the Hope Baney fund through the College of Education at the University of Oregon.

References

Affleck

J. Q.

Edgar

Levine

Kortering

(1990). Postschool status of students classified as mildly mentally retarded, learning disabled, or nonhandicapped: Does it get better with time. Education and Training in Mental Retardation, 25, 315–324.

Agostin

T. K.

Bain

S. K.

(1997). Predicting early school success with developmental and social skills screeners. Psychology in the Schools, 34, 219–228. doi:10.1002/(SICI)1520-6807(199707)34:3<219::AID-PITS4>3.0.CO;2-J

Ali

S. R.

McWhirter

E. H.

Chronister

K. M.

(2005). Self-efficacy and vocational outcome expectations for adolescents of lower socioeconomic status: A pilot study. Journal of Career Assessment, 13, 40–58.

Alwell

Cobb

(2009). Social and communication interventions and transition outcomes for youth with disabilities: A systematic review. Career Development for Exceptional Individuals, 32, 94–107. doi:10.1177/0885728809336657

Baer

R. M.

Daviso

Queen

R. M.

Flexer

R. W.

(2011). Disproportionality in transition services: A descriptive study. Education and Training in Autism and Developmental Disabilities, 46, 172–185.

Benitez

D. T.

Morningstar

M. E.

Frey

B. B.

(2009). A multistate survey of special education teachers’ perceptions of their transition competencies. Career Development for Exceptional Individuals, 32, 6–16. doi:10.1177/0885728808323945

Benz

M. R.

Yovanoff

Doren

(1997). School-to-work components that predict postschool success for students with and without disabilities. Exceptional Children, 63, 151–164.

Betz

N. E.

(2007). Career self-efficacy: Exemplary recent research and emerging directions. Journal of Career Assessment, 15, 403–422. doi:10.1177/1069072707305759

Brown

Leigh

J. E.

(1986). Adaptive Behavior Inventory. Austin, TX: Pro-Ed.

10.

Cameto

(2005). Employment of youth with disabilities after high school. In Wagner

Newman

Cameto

Garza

Levine

(Eds.), After high school: A first look at the postschool experiences of youth with disabilities. A report from the National Longitudinal Transition Study–2 (NLTS2) (pp. 5/1–5/21). Menlo Park, CA: SRI International.

11.

Diegelman

N. M.

Subich

L. M.

(2001). Academic and vocational interests as a function of outcome expectancies in Social Cognitive Career Theory. Journal of Vocational Behavior, 59, 394–405. doi:10.1006/jvbe.2001.1802

12.

Elksnin

L. K.

Elksnin

(1996). Strategies for transition to employment settings. In Deshler

D. D.

Ellis

E. S.

Lenz

B. K.

(Eds.), Teaching adolescents with learning disabilities: Strategies and methods (pp. 525–578). Denver, CO: Love.

13.

Elksnin

L. K.

(2001). Adolescents with disabilities: The need for occupational social skills training. Exceptionality, 9, 91–105. doi:10.1207/S15327035EX091&2_7

14.

Fouad

N. A.

Guillen

(2006). Outcome expectations: Looking to the past and potential future. Journal of Career Assessment, 14, 130–142. doi:10.1177/1069072705281370

15.

Greenwood

C. R.

(1991). Longitudinal analysis of time, engagement, and achievement in at-risk versus non-risk students. Exceptional Children, 57, 521–535.

16.

Gresham

F. M.

Elliott

S. N.

(1990). Social Skills Rating System (SSRS). Circle Pines, MN: American Guidance.

17.

Halpern

(1992). Transition: Old wine in new bottles. Exceptional Children, 58, 202–211.

18.

Hamzah

M. S.

Abdullah

S. K.

(2009). Generic skills in personnel development. European Journal of Social Sciences, 11, 684–689.

19.

Hasazi

S. B.

Gordon

L. R.

Roe

C. A.

Hull

Finck

Salembier

G. A.

(1985). A state-wide follow-up on the post high school employment and residential status of students labeled mentally retarded. Education and Training of the Mentally Retarded, 20, 222–234.

20.

Institute for Research and Reform in Education. (1998). Research Assessment Package for Schools (RAPS): Manual for elementary and middle school assessments. Philadelphia: Author.

21.

Johnson

M. D.

Bullis

Benz

M. R.

Hollenbeck

(2004). Working at Gaining Employment Skills (WAGES): A job-related social skills curriculum for adolescents. Longmont CO: Sopris West.

22.

Konold

T. R.

Jamison

K. R.

Stanton-Chapman

T. L.

Rimm-Kaufman (2010). Relationships among informant based measures of social skills and student achievement: A longitudinal examination of differential effects by sex. Applied Developmental Science, 14, 18–34.doi:10.1080/ 10888690903510307

23.

Kratochwill

T. R.

Sheridan

S. M.

Carlson

Lasecki

K. L.

(1999). Advances in behavioral assessment. In Reynolds

C. R.

Gutkin

T. B.

(Eds.), The handbook of school psychology (pp. 350–382). New York, NY: Wiley.

24.

Lent

R. W.

Brown

S. D.

Hackett

(1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45, 79–122. doi:10.1006/jvbe.1994.1027

25.

Lent

R. W.

Sheu

Singley

Schmidt

J. A.

Schmidt

L. C.

Gloster

C. S.

(2008). Longitudinal relations of self-efficacy to outcome expectations, interests, and major choice goals in engineering students. Journal of Vocational Behavior, 73, 328–335. doi:10.1016/j.jvb.2008.07.005

26.

Marder

Wagner

Sumi

(2003). The social adjustment of youth with disabilities. In Wagner

Marder

Blackorby

Cameto

Newman

Sumi

(Eds.), The achievements of youth with disabilities during secondary school: A report from the National Longitudinal Transition Study–2 (NLTS2). Menlo Park, CA: SRI International.

27.

McWhirter

E. H.

Rasheed

Crothers

(2000). The effects of high school career education on social-cognitive variables. Journal of Counseling Psychology, 47, 330–341. doi:10.1037//0022-0167.47.3.330

28.

Miles

S. B.

Stipek

(2006). Contemporaneous and longitudinal associations between social behavior and literacy achievement in a sample of low-income elementary school children. Child Development, 77, 103–117. doi:10.1111/j.1467-8624.2006.00859.x

29.

Mithaug

D. E.

Horiuchi

C. N.

Fanning

P. N.

(1985). A report on the Colorado statewide follow-up survey of special education students. Exceptional Children, 51, 397–404.

30.

Newman

Wagner

Cameto

Knokey

A.-M.

Shaver

(2010). Comparisons across time of the outcomes of youth with disabilities up to 4 years after high school. A report of findings from the National Longitudinal Transition Study (NLTS) and the National Longitudinal Transition Study–2 (NLTS2) (NCSER 2010-3008). Menlo Park, CA: SRI International.

31.

Ogden

(2003). The validity of teacher ratings of adolescents’ social skills. Scandinavian Journal of Educational Research, 47, 63–76. doi:10.1080/00313830308605

32.

Pierson

M. R.

Carter

E. W.

Lane

K. L.

Glaeser

B. C.

(2008). Factors influencing the self-determination of transition- age youth with high-incidence disabilities. Career Development for Exceptional Individuals, 31, 115–125. doi:10.1177/ 0885728808317659

33.

Plotner

A. J.

Trach

J. S.

Strauser

D. R.

(2012). Vocational rehabilitation counselors identified transition competencies: Perceived importance, frequency, and preparedness. Rehabilitation Counseling Bulletin, 55, 135–143. doi:10.1177/0034355211427950

34.

Sanford

Newman

Wagner

Cameto

Knokey

A.-M.

Shaver

(2011). The post-high school outcomes of young adults with disabilities up to 6 years after high school. Key findings from the National Longitudinal Transition study-2 (NLTS2) (NCSER 2011-3004). Menlo Park, CA: SRI International.

35.

Segrin

Flora

(2000). Poor social skills are a vulnerability factor in the development of psychosocial problems. Human Communication Research, 26, 489-514.

36.

Singh

Granville

Dike

(2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. Journal of Educational Research, 95, 323–332. doi:10.1080/00220670209596607

37.

Strahan

E. Y.

(2002). The effects of social anxiety and social skills on academic performance. Personality and Individual Differences, 34, 347–366. doi:10.1016/S0191-8869(02)00049-1

38.

Test

D. W.

Fowler

C. H.

Richter

White

J. A.

Mazzotti

V. L.

Walker

A. R.

. . . Kortering

(2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32, 115–128. doi:10.1177/0885728809336859

39.

Test

D. W.

Mazzotti

V. L.

Mustian

A. L.

Fowler

C. H.

Kortering

Kohler

(2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32, 160–181. doi:10.1177 /0885728809346960

40.

Wagner

(2005). The leisure activities, social involvement, and citizenship of youth with disabilities after high school. In Wagner

Newman

Cameto

Garza

Levine

(Eds.), After high school: A first look at the early postschool experiences of youth with disabilities. A report from the National Longitudinal Transition Study–2 (NLTS2) (pp. 7/1–7/26). Menlo Park, CA: SRI International.

41.

Walker

H. M.

Seeley

J. R.

Small

Severson

H. H.

Graham

B. A.

Feil

E. G.

. . . Forness

S. R.

(2009). A randomized controlled trial of the first step to success early intervention: Demonstration of program efficacy outcomes in a diverse, urban school district. Journal of Emotional and Behavioral Disorders, 17, 197–212. doi:10.1177/1063426609341645