Increasing Collaboration Self-Efficacy to Improve Educational Programming for Students With Autism

Abstract

Collaborative teaming among specialists can enhance educational success by leveraging resources and building capacity to maximize effectiveness. Special educators’ change in perceptions of their efficacy as collaborators in the education of students with autism spectrum disorder (ASD) was studied before and after completing a year-long federally funded graduate certificate program in autism. Self-perceptions of collaborative expertise were examined in eight cohorts of educators over 7 years. Analysis of the results indicated a significant increase in perceptions of efficacy level in six of the cohorts at ≤ .006. The size of the effect of the program on participants’ self-perceptions of their efficacy in working collaboratively with other school personnel was large for seven of the cohorts and medium for one. Most importantly, results from Greenhouse–Geisser analysis showed that the posttest scores on collaboration efficacy level were significantly higher than the pretest scores across all cohorts, and that the pattern of change in pretest to posttest scores was similar across all eight cohorts.

Keywords

collaboration autism spectrum disorder autism self-efficacy personnel development

Autism is a complex disorder that manifests itself through a variety of characteristics and behaviors. The wide range of learning and behavior needs of students with autism spectrum disorder (ASD) requires the involvement and expertise of an array of professionals with specific knowledge and skills. Specific areas of expertise necessary in educating and treating individuals with ASD include behavior management, language and communication, social interaction, executive functioning, self-determination, academics, and life skills. To effectively serve children and youth with ASD in these areas, collaboration among professionals from a variety of disciplines is critical. Collaborative teaming among specialists can enhance educational success by leveraging resources, sharing expertise, and building capacity to maximize effectiveness. Such collaboration improves student outcomes (Kelleher, Riley-Tillman, & Power, 2008), facilitates transitions (Noonan, Erickson, & Morningstar, 2012), and strengthens relationships among service providers (Noonan, McCall, Zheng, & Gaumer Erickson, 2012). However, in the absence of effective collaboration, problems from inconsistent programming and inadequate follow-through may limit student growth and frustrate service providers and family.

Characteristics and Elements of Collaboration

Collaboration is a conceptual construct in which principles of shared respect and responsibility are applied to practice. It is a dynamic process and is mandated by the Individuals with Disabilities Education Improvement Act (IDEA) of 2004 in specific areas of service delivery. In the educational arena, collaboration refers to acts related to consultation among professionals from varied disciplines (e.g., special education teachers, speech pathologists, psychologists). It involves shared planning and decision making, as well as interaction in varied activities (Kelly & Tincani, 2013). Through collaboration, students may be enabled to meaningfully participate in school activities that might otherwise be inaccessible (Lawrence-Brown, 2004), leading to higher levels of achievement and improved student achievement.

Essential elements of successful collaboration have been identified. Some key elements include (a) adequate funding for personnel and resources, (b) commitment to team and students, (c) understanding and respecting cultures of other collaborators’ disciplines and backgrounds, (d) communication among stakeholders, and (e) removing turf issues (Johnson, Zorn, Yung Tam, Lamontagne, & Johnson, 2003). A key characteristic of effective collaboration is that the process is cyclical and involves ongoing information flow and feedback (Ellsworth, 2000). Communication among team members is instrumental to sustaining the flow of information.

This study examined the effectiveness of a federally supported personnel development program in increasing collaboration self-efficacy in educators who were working with students with ASD, through survey self-assessment. To achieve the goal of evaluating the effectiveness of the training, this research asked the following question: Can a graduate level certificate program consisting of a combination of professional development coursework and fieldwork in autism increase in-service teacher perceptions of their self-efficacy in collaborating with other professionals and families?

It would seem the answer to such a question would be an unequivocal “yes.” However, expected results from treatments do not always occur, and the role of science is to test hypotheses and seek empirical answers. One case in point is hormone replacement therapy, which was assumed to have a host of benefits for post-menopausal women. The reverse turned out to be the case (Grady et al., 2002). In addition, many studies are one-time, pre–post designs. This study shares results of a program that was repeated 7 times, each with a different group, making it a unique contribution to the literature in this field.

Self-Efficacy

Participants’ perceptions of their self-efficacy in the collaborative process of educating students with ASD were examined through anonymous questionnaires, which were completed prior to and following a year-long federally funded graduate certificate program in autism. Collaborative efficacy level, defined by nine areas, was examined in eight cohorts of educators over 7 years. These nine areas included (a) managing behavior, (b) collaborating with parents, (c) working with other special educators, (d) assessing levels of academic functioning, (e) creating program change, (f) working with administrators, (g) working with general educators, (h) participating as a team member to assess and plan instruction, and (i) collaborating with related service providers.

Self-efficacy is aligned with social cognitive theory. According to Bandura (1986), individuals judge their abilities in part from information on their physiological states. If they feel tense, anxious, or very fearful, they may judge themselves as unable to perform a task, as the thought of doing so would produce such high levels of discomfort. Conversely, if they judge themselves competent to perform the task, they are likely to experience lower levels of fear and anxiety, and high levels of comfort or ease, increasing the likelihood that they would attempt and succeed at the task.

There is a large body of extant research regarding the relationship between anxiety and performance (e.g., Boyd, Foster, Smith, & Boyd, 2014; Morris, Davis, & Hutchinson, 1981; Mueller, 1992; Seipp, 1991; Zeidner, 2014), and although not as numerous, some studies on the relationship between comfort level and performance have been conducted (Palohiemo & Stenman, 2006). Hastings and Brown (2002) found that staff with perceived higher self-efficacy regarding their ability to deal with difficult student behaviors reported having fewer negative emotional reactions including fear and anxiety. Wilson, Kickul, and Marlino (2007) reported possible connections between entrepreneurial self-efficacy, personal comfort thresholds, and career behaviors.

High perceptions of self-efficacy are associated with better performance (Boyd et al., 2014; Morris et al., 1981; Mueller, 1992; Seipp, 1991; Zeidner, 2014). For example, staff with perceived higher self-efficacy reported having fewer negative emotional reactions, such as fear and anxiety when dealing with difficult student behaviors (Hastings & Brown, 2002). Wilson et al. (2007) reported possible connections between self-efficacy, personal comfort thresholds, and professionals’ behavior. Kimble (2013) found that the number of workshops speech–language pathologists attended was correlated with their perceived self-efficacy in assessing and providing interventions for English language learners and limited English proficient students. Leh (2000) reported that teachers’ participation in a technology course increased their efficacy and confidence in technology use. Margolis (2011) noted that after participating in brief counseling training modules, the majority of graduate students majoring in communicative disorders reported significantly increased perceptions of self-efficacy in counseling clients and their families. In this study, we examined the impact of a year-long Autism Specialists’ Program on participants’ perceived collaboration self-efficacy.

The Autism Specialist Program

By providing a specialist program beyond the basic special education master’s degree, integrated training and practice opportunities were offered, leading to a state endorsement in severe and multiple disabilities, including autism. The 18-credit program was comprised of six courses, including a practicum experience to prepare personnel to work with students with ASD and other related disabilities. Program participants in this graduate certificate program included certified special educators from diverse ethnic, racial, social, and economic backgrounds who were employed by a large urban school district. A major focus of the program was to prepare practicing educators to participate in collaborative teams to improve service delivery and impact. Curriculum was aligned with the advanced professional standards of the Council for Exceptional Children’s Division on Autism and Developmental Disabilities. Courses included diagnosis and treatment in autism and related disorders, consultation and collaboration in autism and developmental disabilities, teaching students with severe disabilities, instructional methods in autism, and practicum in autism and severe disabilities. Collaborative activities, such as action research projects that required partnerships among the graduate students, as well as with other service providers and families, were integrated throughout the training program and practicum experiences.

Candidates

The candidates were special education teachers, related service providers, and administrators who participated in in-service training. Over the course of the first funding period, there were four cohorts with 16 candidates per cohort. In the second funding period, each cohort consisted of 10 candidates. In total, 104 candidates enrolled in the training program in eight cohorts. Approximately a third of the candidates represented minority and underrepresented groups. Their ages ranged from 23 to 55. All students had a master’s degree in special education, and all had experience working with students with ASD. The number of years teaching ranged from 2 years to 20 years. All students in the program were in positions that required collaboration with other team members. Participants were employed in urban high-need schools, working with children and youth in Grades K-12. Within the program, the graduate students were referred to as scholars because they received tuition scholarships.

In light of evidence that collaboration among special educators and related service providers is essential for effective education and intervention in schools, it is especially interesting to note the low levels of perceived efficacy in the participants included in the study, prior to training in autism. The purpose of the study, therefore, was to examine changes in perceived self-efficacy levels with respect to collaborative expertise in delivering educational programming for students with autism, pre and post participants’ engagement in the Autism Specialist Program. Two questions guided the research:

Research Question 1: Was there a significant pretest–posttest difference in perceived self-efficacy with respect to collaboration expertise, and

Research Question 2: What was the size of the effect of program participation on participants’ self-efficacy perceptions with respect to collaboration expertise?

Method

Participants

Participants consisted of the candidates recruited for all cohorts of both the first and second funding cycles of the Autism Specialist Program. Of the 104 participants who attended the program, 94 (90%) completed the Self-Efficacy Study Scale both at the beginning and end of the program. For Cohort 1, 14 (88%) completed both surveys; for Cohort 2, 15 (94%); for Cohort 3, 14 (88%); for Cohort 4, 13 (81%); for Cohort 5, 10 (100%); for Cohort 6, eight (80%); for Cohort 7, 10 (100%); and for Cohort 8, 10 (100%). Although the surveys were distributed as part of the required evaluation component of the grant, verbal consent was sought from participants, who were clearly told that their participation was voluntary. Because of the small number of participants in each of the cohorts, demographic information was not collected, to keep responses anonymous and confidential.

Measure

The Self-Efficacy Study Scale was developed to identify participants’ entry self-efficacy in key areas to be addressed in the Autism Specialists Program, and to measure the effect of the training program on their self-efficacy regarding these skills and competencies.

At the beginning of the first year of the program, the authors, along with two other members of the University’s School of Education, met to develop the Self-Efficacy Study Scale. These content experts developed items that described the skills and competencies that participants were expected to acquire during the training program. The skills and competencies were aligned with Professional Competencies in Autism and Developmental Disabilities of the Division on Autism and Developmental Disabilities of the Council for Exceptional Children. The result was a 41-item scale that lists competencies, which the personnel training program was designed to help participants acquire. The extent to which the items match the course content is a measure of its face validity.

This scale asked participants to respond to the statement, “In working with students with autism or severe disabilities, how comfortable do you feel in . . .,” on a 5-point Likert-type scale of 1 = very uncomfortable, 2 = uncomfortable, 3 = neutral, 4 = somewhat comfortable, and 5 = very comfortable. Four underlying constructs were identified from 35 items: Collaboration, Helping to Improve Students’ Social Skills, Knowledge and Application of Strategies, and Writing and Research. The nine items identified as defining Collaboration Self-Efficacy were (a) managing classroom behavior, (b) collaborating with parents, (c) collaborating with other special educators, (d) assessing students’ levels of academic functioning, (e) working within your school to create program change, (f) collaborating with administrators, (g) collaborating with general educators, (h) participating as a team member in assessing and planning instruction, and (i) collaborating with related service providers. The internal consistency reliability (Cronbach’s alpha) for collaboration self-efficacy across all cohorts was .83 on the pretest and .68 on the posttest. Cronbach’s alpha for the total scale across all cohorts was .96 on the pretest and .95 on the posttest.

Included in the nine items defining collaboration self-efficacy are three that explicitly address specific areas of expertise necessary in educating and treating individuals with ASD: managing classroom behavior, assessing students’ levels of academic functioning, and participating as a team member in assessing and planning instruction. Collaboration with parents, administrators, general and special educators, and related service providers facilitates the practice and effectiveness of the other specific areas of expertise measured under the other constructs of the Self-Efficacy Study Scale.

Administration of survey

The participants were administered the Self-Efficacy Study Scale at the beginning of the program to identify areas in which they most needed to build competencies and skills, and again at the end of the program, to find out what differences resulted from their participation in the program. The surveys were administered either by the program’s research assistant or the independent evaluation consultant who processed the data and prepared the reports.

To ensure confidentiality, and ultimately anonymity, we developed a procedure that allowed us to link the pre- and post-survey results of participants through research identification numbers; a list that linked the names to the identification numbers was not kept. When each new cohort was selected, two surveys were prepared for each person. The two surveys were labeled with the same unique identification number and put in an envelope with the participant’s name written on the outside. At the beginning of the program, each participant received one of these envelopes. They completed one survey for the pretest, then sealed the envelope with the other survey in it, and wrote their name across the seal. At the end of the program, each participant received his or her sealed envelope with the numbered survey. This way, there was no mechanism for linking the name of a participant with a research identification number, but pre- and post-survey responses could be linked.

In April 2006 and December 2006, Cohort 1 participants were administered the Self-Efficacy Study Scale. In November 2006 and December 2007, Cohort 2 participants were administered the scale. Due to preparations for the first cohort and the need to develop the instrument, this group was about 3 months into the program before the first administration of the scale. However, the first administration of the Self-Efficacy Study Scale to Cohort 2 was a true pretest. The scale was administered to Cohort 3 participants in February 2008 and again in December 2008. The scale was administered to Cohort 4 participants in January 2009 and again in December 2009.

The scale was administered to the 10 Cohort 5 participants in January 2009 and again in December 2009, at the same time that it was administered to Cohort 4 participants. The scale was administered to Cohort 6 participants in December 2009 and again in December 2010. The Scale was administered to the 10 Cohort 7 participants in January 2011 and again in December 2011. The scale was administered to the 10 Cohort 8 participants in January 2012 and again in December 2012. Table 1 shows the dates of administration of the Self-Efficacy Study Scale by cohort.

Table 1.

Dates of Administration of the Self-Efficacy Study Scale by Cohort.

Cohort	Pretest date	Posttest date
1	04/2006	12/2006
2	11/2006	12/2007
3	02/2008	12/2008
4	01/2009	12/2009
5	01/2009	12/2009
6	12/2009	12/2010
7	01/2011	12/2011
8	01/2012	12/2012

Analyses

The results are given in three sections: (a) descriptive analyses that show percent changes in participants’ perceived self-efficacy across all cohorts from the beginning to the end of the program, (b) related t statistics to test the statistical significance of the posttest–pretest difference for each of the eight cohorts, and (c) effects sizes for the mean differences between the pretest and posttest scores for each of the eight cohorts. Although Cohorts 4 and 5 were coterminous, because the selection criteria and grade-level focus varied, the decision was made to analyze each group separately.

The decision to analyze the results by cohort was made as an effort to reduce the effect of history that might threaten the internal validity of the treatment. Because each iteration of the program took place over a period of 1 year, during which time the participants remained in their same career position, we felt that analyzing the results by cohort would greatly reduce the probability that something other than the program caused improved collaboration comfort level.

The design was a single group pretest–posttest (Trochim, 2000), repeated 7 times over 7 years, once with each group. Because Cohorts 4 and 5 were coterminous, the surveys were administered to these two groups at the same time. Although there was no control or comparison group, because the process was repeated several times, consistent results “provide a stronger basis for inferring the effects of treatment” (Kazdin, 1998). In addition, in response to two questions on an open-ended exit questionnaire, some participants indicated that the program afforded them collaborative experiences and had an impact on their collaboration with colleagues and parents. The program design required a significant amount of collaboration among participants for its successful completion.

By using the same measure for the pretest and posttest, some amount of conditioning may have occurred. However, this was mitigated by the fact that the time elapsed between pretest and posttest administrations was on average about 10 months, making it highly unlikely that participants would have remembered what they wrote during the pretest, minimizing the effect of conditioning.

Related samples t tests were conducted on the collaboration perceptions of self-efficacy variable for each of the eight cohorts to examine significant differences in perceived self-efficacy between the pretest means of the participants at the start of the program and their posttest means upon completion of the program. Because eight t tests were conducted at α ≤ .05, the value for rejecting each null hypothesis was p < .006 (0.05/8).

Effect sizes were calculated using Cohen’s d_z, which is an appropriate statistic for within-participants design (Cohen, 1988) using the equation

C o h e n ’ s d_{z} = \frac{M_{x}}{σ_{z}} .

where M_z is the mean of difference scores (posttest scores minus pretest scores) and the denominator (σ_z) is the standard deviation of the difference scores.

Results

Descriptive Analyses

As shown in Table 2,¹ there was a significant increase from the beginning to the end of the program in the percentage of participants who reported feeling very comfortable with each of the nine skill areas. There was also a marked shift from the beginning to the end of the program in the percentage of participants who reported feeling neutral, somewhat uncomfortable, or very uncomfortable to their feeling somewhat or very comfortable with each of these skills.

Table 2.

Pre–Post Comparison of Percentage of Scholars Who Rated Themselves Somewhat or Very Comfortable on Collaboration Items.

ItemIn working with students . . . how comfortable do you feel in	% somewhat comfortable		% very comfortable		% somewhat or very comfortable		% moved to comfortable^a	n
	Pre	Post	Pre	Post	Pre	Post	% moved to comfortable^a	n
a) Managing classroom behavior	52	30	32	65	84	96	14	94
b) Collaborating with parents	35	28	38	67	73	95	23	94
c) Collaborating with other special educators	51	12	44	86	95	98	5	94
d) Assessing students’ levels of academic functioning	43	44	21	51	64	95	32	94
e) Working within your school to create program change	30	34	24	59	55	93	38	94
f) Collaborating with administrators	38	27	33	65	71	91	20	93
g) Collaborating with general educators	27	27	29	61	55	87	34	94
h) Participating as a team member in assessing and planning instruction	35	14	43	84	78	98	22	94
i) Collaborating with related service providers	35	5	57	95	91	100	9	92

Note. n = Number responding to item on both the pretest and posttest.

The number and percent of scholars who moved from neutral or uncomfortable to somewhat comfortable or very comfortable with these skills by the end of the program.

Thirty-two percent of the respondents reported feeling very comfortable managing classroom behavior at the beginning of the program and 65% at the end of the program. Fourteen percent moved from neutral or uncomfortable to comfortable.² By the end of the program, 96% of respondents felt somewhat or very comfortable managing classroom behavior.

Similarly, 38% of the respondents felt very comfortable collaborating with parents at the beginning of the program and 67% at the end, with 95% of the participants feeling somewhat or very comfortable collaborating with parents. With respect to collaborating with other special educators, 44% of the respondents felt very comfortable at the beginning of the program and 86% at the end, with 98% of the participants feeling somewhat or very comfortable collaborating with other special educators.

Fewer than a quarter (21%) of the participants felt very comfortable assessing students’ levels of academic functioning at the beginning of the program, but by the end, 51% felt very comfortable, with 95% of the participants feeling somewhat or very comfortable assessing students’ levels of academic functioning.

At the beginning of the program, only 24% of the participants felt very comfortable working within their school to create program change; by the end, 59% were very comfortable. Similarly, 33% of the participants felt very comfortable collaborating with administrators at the start of the program. By the end, it rose to 65%. Accordingly, 29% felt very comfortable collaborating with general educators at the beginning of the program, but 61% reported they did at the end. This rose to 87% when results for participants who felt somewhat or very comfortable collaborating with general educators were combined.

Fewer than half (43%) the number of the participants reported feeling very comfortable participating as a team member in assessing and planning instruction at the beginning of the program. This almost doubled to 84% at the end of the program. And although more than half (57%) of the respondents felt very comfortable collaborating with related service providers at the beginning of the program, the vast majority (95%) were very comfortable at the end.

The posttest mean score for each of the eight cohorts was larger than the pretest mean score, showing improvement on participants’ comfort level for every cohort. These consistent results are illustrated in Figure 1 and shown in Table 3. Cohort 1 had the highest pretest mean (4.25) and the highest posttest mean (4.77). Cohort 6 had the lowest pretest mean (3.53), but Cohort 4 had the lowest posttest mean (4.44). Cohorts 4 and 5 were coterminous. It is worth noting that for every cohort except Cohort 7, the posttest standard deviation is smaller than the pretest standard deviation, indicating a narrowing of spread in the responses, which suggests that the average increase in collaboration comfort level is due to group movement rather than the effect of extreme scores.

Figure 1.

Pretest and posttest estimated marginal means on collaboration self-efficacy by cohort.

Table 3.

Pretest and Posttest Means and Standard Deviations on Collaboration Self-Efficacy Level by Cohort.

Cohort	Pretest		Posttest		n
Cohort	M	SD	M	SD	n
1	4.25	0.83	4.77	0.23	14
2	4.19	0.55	4.76	0.21	15
3	3.88	0.45	4.53	0.32	14
4	3.95	0.58	4.44	0.30	13
5	4.03	0.72	4.72	0.30	10
6	3.53	0.55	4.56	0.41	8
7	3.71	0.42	4.56	0.42	10
8	4.00	0.68	4.70	0.31	10

Selected responses to two questions asked of participants at focus groups toward the end of their experience with the program can help to explain the increase in self-efficacy reported above. Responses to the question, “What opportunities have the program afforded you to enhance your knowledge and skills with respect to working with children and youth with autism and severe developmental disabilities?” included, “I was able to collaborate with my peers and conduct a research-based project in my classroom and present it at a research-based conference,” “collaborating with colleagues,” “the opportunity to collaborate with colleagues from different schools and of different positions,” “positive experience collaborating with my peers,” and “It has given me wonderful peers to collaborate with and access to excellent professors.”

In response to the question, “What impact has the program had on you and your work to date?” answers included statements such as, “strong impact on collaboration and research,” “I expect to be able to mentor other teachers and be a source of information for them when they need help,” and “The program has helped me become more confident as a leader in my school and working with adults.”

Results for Research Question 1: Was there a significant pretest–posttest difference in participants’ perceived self-efficacy with respect to collaboration expertise?

Related samples t tests were conducted on the collaboration efficacy level variable for each of the eight cohorts to examine significant differences between the pretest means of respondents at the start of the program and their posttest means upon completion of the program. At α ≤ .006, all cohorts except Cohorts 1 and 5 (the first cohort of each funding period) reported significantly higher collaboration efficacy after completion of the Autism Specialist Program than before participating in the program. Table 4 contains the mean posttest–pretest difference (M_diff), the standard deviation of the mean difference (SD_diff), and the t value and significance level, as well as the effect size for the posttest–pretest difference for each cohort. Because participants selected for each cohort had varying degrees of experience with collaboration and the number in each cohort was small, we decided to test the null hypothesis of no significant difference in pretest and posttest means as opposed to a directional hypothesis of significant increase in posttest means. Table 5 shows the correlation between the pretest and posttest scores for each cohort.

Table 4.

Results for Related t Tests Analyses on Collaboration Self-Efficacy Level by Cohort.

Cohort	d_z (M_diff ES)	Paired differences on collaboration posttest–collaboration pretest
		M _diff	SD _diff	SE M	Lower limit	Upper limit	95% Confidence interval of the difference
		M _diff	SD _diff	SE M	Lower limit	Upper limit	t	df	p (two-tailed)
1	0.66	0.53	0.79	.21	0.07	0.99	2.49	13	.027
2	1.33	0.57	0.43	.11	0.33	0.81	5.14	14	<.001
3	1.22	0.65	0.53	.14	0.34	0.96	4.56	13	.001
4	1.04	0.49	0.47	.13	0.21	0.78	3.77	12	.003
5	0.95	0.69	0.72	.23	0.17	1.21	3.01	9	.015
6	2.45	1.03	0.42	.15	0.68	1.38	6.94	7	<.001
7	2.25	0.85	0.38	.12	0.58	1.12	7.12	9	<.001
8	1.29	0.70	0.54	.17	0.31	1.09	4.07	9	.003

Table 5.

Paired-Samples Pretest–Posttest Correlations of Collaboration Self-Efficacy Level by Cohort.

Cohort	Number	Correlation	p
1	14	.27	.343
2	15	.69	.004
3	14	.08	.792
4	13	.57	.041
5	10	.20	.571
6	8	.65	.079
7	10	.59	.072
8	10	.63	.05

Cohort 2 reported significantly higher collaboration efficacy after completion of the Autism Specialist Program than before participating in the program, t(14) = 5.14, p < .001. Cohort 3 participants reported significantly higher collaboration efficacy level, t(13) = 4.56, p = .001; Cohort 4 participants reported significantly higher collaboration efficacy level, t(12) = 3.77, p = .003; Cohort 6 participants reported significantly higher collaboration efficacy level, t(7) = 6.94, p < .001; Cohort 7 participants reported significantly higher collaboration efficacy level, t(9) = 7.12, p < .001; and Cohort 8 participants reported significantly higher collaboration efficacy level, t(9) = 4.07, p = .003, after completion of the program than before participating in the program.

In addition, we used within-participants one-way ANOVA (repeated measures ANOVA) to examine the statistical significance in the change in pretest to posttest scores across all cohorts (Maxwell & Delaney, 2004), as well as any time by cohort interactions. We did not use ANCOVA, because we were not interested in the differences among the groups with respect to outcomes. Time, with two levels (collaboration pretest and posttest), was the within-participants factor, and cohort (with eight levels) was the between-participants factor. Results show a statistically significant increase in the participants’ collaboration efficacy level. Results from Greenhouse–Geisser analysis, F(1, 86) = 135.55, p < .001, indicated that there was a main effect for time, meaning that the posttest scores on collaboration efficacy level were significantly higher than the pretest scores across all the cohorts. There was no significant time by cohort interaction: F(7, 86) = 0.99, p = .44, meaning that the pattern of change in pretest to posttest scores was similar across all eight cohorts. Figure 1 illustrates this. The fact that the pattern of change in collaboration efficacy level does not differ as a function of cohort across 7 years supports the T test results that the program made a difference in participants’ perceived efficacy level as collaborators in the education of students with ASD.

Results for Research Question 2: What was the size of the effect of the program on the participants’ self-efficacy level with respect to collaboration expertise?

The size of the effect of the program on the participants’ perceived collaboration self-efficacy level was large for every cohort except Cohort 1. Effect sizes are considered small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8) based on Cohen’s (1988) recommendation. However, as Lakens (2013) noted, even small effects can have large consequences in the real world. Therefore, although the pretest–posttest mean difference was not statistically significant for Cohorts 1 and 5 at α ≤ .006, we decided to calculate the effect sizes. The results indicate a consistency in the trend toward improved collaboration self-efficacy.

Effect sizes for pre–post changes in respondents’ perceived collaboration self-efficacy by cohort were as follows: Cohort 1, d_z = 0.66; Cohort 2, d_z = 1.33; Cohort 3, d_z = 1.22; Cohort 4, d_z = 1.04; Cohort 5, d_z = 0.95; Cohort 6, d_z = 2.45; Cohort 7, d_z = 2.25; and Cohort 8, d_z = 1.29. The size of the effects for Cohorts 1 and 5 indicates that although the differences in pre–post results were not statistically significant, they had practical significance for these groups.

Discussion

The program had strong effects for the participating graduate students in increasing their perceived self-efficacy in working to create program change and collaborating with general educators, with somewhat less in traditional areas of experience, such as managing classroom behavior and collaborating with other special educators. It is unusual in social sciences to have such large program effect sizes. The results of the analyses indicated that, for the participants in each cohort, the personnel training program greatly increased the level of collaboration self-efficacy.

It is noteworthy that the only two cohorts for which the increase in reported collaboration efficacy level was not statistically significant at α ≤ .006 were the first cohort of each funding period: Cohorts 1 and 5. Adjustments made to the program after the first year of each cycle likely contributed to its increased effectiveness. In addition, for Cohort 1, the survey was administered to participants 3 months into the program as opposed to within 1 month for the other cohorts. Changes in perceived collaboration self-efficacy may have already started to occur by the time the survey was administered.

Another issue to consider is that the programs for Cohorts 4 and 5 were coterminous and very similar. This resulted in 26 scholars participating in the program at the same time. The size of the combined group may have been a factor in the program’s effect on collaboration efficacy level. Although large for both groups, the pretest–posttest effect sizes for these two groups were lower than for all other groups except Cohort 1. It may be that the optimum size of a cohort in such a program is 16.

Limitations of the Study

One limitation of the study was the absence of a comparison or control group in the design, which would have reduced the threat to internal validity. This was addressed, in part, by using the pretest as the counterfactual and repeating the study over a period of 7 years. Another limitation of the study was that the survey solicited information on self-efficacy with the various areas of collaboration, but because data were not collected on actual practice, the extent to which efficacy level reflected collaboration practice is unknown. The survey did not request information on the manner in which the participants collaborated: face-to-face, phone, email, webinar, blogs, texting/instant messaging, or video chat (Kelly & Tincani, 2013). In addition, specific examples from participants on how their collaboration changed in each of the areas would have added both depth and breadth to the understanding of their view of collaboration and the extent of the increase in collaboration behavior.

Implications for Further Study, Training, and Practice

Findings from this study demonstrate that special educators’ collaboration perceived self-efficacy level can be enhanced through evidence-based personnel training coursework and fieldwork. Insights into specific training activities that may contribute to gains in perceptions of self-efficacy level could be provided by further examination of course syllabi. Although information was gathered through three focus groups conducted with each cohort regarding participants’ experiences with coursework and supervision, and used to improve the program, more detailed information could be gathered to determine the participants’ opinions regarding the value of particular assignments in increasing their collaboration self-efficacy. Triangulation of data could be accomplished by examining the pre- and post-training questionnaires, examining syllabi and grades on corresponding assignments, and gathering information through focus groups to discern perceived value of assignments in relation to teaching effectiveness. In this manner, information about the value of specific instructional activities and experiences could be collected and used to inform practice, resulting in improved personnel development in autism.

Conclusion

The results of the study support the conclusion that the participants’ reported collaboration efficacy level for educating students with autism was enhanced through the personnel training program. The consistency of findings over a 7-year period suggests that the experiences participants received through the training program increased their perceived self-efficacy level in collaborating with colleagues, parents, and other adults in the process of delivering educational programming for students with autism.

There is the need for future research that would include measures of the extent and manner in which participants collaborated, specific examples of how their collaboration practices changed, and how student outcomes improved because of collaboration.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by two grants from the U.S. Department of Education: PR/Awards H325K052042 and H325K080209.

Notes

References

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Boyd

Foster

Smith

Boyd

W. E.

(2014). Feeling good about teaching mathematics: Addressing anxiety amongst pre-service teachers. Creative Education, 5, 207–217. doi:10.4236/ce.2014.54030

Cohen

(1988). Statistical power analyses for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Ellsworth

J. B.

(2000). Surviving change: A survey of educational change models. Syracuse, NY: ERIC Clearinghouse on Information and Technology.

Grady

Herrington

Bittner

Blumenthal

Davidson

Hlatky

. . . Wenger

(2002). Cardiovascular disease outcomes during 6.8 years of hormone therapy heart and estrogen/progestin replacement study follow-up (HERS II). Journal of the American Medical Association, 288, 49–57. doi:10.1001/jama.288.1.49

Hastings

R. P.

Brown

(2002). Behavioural knowledge, causal beliefs and self-efficacy as predictors of special educators’ emotional reactions to challenging behaviours. Journal of Intellectual Disability Research, 46, 144–150.

Individuals with Disabilities Education Improvement Act of 2004, Pub. L. No. 108-446, 118 Stat. 2647 (2004) (amending 20 U.S.C. §§ 1400 et seq.).

Johnson

L. J.

Zorn

Yung Tam

Lamontagne

Johnson

S. A.

(2003). Stakeholders’ views of factors that impact successful interagency collaboration. Exceptional Children, 69, 195–209.

Kazdin

A. E.

(1998). Research design in clinical psychology (3rd ed.). Boston, MA: Allyn & Bacon.

10.

Kelleher

Riley-Tillman

T. C.

Power

T. J.

(2008). An initial comparison of collaborative and expert-driven consultation on treatment integrity. Journal of Educational and Psychological Consultation, 18, 294–324. doi:10.1080/10474410802491040

11.

Kelly

Tincani

(2013). Collaborative training and practice among applied behavior analysts who support individuals with autism spectrum disorders. Education and Training in Autism and Developmental Disabilities, 48, 120–131.

12.

Kimble

(2013). Speech-language pathologists’ comfort levels in English Language Learner service delivery. Communication Disorders Quarterly, 35, 21–27. doi:10.1177/1525740113487404

13.

Lakens

(2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 1–12.

14.

Lawrence-Brown

(2004). Differentiated instruction: Inclusive strategies for standards based learning that benefit the whole class. American Secondary Education, 32(3), 34–62.

15.

Leh

A. S. C.

(2000). Teachers’ comfort level, confidence, and attitude toward technology at a technology course. In Willis

Price

Wills

(Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference (pp. 343–347). Chesapeake, VA: Association for the Advancement of Computing in Education.

16.

Margolis

R. N.

(2011). Counseling training modules and their effect on the comfort level of graduate students in the provision of counseling to patients with communicative disorders and their families (Master’s thesis). Available from ProQuest Dissertations and Theses database. (UMI No. 1507692)

17.

Maxwell

S. E.

Delaney

H. D.

(2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

18.

Morris

L. W.

Davis

M. A.

Hutchinson

C. H.

(1981). Cognitive and emotional components of anxiety: Literature review and the revised Worry-Emotionality Scale. Journal of Educational Psychology, 73, 541–556.

19.

Mueller

J. H.

(1992). Anxiety and performance. In Smith

A. P.

Jones

D. M.

(Eds.), Handbook of Human Performance: Vol. 3. State and trait (pp. 127–160). London, England: Academic Press.

20.

Noonan

P. M.

Erickson

A. G.

Morningstar

(2012). Effect of community transition teams on interagency collaboration for school and adult agency staff. Career Development and Transition for Exceptional Individuals, 36, 96–104.

21.

Noonan

McCall

Zheng

Gaumer Erickson

A. S.

(2012). An analysis of collaboration in a state-level interagency transition team. Career Development for Exceptional Individuals, 35, 143–154.

22.

Palohiemo

Stenman

(2006). Gender, communication and comfort level in higher level computer science education. In Frontiers in Education Conference Proceedings (pp. 13–18). doi:10.1109/FIE.2006.322486

23.

Seipp

(1991). Anxiety and academic performance: A meta-analysis of findings. Anxiety Research, 4, 27–41. doi:10.1080/08917779108248762

24.

Trochim

(2000). The research methods knowledge base (2nd ed.). Cincinnati, OH: Atomic Dog Publishing.

25.

Wilson

Kickul

Marlino

(2007). Gender, entrepreneurial self-efficacy, and entrepreneurial career intentions: Implications for entrepreneurship education. Entrepreneurship Theory and Practice, 31, 387–406.

26.

Zeidner

(2014). Anxiety in education. In Pekrun

Linnenbrink-Garcia

(Eds.), International handbook of emotions in education (pp. 265–288). New York, NY: Routledge.