The Role of Performance Feedback and Implementation of Evidence-Based Practices for Preservice Special Education Teachers and Student Outcomes: A Review of the Literature

Abstract

Given the importance of evidence-based practices (EBPs) for improving outcomes for students with disabilities, it is key that preservice special education teachers have the opportunity to implement EBPs with high levels of fidelity during their teacher preparation program. For this reason, the authors conducted a systematic review of the literature to answer the question: Does providing performance feedback improve preservice special education teachers’ fidelity of implementation of EBPs and outcomes for students with disabilities? Five studies were found which met inclusion criteria. These studies demonstrated a clear functional relationship between performance feedback and preservice teachers’ increased fidelity to the EBP(s). Across studies, there were mixed effects in student outcomes when preservice teachers increased their fidelity to EBPs. Limitations of the current analysis and the included studies are discussed along with future implications for researchers and practitioners.

Keywords

performance feedback evidence-based practices teacher preparation practices and outcomes outcomes for students with disabilities

Implementing evidence-based practices (EBPs) with fidelity is important because EBPs allow students with disabilities to receive high-quality instruction, likely resulting in better student outcomes and permits special education teachers to efficiently use their instructional and planning time (Fallon, Collier-Meek, Maggin, Sanetti, & Johnson, 2015). In classroom settings, however, implementation fidelity of EBPs is typically low (Cook & Odom, 2013; Stahmer et al., 2015). As a result, the quality of instruction students receive varies, and students with disabilities likely make fewer gains than if they received instruction through well-implemented EBPs as mandated by the Individuals With Disabilities Education Improvement Act (IDEIA; 2004; McLeskey & Billingsley, 2008; Stahmer et al., 2015). It takes significant time, energy, and money to train in-service special education teachers to implement EBP with high levels of fidelity, and even then, not all teachers will implement EBPs (Auld, Belfiore, & Scheeler, 2010; Cook & Odom, 2013; Suhrheinrich, 2011). Given the resources needed to provide effective professional development to in-service special education teachers to use and implement EBPs with high fidelity, as well as the positive effects for students, it is important that preservice teachers are taught to implement EBPs with fidelity. Effective training in EBPs will help increase the likelihood that preservice special education teachers are able to implement EBPs from the start of their teaching careers, and initial research suggests that with changes to teacher preparation practices, preservice teachers coached in EBPs can retain most of the skills over time (Rock et al., 2014; Scheeler, Budin, & Markelz, 2016).

Improving Preservice Special Education Teacher Training in EBPs

Substantial changes have occurred in the preparation of preservice special education teachers since the start of the 21st century (Boe, Shin, & Cook, 2007; Dukes, Darling, & Doan, 2014). Special education teacher preparation programs are now available through increasingly diverse formats (e.g., traditional certification programs, alternative certification programs, face-to-face instruction, and/or online courses), all with different fieldwork requirements and supports (Sindelar, Brownell, & Billingsley, 2010). Roughly, one third of new teachers are being prepared through alternative certification programs which typically provide less coursework and fewer fieldwork experiences than traditional certification programs (Kee, 2012). Simultaneous to the changes in teacher preparation programs, there continues to be a significant shortage of faculty to prepare future special education teachers (West & Hardman, 2012). As a result, the faculty running special education teacher preparation programs are likely overworked or assigned to oversee a large number of practicum students. It is estimated that preservice teachers in fieldwork positions receive feedback from their university supervisor at most once a week but likely on a less frequent basis (Capizzi, Wehby, & Sandmel, 2010).

Given the potentially limited support preservice teachers may receive from university staff, serious consideration should be given as to who is mentoring preservice teachers in their fieldwork placements (Darling-Hammond, 2014). During an intervention to improve preservice teachers’ behaviors using feedback, researchers noted that some preservice teachers “faced philosophical differences in classroom management styles with their cooperating teacher who was evaluating them” (Auld et al., 2010, p. 180). As a result, these preservice teachers were not able to fully implement the targeted strategies. When sending preservice teachers into fieldwork settings, it is imperative that teacher preparation programs understand that school staff may not provide adequate support to preservice teachers attempting to implement EBPs with high levels of fidelity. As a result, it is important that preservice teachers receive feedback and support from coaches or mentors knowledgeable in EBPs (Scheeler et al., 2016).

Research shows that high-quality teacher preparation programs provide preservice teachers with extensive supportive fieldwork opportunities and intensive coursework in relevant content areas and pedagogy (Darling-Hammond, Chung, & Frelow, 2002; Scheeler et al., 2016). The recent trend in teacher preparation programs, however, is the focus on content knowledge (Boe et al., 2007). The decreased emphasis on pedagogy may be a disservice to preservice teachers. Understanding pedagogy is important for teachers to effectively teach content to students; in essence, how to teach is just as, if not more important than, what to teach (Boe et al., 2007). To support preservice teachers in implementing pedagogy and content, teacher preparation programs can turn to an established body of research demonstrating the positive effects between providing preservice teachers targeted feedback and an improvement in their pedagogical practices (Cornelius & Nagro, 2014; Kretlow & Bartholomew, 2010; Rathel, Drasgow, & Christle, 2008). Furthermore, Fallon et al. (2015) determined that performance feedback can be considered an EBP for teachers to improve their treatment fidelity of EBPs.

Despite this body of research, there is limited research looking only at preservice special education teachers and whether (a) performance feedback improves their implementation of EBPs and (b) improved implementation of EBPs impacts student outcomes. Since it is critical that EBPs are correctly implemented, it is also important to evaluate teacher preparation practices and how preservice teachers can be supported to correctly implement EBPs. This literature review sought to extend the current literature through the following research question:

Research Question: Does performance feedback improve preservice special education teachers’ fidelity of implementation of EBPs and outcomes for students with disabilities?

Method

Search Procedures

A comprehensive review of the literature was conducted using ERIC, PsychINFO, PsychARTICLES, and Google Scholar. Given the variety of terms used to describe the concepts being searched, R.A.S. used the following terms in various combinations when searching each database: “feedback,” “coaching,” “mentoring,” “evidence based practice,” “EBP,” “fidelity of implementation,” “single-subject,” “single,” “special education teacher,” “special education,” “special,” “novice,” “preservice,” “preservice teacher, “teachers in training,” “trainee teachers,” “student teacher,” and “prospect teachers.” The references of articles relating to providing feedback to improve preservice special education teachers’ instructional skills were reviewed as well as the references of previously conducted literature reviews related to these topics. The references of the identified studies were reviewed for potential studies as well. Articles were included in the review if they met the following criteria: (a) single-subject research design studies, (b) published in peer-reviewed journals, (c) included feedback intervention results for the preservice special education teacher, and (d) the immediate impact this instruction had on the students with disabilities. Due to the small body of relevant literature, the search was opened geographically, so long as the instructional practices used in the study were identified as an EBP in the United States. Rather than limiting the review to studies which met quality indicators for high-quality research, all studies which met inclusion criteria were included so that the current research base as a whole could be analyzed for the rigor of the research presented. R.A.S. screened all studies to determine the final inclusion of articles in the review.

The first stage of evaluating potential studies for inclusion was to screen study titles and abstracts during the initial database search to see whether they fit with the research questions. During Stage 1, 43 potential studies were identified for review in Stage 2. In the second stage, the “Methods” and “Findings” sections of the potential studies were reviewed to determine inclusion in this review. Twenty potential articles were excluded because either they used a design other than single-subject research design such as randomized control group study (e.g., Wills, Kamps, Fleming, & Hansen, 2016), mixed methods (e.g., Rock et al., 2009), or qualitative (e.g., Ottley, Coogle, & Rahn, 2015) or the article was not peer reviewed (e.g., Conderman et al., 2013). An additional nine articles were excluded because they included nonpreservice special education teachers as participants, such as preservice general education teachers (e.g., Kohler, Crilley, Shearer, & Good, 1997; Mallette, Maheady, & Harper, 1999). Last, nine more studies were excluded because researchers did not include outcomes for students with disabilities (e.g., Coogle, Rahn, & Ottley, 2015; Garland, Holden, & Garland, 2016). Once all criteria were exhausted, five studies met the inclusion criteria for this literature review: Barton, Chen, Pribble, Pomes, and Kim (2013); Barton and Wolery (2007); Morgan, Menlove, Salzberh, and Hudson (1994); Rakap (2017); and Scheeler, McAfee, Ruhl, and Lee (2006) and are summarized in Table 1.

Table 1.

Summary of Interventions.

Study	Participants (coaches/preservice teachers/students)	Feedback model/evidence-based practice(s) targeted	Dependent measures	Results
Barton and Wolery (2007)	One investigator Three graduate special education student teachers 42 children in three different inclusive public preschool classrooms (20 of the children had IEPs)	E-mail feedback with verbatim examples and frequency counts Expansions and behavior-specific praise	Use of expansions and behavior-specific praise Missed opportunities for expansion Students’ frequency of utterances	Functional relationship between e-mail feedback and expansions for all teachers Functional relationship between e-mail feedback and behavior-specific praise for two student teachers Increased utterances for children proximal (1.5 m) from teacher
Barton, Chen, Pribble, Pomes, and Kim (2013)	Four doctoral students in early childhood special education Four graduate preservice special education teachers Eight preschool children with disabilities	Didactic training plus coaching Evidence-based intervention package (contingent imitation, system of least prompts, providing specific praise and appropriate play response when a child is engaged in a target play behavior)	Using contingent imitation Implementing system of least prompts Providing behavior-specific praise Appropriate play response when a child is engaged in a target play behavior Frequency of unprompted pretend play behaviors	Functional relationship between the coaching and implementation fidelity for the preservice teachers Increased unprompted pretend play for the children
Morgan, Menlove, Salzberh, and Hudson (1994)	Three high-performing undergraduate preservice special education teachers Five low-performing undergraduate preservice special education teachers 21 first- to fourth-grade students with disabilities	Peer coaching Direct instruction in Reading Mastery and spelling (two preservice teachers only)	Percentage of effective teaching behaviors in reading sessions Probes of effective teaching behaviors in spelling sessions for two trainees Rate of praise statements Rate of pupil responses Number of lessons mastered by groups of pupils	Functional relationship between the coaching and improved instruction Each classroom teacher reported that the students had mastered the same number of lessons in the program as when she had taught the lessons
Rakap (2017)	One master’s level educational coach Three preservice teachers, juniors in college Three preschool children with disabilities	Training plus coaching EILTs	Percentage of correctly implemented EILTs for the target behavior Generalization of EILTs during mealtime routines Correct unprompted responses by children	Functional relationship between the coaching and improved instruction (including maintenance and generalization) Functional relationship between improved instruction and correct unprompted responses
Scheeler, McAfee, Ruhl, and Lee (2006)	One researcher Five preservice special education teachers, juniors in college 26 students with disabilities, preschool to fifth grade	Live corrective feedback via bug-in-ear Direct Instruction: reading (two teachers), spelling (one teacher), math (one teacher), calendar skills (one teacher)	Teacher’s percentages of completed three-term contingency trials Student’s percentage of correct responses per session Level of satisfaction with feedback provided through wireless technology	Functional relationship between the feedback and improved instruction (including maintenance) Mixed results for the students (demonstrated improvement or no change in target behavior)

Note. IEP = Individualized Education Program; EILTs = embedded instruction learning trials.

Quality of Reviewed Research

Horner et al. (2005) described quality indicators which can be used as guidelines to determine the rigor of single-subject research and determine whether sufficient high-quality research exists to consider an EBP. Overall, the five studies were moderate to high-quality studies of single-subject research and met most of the quality indicators. All studies made three or more attempts to demonstrate the effect of the intervention and had clearly defined baseline and intervention conditions which allowed for potential experimental control to be demonstrated. Visual analysis of the data showed clear differences in the level of the behaviors measured across baseline and intervention conditions, with exception of Barton and Wolery (2007). In the feedback intervention across studies, the independent variable was under the control of the researcher (demonstrated by experimental control), with moderate experimental control demonstrated between preservice implementation of EBPs and improved student outcomes. All studies measured interobserver agreement (IOA) though there were a few average measures below the 80% standard, and Morgan et al. (1994) did not report separate IOA for measures of preservice teacher and student behaviors. Four of the studies collected and reported social validity measures though Morgan et al. (1994) did not. A drawback to these studies is that four of the studies, excluding Rakap’s (2017) study, did not describe participants, particularly students, with replicable details, such as specifics of students’ disabilities or diagnoses, which may have a significant impact on the effectiveness of any given EBP.

Results

Participants

Coaches

Across the five studies, 10 coaches provided feedback to 20 preservice teachers. Coaches ranged in experience level and included PhD-level researchers, (Barton & Wolery, 2007; Scheeler et al., 2006) early childhood special education doctoral students with master’s degrees in early childhood special education, (Barton et al., 2013), undergraduate preservice special education teachers (Morgan et al., 1994), and one coach with a master’s degree in special education and 2 years experience coaching presevice and in-service teachers on the EBP used in this study (Rakap, 2017). Overall, the number of coaches ranged from 1 to 4 (M = 2) per study, and the number of preservice teachers that they coached ranged from 1 to 5 (M = 2) per study.

Preservice teachers

Across the studies, 20 preservice special education teachers participated. All 20 preservice special education teachers were college or university students, eight at the undergraduate level (Rakap, 2017; Scheeler et al., 2006), seven at the graduate level (Barton et al., 2013; Barton & Wolery, 2007), and the academic level of five were not specified (Morgan et al., 1994). Reported ages ranged from 21 to 49, though some studies did not report age. The number of preservice teachers per study ranged from 3 to 5 (M = 4). Across preservice teachers, the number of students taught ranged from 1 to 15 (M = 5).

Students

Across the studies there were 100 student participants. They ranged from pre-kindergarten through fifth grade, and 78% of students had disabilities. The majority of the students (n = 53) were in preschool (Barton et al., 2013; Barton & Wolery, 2007; Rakap, 2017), though 22 did not have disabilities (Barton & Wolery, 2007). Two studies included students from a range of grade levels, for example, preschool through fifth grade (Scheeler et al., 2006) or first through fourth grade (Morgan et al., 1994) across multiple school sites.

Research Design

All studies used a single-subject, multiple-baseline-across-participants research design. In each study, the multiple baseline was across preservice teachers, performance feedback and specific EBPs were the interventions, and preservice teacher implementation fidelity and student outcomes were the dependent variables. Barton and Wolery (2007) added a second intervention during the intervention phase for one preservice teacher because her results were variable and she was not maintaining behaviors. These studies ranged from 5 to 14 weeks.

Independent Variables

In the studies, there were two (or more) independent variables per study as two interventions were simultaneously conducted: feedback to the preservice teacher and the preservice teacher’s implementation of the EBP with the students.

Feedback intervention

Each study provided a different type of feedback to the preservice teacher. Barton and Wolery (2007) used twice weekly e-mail feedback with verbatim examples and frequency counts. Barton et al. (2013) used training plus coaching intervention with preservice teachers. This included a one-time, 1 hour training session followed by visual and verbal feedback to preservice teachers prior to, during, and after each play session. Morgan et al. (1994) had peer coaches who met individually with preservice teachers for 30- to 45-minute sessions twice a week to review videotaped lessons. Rakap (2017) implemented a training plus coaching package to preservice teachers. The training involved the coach individually training the preservice teacher and then coaching involved the coach observing the preservice teacher and student for approximately 20 to 30 minutes then immediately after the observation, debriefed with the preservice teacher for 10 minutes. Scheeler et al. (2006) provided live feedback to preservice teachers solely through bug-in-ear technology during lessons with no written or verbal feedback exchanged.

Student instruction

Across studies, behavior-specific praise and Direct Instruction were the most commonly targeted EBPs. Barton et al. (2013) targeted an evidence-based intervention package of contingent imitation, system of least prompts, behavior-specific praise, and appropriate play response when a child is engaged in a target play behavior, while Barton and Wolery (2007) targeted language expansions and behavior-specific praise. Morgan et al. (1994) focused on Direct Instruction in Reading Mastery for all preservice teachers and spelling for two preservice teachers. Rakap (2017) targeted embedded instruction learning trials (EILTs), also known as three-term contingency learning trials, with individualized target behaviors for each student. Scheeler et al. (2006) focused on Direct Instruction but limited teachers by subject: reading (two teachers), spelling (one teacher), math (one teacher), or calendar skills (one teacher).

Dependent Variables

Implementation of EBP

The dependent variables for preservice teacher feedback were unique to each study. Barton and Wolery (2007) collected frequencies on expansions, behavior-specific praise, and missed opportunities for expansion. Barton et al. (2013) measured frequency of contingent imitation, implementing system of least prompts, providing behavior-specific praise, and appropriate play response when a child engaged in a target play behavior. Morgan et al. (1994) measured percentage of effective teaching behaviors in reading sessions, probes of effective teaching behaviors in spelling sessions for two trainees, and rate of praise statements. Effective teaching behaviors were systematically defined and included using an effective cue or pause and correcting errors (Morgan et al., 1994). Rakap (2017) measured percentage of correctly implemented EILTs for the target behavior and generalization during mealtime routines. Scheeler et al. (2006) measured teachers’ percentages of completed three-term contingency trials which involved the teacher asking a question, the student responding, and the teacher responding to the student’s response. Scheeler et al. (2006) reported the students’ percentage of correct responses per session and the level of satisfaction with feedback provided through wireless technology.

Student outcomes

Frequency of targeted behaviors was the most common result reported for the student outcomes in each study. These included students’ frequency of utterances (Barton & Wolery, 2007), unprompted pretend play behaviors (Barton et al., 2013), and rate of pupil response (Morgan et al., 1994). Morgan et al. (1994) also reported the number of lessons mastered by groups of students for each preservice teacher. Rakap (2017) measured the percentage of unprompted correct responses by the children. Scheeler et al. (2006) measured the change in student responses related to the three-term contingency trials implemented by the preservice teachers.

Procedural Fidelity

In each study, we presented procedural fidelity, implementation fidelity, or treatment integrity of coach training and/or coaching procedures. Barton and Wolery (2007) assessed procedural fidelity by checklist and found 100% compliance. Barton et al. (2013) measured implementation fidelity for all didactic training, coaches’ training procedures, and coaching sessions and was found to be 100% in all areas. Morgan et al. (1994) observed a total of 13% of meetings between coaches and preservice teachers and assessed 14 different procedures, finding 95% fidelity to procedures. Rakap (2017) assessed implementation fidelity by checklist for coach training, preservice teacher training, and 25% of coaching sessions for each participant and found 100% fidelity across all procedures. Scheeler et al. (2006) collected treatment integrity using a checklist and immediacy of verbal feedback for 25% of sessions and found it to be 100%.

IOA

Preservice teacher behaviors

All studies collected and reported IOA. Percentage of sessions within phases with IOA ranged from 18% (Scheeler et al., 2006) to 60% (Scheeler et al., 2006) and mean scores ranged from 62% (Barton & Wolery, 2007) to 100% (Barton & Wolery, 2007; Scheeler et al., 2006). Barton and Wolery (2007) assessed and reported IOA for at least 20% of observations in each experimental condition and ranged from 62% to 100% across all phases, participants, and behaviors. Barton et al. (2013) collected IOA for at least 30% sessions across all participants and conditions with a mean of 95%. Morgan et al. (1994) calculated IOA for 38 reading sessions (37% of total sessions) with a mean of 90.7%. Rakap (2017) measured IOA for at least 33% of sessions across all participants and conditions, for both incident occurrence (mean range = 93%-98%) and component agreement (87%-92%). Scheeler et al. (2006) collected IOA on three-term contingency trials for 18% to 33% of sessions across participants in baseline and intervention and 60% of observations in maintenance. They reported 92% mean agreement across participants’ baseline, mean agreement of 97% across participants for intervention, and 100% agreement in the maintenance phase.

Student behaviors

Percentage of sessions with IOA ranged from 0% (Morgan et al., 1994) to 60% (Scheeler et al., 2006), and the mean scores ranged from 78% to 100% (Barton & Wolery, 2007). Scheeler et al. (2006) reported IOA on correct student responses for 18% to 33% of sessions across participants in baseline and intervention and 60% of observations in maintenance and averaged: 93% (baseline), 95% (intervention), and 98% (maintenance). Barton and Wolery (2007) calculated IOA on utterances expanded (average ranged 90%-100%) and missed opportunities (average ranged from 78%-88%), which were combined to calculate the total child utterances. Barton et al. (2013) assessed IOA on at least 30% of sessions for child behaviors and reported an average of 93% IOA across children, behaviors, and sessions. Rakap (2017) indirectly collected IOA on student behavior as it was the second component of the EILTs but did not report separate IOA for student behaviors. Morgan et al. (1994) did not report separate IOA for student behaviors.

Intervention Results

Feedback intervention

All studies showed a functional relationship between receiving feedback and increased implementation fidelity of EBPs for all preservice special education teachers except one participant who increased expansions but not behavior-specific praise (Barton & Wolery, 2007). Furthermore, Barton and colleagues (2013) found that the preservice teachers generalized the intervention across students, while Rakap (2017) found that preservice teachers generalized the intervention across settings. Scheeler and colleagues (2006) also found that four out of five of their preservice special education teachers maintained the increased levels of implementation fidelity upon a 4-week follow-up, and Rakap (2017) found all preservice teachers maintained or increased in maintenance (range = 1-8 weeks).

Student outcomes

The student outcome results varied by study. Barton and Wolery (2007) noted a moderate effect in increased total utterances but no functional relationship between the percentage of utterances expanded and the intervention. Barton and colleagues (2013) and Rakap (2017) demonstrated a functional relationship between implementing the intervention package with high-fidelity and unprompted pretend play. Morgan and colleagues (1994) observed that the rate of pupil response increased for three out of five preservice teachers and that the cooperating teacher reported that the class mastered about the same number of lessons as when she (the cooperating teacher) taught the class. Scheeler and colleagues (2006) observed an increase in correct student responses for four of five preservice teachers, noting that the students with the lowest baseline increased the most.

Discussion

A review of the literature was conducted to determine whether performance feedback improves preservice special education teachers’ implementation fidelity of EBPs and improves outcomes for students with disabilities. Given the federal mandate to use EBPs and the positive benefits for students with disabilities, preservice special education teachers must have the training and opportunity to implement EBPs with fidelity prior to running their own classroom or caseload.

Across the performance feedback interventions, 19 out of 20 preservice special education teachers improved their implementation fidelity of EBPs (Barton et al., 2013; Barton & Wolery, 2007; Morgan et al., 1994; Rakap, 2017; Scheeler et al., 2006). Furthermore, there was evidence that preservice teachers’ increased fidelity to the EBP(s) resulted in improved outcomes for students with disabilities, though the results varied across studies (Barton et al., 2013; Barton & Wolery, 2007; Morgan et al., 1994; Rakap, 2017; Scheeler et al., 2006). The highest student outcomes were demonstrated by Barton et al. (2013) and Rakap (2017). It is important to note that these studies had a low coach to preservice teacher ratio (1:1 and 1:3), lowest preservice teacher to student ratio (1:1), and highest rate of performance feedback (daily), and Barton and colleagues (2013) also provided feedback before and during sessions. Interestingly, while Barton et al. (2013) had the most comprehensive treatment package of EBPs and targeted general “unprompted pretend play,” Rakap (2017) targeted only very specific behaviors for each student which resulted in a stronger experimental effect during the coaching and maintenance conditions because the students were only building on a specific skill set. This suggests that performance feedback may be effective regardless of whether the targeted EBP is individualized or broad. The studies analyzed meet the quality indicators for single-subject research; however, there are neither enough studies nor participants to determine whether performance feedback improves preservice special education teachers’ implementation of EBPs results in a corresponding improvement in student outcomes is an EBP (Cook & Odom, 2013). There is, however, evidence to suggest that this is a promising practice in teacher preparation.

Potential Threats to Internal Validity

A few potential threats to internal validity can be found across studies. During Morgan and colleagues’ (1994) intervention in Direct Instruction, four fourth-grade students replaced three third-grade students for one of the preservice teachers for the second half of the intervention phase. The researchers were transparent about this change and marked it on the graph. There are significant potential confounding implications because there is no baseline for the new students or for the preservice teacher instructing these students. The researchers did not document a baseline for these students’ behaviors and kept the preservice teacher in intervention, which presents the internal validity concern of not being able to compare the new students’ outcomes with any prior measures. Barton and Wolery (2007) reported student outcomes in their study; however, the preservice teachers were working in inclusive public preschools with a mix of students with and without disabilities. Barton and Wolery (2007) did not indicate whether the student outcomes data refer to students with or without disabilities; therefore, it is unclear that how much preservice teacher instruction targeted students with disabilities versus typically developing students. This point is a concern because the dependent variable was preservice teachers’ expansion of statements for children within 1.5 meters, though no data were collected on which students (with or without disabilities) were close to the preservice teachers. Since there is a high likelihood that the response rate for students with and without disabilities would be different, it is important to know with whom the preservice teachers were working.

Potential Threats to External Validity

A noteworthy threat to external validity is that in two of the studies, a researcher or investigator, rather than a practicum supervisor or mentor teacher, provided the feedback or coached the preservice teacher (Barton & Wolery, 2007; Scheeler et al., 2006). Although the researcher or investigator is likely to be a highly trained expert in special education pedagogy, their characteristics and qualifications were not described in either study. Given their unknown backgrounds, their feedback may be different from that of a practicum supervisor or mentor teacher. Interestingly, across the studies, the individuals providing feedback were doctoral students, researchers, experienced teacher coach, or peers, yet no coaches were explicitly identified as current teacher preparation faculty, practicum supervisors, or cooperating teachers: the individuals most likely to provide preservice special education teachers feedback in most fieldwork experiences. The coach in Rakap’s (2017) study was identified as having experience supervising practicum students and coaching in-service teachers, but her current role was not clear. Morgan et al. (1994) stated that they were interested in the efficacy of peers as coaches to help relieve the additional time and resources practicum supervisors may need to support low-performing preservice teachers, though the quality of feedback from a practicum supervisor versus a peer coach may differ greatly. As a result, the effects of these interventions may be different depending on who is giving the feedback or if the feedback is given in an applied setting by the university supervisor or cooperating teacher and not as part of a research study.

The low coach-to-preservice teacher ratio and time spent with the preservice teachers, across studies, is likely a threat to each study’s external validity as well. Across studies, the ratio of coaches to preservice teachers ranged from one to five with an average of just two preservice teachers per coach. Preservice teachers traditionally receive feedback from their university supervisor on a weekly, biweekly, or less frequent basis (Capizzi et al., 2010). Across the studies, however, coaches observed or provided feedback to the preservice teachers at a rate ranging from 1 to 3 days per week. Given the significant shortages of special education faculty (West & Hardman, 2012), it is highly unlikely that special education practicum supervisors would be overseeing and providing feedback to just two or three students per semester and have enough time to coach them on a weekly or daily basis. Across the studies, there is information on how much time the coaches spent with each preservice teacher; however, further analysis is needed to determine which feedback method would be the most time efficient for the coach while still sufficiently benefiting the preservice teacher and students with disabilities.

Measuring Student Outcomes Through Study Replications

One could make the case that collecting data on student outcomes is not necessary when testing an intervention to improve preservice teacher implementation fidelity of EBPs because EBPs are in large part defined by their effectiveness in improving student outcomes. That said, the research used to establish practices as evidence-based was most likely conducted with experienced teachers and researchers and therefore may not yield the same results when implemented by preservice teachers. Not having the experience of managing their own classroom or caseload, preservice special education teachers may be lacking in certain indirect skills needed to implement EBPs with a high level of fidelity and improve student outcomes. Without conducting research on the effects of both independent variables (performance feedback and improved implementation fidelity of EBPs), it may be difficult to tease out the effects of either intervention on its own. The limited number of studies which addresses this specific issue is likely due to the complex nature and potential confounding variables in research studies with more than one independent variable. Interestingly, two of the five studies analyzed in this review were systematic replications or expansions of previous single-subject research studies with the added component of collecting data on the students’ responses (Barton et al., 2013; Barton & Wolery, 2007). Given that these studies were conducted by the same primary investigator, it is important that additional studies are conducted by independent researchers to better understand the relationship between coaching preservice teachers in EBPs to improve student outcomes and develop the necessary breadth of studies needed to establish an EBP.

Limitations of the Review

There are limitations which may impact the conclusions being drawn in this review. The search terms, while varied to encompass all possible terms used in common practice, may not be completely comprehensive, given the variety of acceptable terms for “preservice teacher” alone. Excluding studies that did not include outcomes for students with disabilities notably reduced the number of studies available for the current review and possibly excluded effective feedback interventions. In addition, this review encompassed a small body of research and included notable variation across studies (e.g., experience of coaches and performance feedback schedule), which may further limit the conclusions drawn.

Implications for Research and Practice

In the field of education, preservice special education teachers receive at best, a moderate level of instruction in pedagogy and support in fieldwork settings (Boe et al., 2007; Capizzi et al., 2010; Darling-Hammond et al., 2002) and receive fewer opportunities to develop skills compared with the preparation programs for other helping professions (Grossman et al., 2009). As a result, novice and in-service teachers may implement EBPs with low or mixed levels of fidelity, which in turn may lower outcomes and limit the future potential for students with disabilities. This analysis of the literature suggests that to implement EBPs with fidelity and improve student outcomes, preservice teachers need ongoing, daily support from mentors trained in EBPs. Given that preservice teachers are charged with the education and care of future generations, it is vital we as a field explore, research, and implement systemic changes in teacher preparation to ensure preservice teachers if we expect to provide students, especially those with disabilities, a high-quality, evidence-based education.

Using performance feedback to improve preservice special education teachers’ implementation of EBPs and improve student outcomes has a small, developing research base to establish it as a promising practice in special education teacher education (Darling-Hammond, 2014; Scheeler et al., 2016). Although researchers and practitioners should be cautious about interpreting the results of such a small body of research, these studies lay the foundation for several “next steps” for researchers. Specifically, it may be beneficial for researchers to conduct several effectiveness studies. Studies may focus on the dosage (frequency) of performance feedback, feedback format (e.g., immediate, virtual, delayed), maintenance of skills over time (e.g. once preservice teachers become in-service teachers), or feasibility of who can be an effective coach in the applied university setting. Future research should also explore the level of EBP implementation fidelity needed to impact student outcomes, as there was a limited functional relationship between preservice teachers’ increased implementation fidelity to EBPs and improvement in student outcomes across the reviewed studies. Additional research should explore potential contributing factors to try and determine whether this limited functional relationship might be due to characteristics experienced teachers have but preservice teachers lack, the level of procedural fidelity needed in the EBPs to achieve positive student outcomes, the quality of the EBPs themselves, or other factors not yet considered (Cook & Odom, 2013). Last, future reviews of this area of research should include other research methodologies (e.g., single-subject, group, qualitative) to develop a more comprehensive understanding of how coaching preservice teachers in EBPs impacts outcomes for students with disabilities as well as considering calculating the effect size of the studies analyzed.

The functional relationship between providing feedback (through a variety of coaches and methods) to preservice special education teachers and the increase in their fidelity of implementation is a promising practice for the future of special education teacher preparation. Given the variability in scope and quality of current special education teacher preparation programs, these studies suggest that the small change of providing targeted feedback can have a significant impact on preservice teachers’ instructional practices. Although further research is needed, the fact that two studies demonstrated seven out of eight preservice teachers maintained their behaviors after the intervention through the maintenance phase is also promising (Rakap, 2017; Scheeler et al., 2006). This finding suggests that preservice teachers can be trained to improve implementation fidelity of EBPs and retain fidelity over time (Scheeler, Bruno, Grubb, & Seavey, 2009). These interventions could lead to first-year teachers implementing EBPs with high levels of fidelity which may also result in improved student outcomes.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Rachel Anne Schles is a doctoral student in the vision studies special education program at the University of Pittsburgh and a Scholar with the National Leadership Consortium in Sensory Disabilities. Her research interests include how students are identified for vision services as well as training pre- and in-service teachers of students with visual impairments to make the connection from data collection to IEP development and implementation for students with visual impairments.

Rachel E. Robertson is an assistant professor of special education at the University of Pittsburgh where she teaches and conducts research in the areas of behavior disorders, autism spectrum disorders, and teacher implementation of positive behavior support.

References

Auld

Belfiore

Scheeler

(2010). Increasing pre-service teachers’ use of differential reinforcement: Effects of performance feedback on consequences for student behavior. Journal of Behavioral Education, 19, 169-183. doi:10.1007/s10864-010-9107-4

*Barton

Chen

Pribble

Pomes

Kim

(2013). Coaching preservice teachers to teach play skills to children with disabilities. Teacher Education and Special Education, 36, 330-339. doi:10.1177/0888406413505113

*Barton

Wolery

(2007). Evaluation of email feedback on the verbal behaviors of pre-service teachers. Journal of Early Intervention, 30, 55-72.

Boe

Shin

Cook

(2007). Does teacher preparation matter for beginning teachers in either special or general education? The Journal of Special Education, 41, 158-170. doi:10.1177/00224669070410030201

Capizzi

Wehby

Sandmel

(2010). Enhancing mentoring of teacher candidates through consultative feedback and self-evaluation of instructional delivery. Teacher Education and Special Education, 33, 191-212. doi:10.1177/0888406409360012

Conderman

Koman

Schibelka

Higgin

Cooper

Butler

(2013, November). Learning strategies for adults with mild disabilities. Paper presented at the Illinois Council for Exceptional Children Conference, Lisle.

Coogle

C. G.

Rahn

N. L.

Ottley

J. R.

(2015). Pre-service teacher use of communication strategies upon receiving immediate feedback. Early Childhood Research Quarterly, 32, 105-115. doi:10.1016/j.ecresq.2015.03.003

Cook

Odom

(2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135-144. doi:10.1177/001440291307900201

Cornelius

Nagro

(2014). Evaluating the evidence base of performance feedback in preservice special education teacher training. Teacher Education and Special Education, 37, 133-146. doi:10.1177/0888406414521837

10.

Darling-Hammond

(2014). Strengthening clinical preparation: The holy grail of teacher education. Peabody Journal of Education, 89, 547-561. doi:10.1080/0161956X.2014.939009

11.

Darling-Hammond

Chung

Frelow

(2002). Variation in teacher preparation: How well do different pathways prepare teachers to teach? Journal of Teacher Education, 53, 286-302. doi:10.1177/0022487102053004002

12.

Dukes

Darling

Doan

(2014). Selection pressures on special education teacher preparation: Issues shaping out future. Teacher Education and Special Education, 37, 9-20. doi:10.1177/0888406413513273

13.

Fallon

L. M.

Collier-Meek

M. A.

Maggin

D. M.

Sanetti

L. M. H.

Johnson

A. H.

(2015). Is performance feedback for educators an evidence based practice? A systematic review and evaluation based on single-case research. Exceptional Children, 81, 227-246. doi:10.1177/0014402914551738

14.

Garland

Holden

Garland

(2016). Individualized clinical coaching in the TLE TeachLivE lab: Enhancing fidelity of implementation of system of least prompts among novice teachers of students with autism. Teacher Education and Special Education, 39, 47-59. doi:10.1177/0888406415600769

15.

Grossman

Compton

Igra

Ronfeldt

Shahan

Williamson

(2009). Teaching practice: A cross-professional perspective. Teachers College Record, 111, 2055-2100.

16.

Horner

Carr

Halle

McGee

Odem

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179. doi:10.1177/001440290507100203

17.

Individuals With Disabilities Education Improvement Act of 2004, Pub. L. No. 108-446 § 118 Stat. 2647. Retrieved from http://www.copyright.gov/legislation/pl108-446.pdf

18.

Kee

(2012). Feelings of preparedness amount alternatively certified teachers: What is the role of program features? Journal of Education, 63, 23-38.

19.

Kohler

F. W.

Crilley

Shearer

D. D.

Good

(1997). Effects of peer coaching on teacher and student outcomes. The Journal of Educational Research, 90, 240-250. doi:10.1080/00220671.1997.10544578

20.

Kretlow

Bartholomew

(2010). Using coaching to improve the fidelity of evidence-based practiced: A review of studies. Teacher Education and Special Education, 33, 279-299.

21.

Mallette

Maheady

Harper

G. F.

(1999). The effects of reciprocal peer coaching on preservice general educations’ instruction of students with special learning needs. Teacher Education and Special Education, 22, 201-216.

22.

McLeskey

Billingsley

(2008). How does the quality and stability of the teaching force influence the research-to-practice gap? A perspective on the teacher shortage in special education. Remedial and Special Education, 29, 293-305. doi:10.1177/0741932507312010

23.

*Morgan

Menlove

Salzberh

Hudson

(1994). Effects of peer coaching on the acquisition of Direct Instruction skills by low-performing preservice teachers. The Journal of Special Education, 28, 59-76. doi:10.1177/002246699402800105

24.

Ottley

J. R.

Coogle

C. G.

Rahn

N. L.

(2015). The social validity of bug-in-ear coaching: Findings from two studies implemented in inclusive early childhood environments. Journal of Early Childhood Teacher Education, 36, 342-361. doi:10.1080/10901027.2015.1100146

25.

*Rakap

(2017). Impact of coaching on preservice teachers’ use of embedded instruction in inclusive preschool classrooms. Journal of Teacher Education, 68, 125-139.

26.

Rathel

J. M.

Drasgow

Christle

(2008). Effects of supervisor performance feedback on increasing preservice teachers’ positive communication behaviors with students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 16, 67-77. doi:10.1177/1063426607312537

27.

Rock

M. L.

Gregg

Thead

B. K.

Acker

S. E.

Gable

R. A.

Zigmond

N. P.

(2009). Can you hear me now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32, 64-82. doi:10.1177/0888406408330872

28.

Rock

M. L.

Schumacker

R. A.

Gregg

Howard

P. W.

Gable

R. A.

Zigmond

(2014). How are they now? Longer term effects of eCoaching through online bug-in-ear technology. Teacher Education and Special Education, 37, 161-181. doi:10.1177/0888406414525048

29.

Scheeler

Bruno

Grubb

Seavey

(2009). Generalizing teaching techniques from university to K-12 classrooms: Teaching preservice teachers to use what they learn. Journal of Behavioral Education, 18, 189-210. doi:10.1007/s10864-009-9088-3

30.

Scheeler

Budin

Markelz

(2016). The role of teacher preparation in promoting evidence-based practice in schools. Learning Disabilities: A Contemporary Journal, 14, 171-187.

31.

*Scheeler

McAfee

Ruhl

Lee

(2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher Education and Special Education, 29, 12-15. doi:10.1177/088840640602900103

32.

Sindelar

Brownell

Billingsley

(2010). Special education teacher education research: Current status and future directions. Teacher Education and Special Education, 33, 8-24. doi:10.1177/0888406409358593

33.

Stahmer

Rieth

Lee

Reisinger

Mandell

Connell

(2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the Schools, 52, 181-195. doi:10.1002/pits.21815

34.

Suhrheinrich

(2011). Training teachers to use pivotal response training with children with autism: Coaching as a critical component. Teacher Education and Special Education, 34, 339-349. doi:10.1177/0888406411406553

35.

West

Hardman

(2012). Averting current and future special education faculty shortages: Policy implications and recommendations. Teacher Education and Special Education, 35, 154-160. doi:10.1177/0888406412444456

36.

Wills

Kamps

Fleming

Hansen

(2016). Student and teacher outcomes of the class-wide function-related intervention team efficacy trial. Exceptional Children, 83, 58-76. doi:10.1177/0014402916658658