Video Performance Feedback and Video Self-Monitoring to Improve Systematic Instruction Implementation for Pre-Service Teachers

Abstract

Although special education teachers are expected to implement evidence-based practices, teachers of students with extensive support needs have reported challenges in identifying and implementing effective practices to meet the unique needs of their students. Systematic instruction has been identified as an evidence-based practice for students with extensive support needs and can be used to support students across a wide range of skill domains. We used an experimental pretest–posttest design to examine the effectiveness of two training approaches, video performance feedback and video self-monitoring, on the implementation of systematic instruction among pre-service special education teachers. Overall, both approaches resulted in improved implementation of systematic instruction by pre-service special education teachers and were viewed as socially valid. We present directions for future research and implications for practice in special education teacher preparation.

Keywords

systematic instruction teacher preparation self-monitoring performance feedback

Special education teachers have reported challenges in identifying and implementing effective practices to meet the unique needs of students with extensive support needs (Agran et al., 2017; Courtade et al., 2015). Students with extensive support needs represent the 1–2% of students who have complex and pervasive learning, communication, and/or adaptive needs. These students are eligible to participate in the state alternate assessment and often receive special education services under the eligibility categories of autism, intellectual disability, and multiple disabilities (Taub et al., 2017). Through systematic research reviews, researchers have identified a number of evidence-based practices (EBPs) that have demonstrated positive effects for this student population (e.g., systematic instruction, functional communication training, technology-aided instruction; Browder et al., 2014; Spooner et al., 2019; Steinbrenner et al., 2020). Not only is the use of EBPs by special education teachers crucial to the success of students with extensive support needs (Brown et al., 2014), but EBPs also have been emphasized in educational law (Every Student Succeeds Act, 2015).

One EBP that is highly effective for students with extensive support needs across a variety of skill domains (e.g., academic, self-help, communication) is systematic instruction (Browder et al., 2014; Spooner et al., 2017). Systematic instruction is based on the principles of applied behavior analysis and uses data to teach personally relevant skills that will generalize across contexts (Brown et al., 2020). First, teachers must define the skill they will be teaching, including identifying whether it is a discrete skill (i.e., single step; e.g., clap hands, say “blue,” touch “3”) or a chained skill (i.e., multistep; e.g., tie your shoes, solve a word problem, write a paragraph). Next, teachers must determine the instructional methods to teach the focus skill. They must determine the context in which the skill will be taught (e.g., individually or in small or whole groups); the response prompting method used to teach the skill (e.g., simultaneous prompting, constant time delay [CTD], system of least prompts [SLP], most-to-least prompting; see Table 1); and the reinforcement, error correction, and fading procedures to be used. Then, the teacher implements the systematic instruction plan, collecting performance data throughout, and finally, reviews the data to make instructional decisions (Browder et al., 2020). In order to be effective in supporting students with extensive support needs, teachers must be able to fluently implement EBPs such as systematic instruction (Kurth et al., 2021; Ruppar et al., 2018).

Table 1.

Response Prompting Strategies.

Response Prompting Strategy	Description	Uses
Constant time delay (CTD)	CTD begins with a 0-s delay session in which the instructor presents the response prompt at the same time as they present the stimulus (e.g., instructional cue). The response prompt provided should be the controlling prompt, or the prompt most likely to elicit the correct response. Once the student begins to anticipate the response, the instructor moves to a delay session, in which they wait a predetermined amount of time (often 3 or 5 s) before prompting the student to provide the correct response.	CTD is typically used to teach discrete tasks but can also be used to teach chained tasks. CTD has been used to teach daily living skills, leisure skills, safety skills, and academic skills, including literacy, math, and science tasks (Collins, 2022).
Simultaneous prompting (SP)	SP begins with a probe session in which the instructor presents the stimulus and waits for the student’s response. No error correction is provided for incorrect responses nor is reinforcement provided for correct responses. Following the probe session is an instructional or training session in which the instructor provides the controlling prompt simultaneously with the stimulus.	Similar to CTD, SP is typically used to teach discrete tasks. SP has been used to teach community skills, daily living skills, leisure skills, safety skills, vocational skills, and academic skills, including computer science, health, math, science, social studies, and writing (Collins, 2022).
System of least prompts (SLP)	SLP incorporates a hierarchy of response prompts, ordered from the least to most intrusive. A common prompt hierarchy moves from verbal to model to physical prompting. The instructor waits a predetermined amount of time (often 3–5 s) before providing the least intrusive response prompt. If the student does not respond correctly to the prompt, the instructor provides the next level of prompting. This continues until the instructor provides the most intrusive controlling prompt.	SLP is typically used to teach chained tasks, particularly those of which the student has some prior knowledge or exposure. SLP has been used to teach daily living skills, leisure and play skills, safety skills, vocational skills, and academic skills, including literacy, math, science, and writing, and communication skills, including listening comprehension and social initiations (Collins, 2022).
Most-to-least prompting (MtL)	MtL prompting also incorporates a hierarchy of response prompts, but they ordered from most to least intrusive. A common prompt hierarchy moves from physical to model to verbal prompting. When first presenting the task, the instructor provides the controlling prompt, often physical, to elicit the appropriate response. After a set number of sessions or once the student begins to anticipate the response, the instructor fades by using a less intrusive prompt. This process of fading continues until the student performs the task independently.	MtL prompting is typically used to teach chained tasks, and particularly those of which the student has no prior knowledge or which may be unsafe to perform incorrectly. MtL has been used to teach daily living skills, leisure skills, and vocational skills (Collins, 2022).

Note. CTD = constant time delay, SP = simultaneous prompting, SLP = system of least prompts, MtL = most-to-least prompting.

Unfortunately, special education teachers may not implement EBPs like systematic instruction with fidelity. Teachers often have a lack of awareness of EBPs, limited access to research on EBPs, and, when access is available, limited time to interpret and translate research (Klingner et al., 2013; Spooner et al., 2017). Additionally, pre-service teachers still report minimal exposure to EBPs for students with extensive support needs in their teacher preparation programs (Hsiao & Petersen, 2019) and are more likely to “trust” EBPs when they are described in popular media, such as a teacher’s blog, rather than a traditionally academic source, such as an academic journal (Hugh et al., 2020). Knight et al. (2019) described similar findings in their investigation of in-service teachers. Additionally, the level of effort required to implement some non-EBPs (e.g., the use of weighted vests or other sensory supports) tends to be more appealing to both in-service and pre-service teachers than more complex, but more effective EBPs such as systematic instruction (Hugh et al., 2020). This is particularly concerning for teachers of students with extensive support needs, as students are less likely to meet their desired outcomes and time and resources are wasted when teachers use practices not based in evidence (Agran et al., 2017).

Although most teacher preparation programs introduce EBPs to pre-service teachers, opportunities to practice and receive feedback on implementation of these practices may be limited (Sayeski et al., 2019). Research has suggested that didactic methods consisting of lecture-based instruction traditionally used in college courses are not the most effective for training pre-service teachers to use EBPs (Brown et al., 2014; Marder & deBettencourt, 2015), and when teachers do not feel adequately prepared to implement EBPs, they are less likely to do so (Knight et al., 2019; Ruppar et al., 2016). As such, it is imperative that teacher preparation programs provide pre-service teachers with the opportunity to practice and receive feedback on using EBPs (Brown et al., 2014; Leko et al., 2015; Marder & deBettencourt, 2015; Schles & Robertson, 2019).

Much research has been conducted with in-service teachers on training strategies to improve implementation of EBPs, including performance feedback and self-monitoring (Brock et al., 2017). Performance feedback is a highly effective training strategy for supporting teachers’ use of EBPs and involves an expert observing teacher behavior and providing specific positive and corrective feedback to increase the accuracy of implementation (Cornelius & Nagro, 2014; Fallon et al., 2015). However, performance feedback delivered by experts is not always feasible based on resources and time available. Addressing this shortcoming, self-monitoring is another approach used to increase implementation of EBPs (Leko et al., 2015; Rispoli et al., 2017) and involves a teacher evaluating their own professional practice with the intent of improving their teaching (Kilbourn, 1991). Self-monitoring has its own drawbacks, requiring that teachers have a comprehensive understanding of the content or practice which they are evaluating.

Despite the large evidence base supporting both performance feedback and self-monitoring for in-service teachers, research exploring these training methods with pre-service teachers has been limited (Artman-Meeker et al., 2017; Leko et al., 2015; Morin et al., 2019; Nagro et al., 2022; Rispoli et al., 2017; Schles & Robertson, 2019). In particular, only a few studies have examined performance feedback and self-monitoring as strategies to prepare pre-service teachers in systematic instruction implementation (e.g., McLeod, 2020; O’Reilly et al., 1992; 1994; Sawyer et al., 2017). O’Reilly et al. (1992, 1994) investigated the relative effectiveness of immediate versus delayed expert performance feedback for pre-service special education teachers on their implementation of systematic instruction in two studies. They found that immediate feedback delivered in field-based settings was generally more effective. Sawyer et al. (2017) implemented performance feedback as a part of a behavioral skills training (BST) package and found that BST improved systematic instruction for all but one pre-service special education teacher. All observations and performance feedback were delivered in-person on campus during role-plays with the researchers. Finally, McLeod (2020) conducted a case study with early childhood pre-service teachers investigating the impact of performance feedback on the implementation of CTD in field-based settings. Participants engaged in video self-monitoring by watching video recordings of their own teaching, writing reflections about strengths and areas for improvement, and subsequently meeting in groups to engage in peer feedback. Findings from this study indicated that implementation fidelity varied across participants (range = 65–100%) with most implementing above 80%.

Although these studies offer preliminary information about the ways in which pre-service teachers learn how to implement systematic instruction, more research is needed to explore further the effectiveness and practicality of video-based performance feedback and video self-monitoring strategies to prepare future teachers to implement systematic instruction with students with extensive support needs. We were interested in investigating the use of video technology as it pertains to performance feedback and self-monitoring strategies because of its historical effectiveness as a reflective tool for in-service teachers (e.g., Morin et al., 2019) and the logistical advantages that reduce the amount of time course instructors must spend observing implementation live and/or in person (Nagro et al., 2022). Therefore, we conducted this investigation to address the following research questions:

(1) Does video performance feedback improve pre-service special education teachers’ implementation of systematic instruction by response prompting system (i.e., CTD, SLP) and skill type (i.e., discrete, chained)?

(2) Does video self-monitoring improve pre-service special education teachers’ implementation of systematic instruction by response prompting system (i.e., CTD, SLP) and skill type (i.e., discrete, chained)?

(3) Does the specific training method (i.e., video performance feedback, video self-monitoring) affect the extent to which pre-service special education teachers’ implementation of systematic instruction improved by response prompting system (i.e., CTD, SLP) and skill type (i.e., discrete, chained)?

(4) Are the training methods socially valid from the perspective of pre-service special education teachers?

Method

We conducted an experimental pretest–posttest design study to examine the effectiveness of (a) video performance feedback and (b) video self-monitoring on the implementation of systematic instruction by pre-service special education teachers enrolled in an undergraduate special education teacher preparation program.

Participants and Setting

The participants in this study were 44 undergraduate pre-service special education teachers who were enrolled in a special education teacher preparation program at a mid-sized university in the Midwest. The special education teacher preparation program was cross categorical in that it offered coursework and field experiences to prepare future special education teachers to support K-12 students with a range of disabilities and support needs. As such, all pre-service teachers enrolled in the program were required to (a) take two courses, one introductory course and one methods course, specific to students with extensive support needs and (b) participate in field-based experiences involving students with extensive support needs. This study was conducted within the methods course that focused on systematic instruction and the application of response prompting systems that fall within the scope of systematic instruction for teaching skills to students with extensive support needs. At the time of the study, all students were in the third year of the program and followed the same course and field-based experience sequence.

We recruited pre-service teachers across two course sections that were offered during the same semester. A total of 44 out of 46 pre-service teachers consented to participate and were assigned randomly to one of two groups: video performance feedback or video self-monitoring. It should be noted that the course instructor did not have knowledge of those who consented to participate until after the semester ended. As such, a graduate assistant naive to the purpose of the study randomly assigned pre-service teachers to one of the two groups. The course instructor, the first author of this study, taught both course sections using the same course materials and activities. At the time of the study, the course instructor was an assistant professor of special education with over 15 years of experience implementing and training others to implement systematic instruction for students with extensive support needs; the course instructor had taught the course multiple times in prior semesters.

As noted earlier, this study took place in the methods course focused on systematic instruction. In addition to assigning chapters in the required course textbook (i.e., Collins, 2012) and related journal articles, the course instructor delivered didactic instruction focused on the procedural elements of each response prompting system and shared video models of the response prompting systems. Pre-service teachers also had opportunities to practice implementing the response prompting systems during class after didactic instruction was delivered while receiving instructor feedback. Although participating pre-service teachers participated in field-based experiences involving students with extensive support needs at the time of the study, study activities did not take place in these settings. Pre-service teachers completed the study activities with a randomly assigned peer from the methods course rather than with a student with extensive support needs in a field-based placement. There were no limitations placed on where and when the study activities were to be completed, which resulted in pre-service teachers completing the study activities at home or in empty learning spaces at the university at any point before assignments were due for class.

Measures

We used measures of implementation fidelity and social validity in this study. To measure the extent to which pre-service teachers in the video performance feedback and video self-monitoring groups implemented the procedural elements of each response prompting system with fidelity, we used an implementation fidelity checklist (see Supplemental Materials) specific to the procedures for each response prompting system (i.e., CTD, SLP) and skill type (i.e., discrete, chained). The implementation fidelity checklists included the following procedural elements common to both response prompting systems: the pre-service teacher (a) delivered an attentional cue, (b) delivered an instructional cue, (c) reinforced correct responses, and (d) delivered error correction for incorrect responses. Specifically, for the CTD implementation fidelity checklist, the instructor recorded whether the pre-service teacher applied the correct response interval during both the 0-second and 5-second demonstrations and the correct predetermined controlling prompt (i.e., verbal prompt for the discrete skill; gesture prompt for the chained skill). For the SLP implementation fidelity checklist, the instructor recorded whether the pre-service teacher applied the response interval and correct prompt levels as defined by the predetermined prompting hierarchy (i.e., indirect verbal prompt → visual prompt → verbal model prompt for the discrete skill; verbal prompt → gesture prompt → model prompt for the chained skill).

To measure the social validity of the video performance feedback and video self-monitoring methods, we asked all participating pre-service teachers to complete an anonymous, online social validity questionnaire at the conclusion of the study. The questionnaire included five items with a Likert-type response scale (1 = strongly disagree and 5 = strongly agree) to measure the perceived appropriateness and effectiveness of the video performance feedback and video self-monitoring methods. The questionnaire also included an open-ended response item to provide participants with an opportunity to clarify prior responses and/or provide additional information about their experiences.

Procedures

Midway through the semester after content related to instructional considerations (e.g., instructional arrangements, instructional trials), phases of learning, data collection, graphing, progress monitoring, and preference and reinforcer assessments had been covered, the course instructor delivered instruction on specific response prompting systems across multiple course sessions. Pre-service teachers also completed a video application assignment that required them to demonstrate systematic instruction through SLP and CTD to teach a peer in the same methods course both a discrete skill (i.e., coin identification) and chained skill (i.e., preparing a snack). For each response prompting system, pre-service teachers (a) submitted an initial video demonstration to the video assessment tool on LiveText^®, (b) either received video performance feedback or engaged in video self-monitoring, and (c) resubmitted the video demonstration to LiveText after having made adjustments to their implementation based on the video performance feedback or video self-monitoring outcomes (see Figure 1). LiveText is a web-based application that was used by the university as a supplemental platform for developing and organizing student assignments, e-portfolios, and assessments. It should be noted that the video application assignment was completed after the course instructor provided in-class instruction on the specific response prompting system; in-class instruction consisted of lecture-based didactic instruction, video models, and in-class practice opportunities with instructor feedback specific to the response prompting system. Prior to the video application assignment, a graduate assistant randomly assigned pre-service teachers to either the video performance feedback group (n = 22) or self-monitoring group (n = 22) regardless of course section. Pre-service teachers only had access to the directions and materials specific to their assigned group (video performance feedback or video self-monitoring) until the conclusion of the study, at which point they received access to materials from the other group.

Figure 1.

Overview of study procedures.

Video performance feedback

Pre-service teachers in the video performance feedback group received feedback from their course instructor on the initial video demonstration submissions. As illustrated in Figure 1, for both CTD and SLP, pre-service teachers in this group submitted an initial video demonstrating the response prompting system to teach a discrete and chained skill. The course instructor subsequently provided detailed performance feedback by embedding time-stamped comments within the videos on LiveText. The feedback was consistent across videos in that the embedded comments always addressed procedural elements that were implemented correctly and those that were not. To ensure that pre-service teachers reviewed the performance feedback before recording another video demonstration for the resubmission phase of the video application assignment, the course instructor required them to submit a summary of their performance feedback; all pre-service teachers assigned to this group submitted an accurate summary of the performance feedback across all initial submissions.

Video self-monitoring

Pre-service teachers in the video self-monitoring group received feedback by completing a self-monitoring checklist for their initial video demonstrations. As illustrated in Figure 1, for both CTD and SLP, pre-service teachers in this group submitted an initial video demonstrating the response prompting system to teach a discrete and chained skill. Next, they accessed an online module with directions for completing video self-monitoring checklists specific to each response prompting system and guided practice that consisted of watching video demonstrations of the response prompting systems, completing corresponding implementation fidelity checklists, and reviewing the answer keys and videos that provided an explanation of the answer key. The online module also included an assessment to measure the pre-service teachers’ ability to correctly use the self-monitoring checklist, with a score of 100% required to move forward with the resubmission. The assessment was included in the online module to ensure that pre-service teachers understood how to accurately complete the checklists, as previous research suggests the possibility of a mismatch between reported and actual implementation (Nagro et al., 2022). A majority of pre-service teachers reached this performance criterion in one or two attempts for both CTD (72.7%) and SLP (95.5%).

Subsequently, pre-service teachers completed self-monitoring implementation fidelity checklists to identify procedural elements that were implemented correctly and those that were not; these checklists were identical to those used by the course instructor for measuring implementation fidelity. To ensure that pre-service teachers engaged in self-monitoring before recording another video demonstration for the resubmission phase of the video application assignment, the course instructor required them to submit a copy of their self-monitoring implementation fidelity checklist; all pre-service teachers submitted their checklists for all initial submissions.

Data Collection and Analysis

The course instructor collected implementation fidelity data for each pre-service teacher across both response prompting systems (i.e., CTD, SLP) and skill types (i.e., discrete, chained). The instructor recorded implementation data using implementation fidelity checklists by viewing each initial and resubmitted video application assignment on LiveText and recording a “+” when the pre-service teacher implemented a procedural element correctly, a “−” when the pre-service teacher incorrectly implemented a procedural element, and a “0” when there was no opportunity to implement a procedural element. We calculated the percentage of procedural elements completed correctly as the total number of elements implemented correctly divided by the total number of elements that should have been implemented and multiplied by 100 to yield a percentage. A secondary coder, a graduate student who was naive to the purposes of the study, reviewed 30% of the video submissions for each group to assess interobserver agreement. Overall agreement was 98% (range: 87–100%). We also collected social validity data at the conclusion of the study. Pre-service teachers completed an anonymous online questionnaire that was available through the course management website.

To analyze the data, we conducted dependent t-tests to compare the initial and resubmission mean scores for both the discrete skill and chained skill within the video performance feedback group and within the self-monitoring group. We also conducted independent t-tests to compare gain scores for both the discrete skill and chained skill across the video performance and video self-monitoring groups. Finally, we calculated descriptive statistics by summarizing ratings to analyze the social validity data.

Results

Our findings indicate that implementation fidelity scores improved for the video performance feedback and video self-monitoring groups across prompting systems (CTD and SLP) and skill type (discrete and chained), with the exception of the discrete skill for the SLP under both the video performance feedback and video self-monitoring conditions. In the sections that follow, we present findings across each research question.

Video Performance Feedback

Table 2 displays the dependent t-test results comparing implementation fidelity scores for the initial and resubmission videos across the CTD (with both discrete and chained skills) and SLP (with both discrete and chained skills) response prompting systems under the video performance feedback condition. Three of the four comparisons were significantly different at α = .05 level. There were significant mean differences in performance for the videos demonstrating both the discrete and chained skill when CTD was implemented. The mean increased by 18.59 in performance demonstrating the discrete skill (t(21) = 4.27, p < .001), and the mean increased by 24.95 in performance demonstrating the chained skill (t(21) = 4.37, p < .001). The effect sizes (i.e., Cohen’s d) were large and the values were greater than 1, indicating that the mean differences in implementation fidelity scores between the initial and resubmission videos were greater than 1 SD. When SLP was implemented, the mean difference in performance was significant for the chained skill, but not significant for the discrete skill. When the chained skill was demonstrated, the mean increased by 4.55 in performance (t(21) = 3.02, p < .01) with a medium effect size (d = 0.59).

Table 2.

Initial and Resubmission Comparison Under Video Performance Feedback Condition: Dependent t-test.

			Initial		Resubmission
		n	M	SD	M	SD	M diff	95% CI	df	t	Cohen’s d
CTD	Discrete	22	71.41	18.03	90.00	13.24	18.59	9.53, 27.65	21	4.27***	1.18
CTD	Chained	22	52.68	23.40	77.64	22.53	24.95	13.09, 36.82	21	4.37***	1.09
SLP	Discrete	22	91.95	11.32	95.00	8.12	3.05	−7.74, 1.65	21	1.35	0.31
SLP	Chained	22	91.41	8.78	95.95	6.28	4.55	1.41, 7.68	21	3.02**	0.59

			Initial		Resubmission
		n	Skewness	Kurtosis	Skewness	Kurtosis
CTD	Discrete	22	−0.159	1.966	−0.942	2.514
CTD	Chained	22	0.189	1.851	−1.119	3.544
SLP	Discrete	22	−1.555	4.807	−1.439	3.832
SLP	Chained	22	−1.308	4.660	−1.847	5.472

**p < .01, *** p < .001. CTD = constant time delay, SLP = system of least prompts, M diff = mean of score difference, CI = confidence interval.

Video Self-Monitoring

Table 3 displays the dependent t-test results comparing implementation fidelity scores for the initial and resubmission videos across CTD (with both discrete and chained skills) and SLP (with both discrete and chained skills) under the video self-monitoring checklist condition. Three of the four comparisons were significantly different at α = .05 level. There were significant mean differences in performance for the videos demonstrating both the discrete and chained skill when CTD was implemented. The mean increased by 20.73 (d = 1.32) in performance demonstrating the discrete skill (t(21) = 5.26, p < .001), and the mean increased by 21.13 (d = 0.97) in performance demonstrating the chained skill (t(21) = 5.49, p < .001). The effect sizes for both mean differences were large. When SLP was implemented, the mean difference in performance was significant for the chained skill, but not significant for the discrete skill. When the chained skill was demonstrated, the mean increased by 6.23 in performance (t(21) = .01, p < .05) with a medium effect size (d = 0.60).

Table 3.

Initial and Resubmission Comparison Under Video Self-Monitoring Condition: Dependent t-test.

			Initial		Resubmission
		n	M	SD	M	SD	M diff	95% CI	df	t	Cohen’s d
CTD	Discrete	22	67.23	16.97	87.95	14.41	20.73	12.53, 28.92	21	5.26***	1.32
CTD	Chained	22	53.46	22.67	74.59	20.68	21.13	13.13, 29.14	21	5.49***	0.97
SLP	Discrete	22	94.68	9.08	93.73	8.05	−0.95	−4.56, 2.65	21	−0.55	0.11
SLP	Chained	22	87.55	11.74	93.77	8.80	6.23	1.54,10.92	21	0.01*	0.60

			Initial		Resubmission
		n	Skewness	Kurtosis	Skewness	Kurtosis
CTD	Discrete	22	−0.032	2.522	−1.077	3.237
CTD	Chained	22	−0.187	2.736	−1.172	3.422
SLP	Discrete	22	−2.127	6.906	−1.608	5.268
SLP	Chained	22	−0.554	1.924	−1.967	6.366

*p < .05, *** p < .001. CTD = constant time delay, SLP = system of least prompts, M diff = mean of score difference, CI = confidence interval.

Comparison of Video Performance Feedback and Video Self-Monitoring

Independent samples t-tests were conducted to compare the difference in performance gains between the video performance feedback and video self-monitoring checklist groups for videos demonstrating both the discrete and chained skills using CTD and SLP. Results of independent samples t-tests in Table 4 show that there were no significant differences in gains for either skill type (discrete or chained) regardless of whether CTD or SLP was used. The mean differences between conditions were all nonsignificant, p > .05. In other words, the training conditions (i.e., video performance feedback, video self-monitoring) did not impact implementation fidelity scores regardless of the response prompting method.

Table 4.

Comparison of Gain Scores for Video Performance Feedback and Video Self-Monitoring Conditions: Independent t-test.

		Condition						95% CI for Mean Difference
		Video Performance			Video Self-Monitoring
		M	SD	n	M	SD	n		df	t	Cohen’s d
CTD	Discrete	18.59	20.43	22	20.72	18.48	22	−9.72, 13.99	42	0.36	0.11
CTD	Chained	24.95	26.76	22	21.13	18.05	22	−17.77, 10.12	36.83	−0.56	0.17
SLP	Discrete	3.05	10.58	22	−0.95	8.13	22	−9.74, 1.74	42	−1.41	0.42
SLP	Chained	4.55	7.06	22	6.23	10.57	22	−3.79, 7.15	42	0.62	0.19

			Video Performance		Video Self-Monitoring
		n	Skewness	Kurtosis	Skewness	Kurtosis
CTD	Discrete	44	−0.077	2.207	−1.031	3.016
CTD	Chained	44	0.008	2.256	−1.118	3.480
SLP	Discrete	44	−1.819	5.741	−1.505	4.538
SLP	Chained	44	−0.911	2.798	−2.088	7.233

Note. CTD = constant time delay, SLP = system of least prompts, CI = confidence interval.

Social Validity

Social validity results are presented in Table 5. Overall, pre-service teachers in both groups either agreed or strongly agreed that video performance feedback and video self-monitoring checklists were appropriate and effective for preparing pre-service special education teachers. Pre-service teachers from the video performance feedback group indicated that the time-stamped feedback provided clear and concrete areas for improvement that guided their implementation for the resubmission videos. Pre-service teachers from the self-monitoring group identified the checklist as a helpful tool for self-reflection and identifying areas for improvement, though two noted that completing the checklist was time consuming and that additional guidance for completing the checklists would have been helpful.

Table 5.

Social Validity Results.

Item	Video Performance Feedback (%)					Video Self-Monitoring Checklist (%)
Item	Strongly disagree	Disagree	Neutral	Agree	Strongly agree	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
This approach was an appropriate instructional method for teaching pre-service teachers systematic instruction prompting procedures.	0	0	0	50.0	50.0	0	0	0	42.9	57.1
Most pre-service teachers would find this approach to be an appropriate instructional method for teaching systematic instruction prompting procedures.	0	0	0	44.4	55.6	0	0	0	52.4	47.6
This approach would be appropriate for a variety of college courses.	0	0	0	44.4	55.6	0	0	4.8	33.3	61.9
I would suggest the use of this approach in other college courses.	0	0	0	50.0	50.0	0	0	0	47.6	52.4
This approach improved my performance in delivering systematic instruction prompting procedures.	0	0	0	27.8	72.2	0	0	0	38.1	61.9
This approach did not result in negative side effects for me.	0	0	0	44.4	55.6	6.5	0	6.5	30.0	57.0

Note. Likert-type scale use (1 = strongly disagree, 2 = disagree, 3 = neutral, 4= agree, 5 = strongly agree).

Discussion

The purpose of this study was to examine pre-service special education teachers’ implementation of systematic instruction, an EBP for students with extensive support needs. This study analyzed the effectiveness of (a) video performance feedback and (b) video self-monitoring on the implementation of two response prompting systems, CTD and SLP, among pre-service teachers enrolled in an undergraduate special education teacher preparation program. Given that opportunities to implement EBPs and receive feedback may be limited in some teacher preparation programs (Sayeski et al., 2019) and the specific need for knowledge and skill in systematic instruction among novice special education teachers who educate students with extensive support needs (Ruppar et al., 2018), it is critical to explore effective strategies to prepare future special education teachers to implement systematic instruction. Our findings extend the literature on special education teacher preparation in several ways. Only a few studies have examined training methods for preparing pre-service special education teachers to implement response prompting systems (McLeod, 2020; O’Reilly et al., 1992; 1994; Sawyer et al., 2017), and among these studies, video performance feedback and video self-monitoring strategies have not been compared in terms of effectiveness and feasibility.

Key Findings and Implications

Findings from this study have several practical implications for teacher preparation programs focused on preparing future special education teachers to implement EBPs for students with extensive support needs. We found that pre-service teachers experienced improvements in implementation across response prompting systems and skill type, with the exception of the discrete skill for SLP, which may be attributed to the fact that many pre-service teachers were already implementing at high levels during the initial SLP video submission. We also found there were no significant differences between the two approaches, suggesting that both strategies may be effective in preparing pre-service special education teachers to implement systematic instruction. These findings are consistent with previous research examining performance feedback and self-monitoring to support pre-service teachers’ implementation of systematic instruction (e.g., O’Reilly et al., 1992; 1994; Sawyer et al., 2017). Given the effectiveness of these approaches for in-service teachers (Brock et al., 2017; Fallon et al., 2015; Rispoli et al., 2017) and emerging evidence for pre-service teachers (Cornelius & Nagro, 2014; Schles & Robertson, 2019), we encourage university course instructors to consider these approaches as they plan practice and application opportunities that align with course content focused on response prompting systems.

In considering these approaches, instructors will need to determine whether they are contextually appropriate by weighing out the advantages and disadvantages of each approach. For example, although video self-monitoring can be completed independent of the instructor, it can be time intensive for pre-service teachers and requires prior understanding of the procedures. In fact, a few pre-service teachers from the current study included feedback in the social validity assessment suggesting that pre-service teachers will require an in-depth understanding of how to complete the self-monitoring checklist for it to be effective, and therefore, training from the instructor will likely be necessary. Likewise, pre-service teachers’ perceptions of their implementation may be different than their actual implementation (Nagro et al., 2022). Video performance feedback, on the other hand, allows for detailed feedback from the instructor who has expertise in the subject matter, though this can be time intensive for the instructor and, if using video performance software available through the university (e.g., LiveText, GoReact^®), requires foundational knowledge to effectively navigate and deliver feedback through the software. Regardless, the use of video technology in supporting pre-service teachers as they acquire EBPs such as systematic instruction can relieve many of the logistical concerns of in-person feedback and supervision.

Both video performance feedback and video self-monitoring are advantageous in that they can be used to encourage additional practice and evaluation outside of the university classroom, including directly supporting and delivering systematic instruction to students with extensive support needs in field-based placements. Likewise, video performance feedback and video self-monitoring might serve as viable alternatives to in-person practice and feedback for courses delivered online, including courses that have switched to an online format due to the ongoing COVID-19 pandemic. Another noteworthy aspect of the current study is that pre-service teachers in both groups video recorded their implementation and reflected on their performance. By encouraging pre-service teacher self-reflection of their implementation through the use of video (e.g., McLeod, 2020), teacher preparation programs can build an important foundation for self-evaluation and professional growth, practices that are characteristic of teacher leaders (Hunzicker, 2017; Ruppar et al., 2018) and often required of student performance assessments in teacher preparation programs (e.g., edTPA^®). Ruppar et al. (2018) emphasized the importance of developing positive teacher orientations in teacher preparation programs focused on students with extensive support needs, specifically noting that expert special education teachers have the “orientation of lifelong learning, which [requires] critical thinking to analyze current situations and continually search for information to improve their practice” (p. 416).

Although video resubmission performance scores reflected an overall improvement in implementation in most cases, our findings suggest that some pre-service teachers continued to perform at inadequate levels, as has been the case in previous research (McLeod, 2020). Specifically, the resubmission scores for CTD and the chained skill were low for both the video performance feedback (M = 77.64) and self-monitoring (M = 74.59) groups. This finding raises important questions about the quality and quantity of support necessary to achieve acceptable levels of implementation fidelity when teaching pre-service teachers complex response prompting systems. It is quite possible that additional practice opportunities and feedback, whether delivered through self-monitoring or by the instructor, would have improved student implementation fidelity to acceptable levels. Perhaps providing video performance feedback until a certain performance criterion is achieved followed by video self-monitoring would yield more substantial and maintained improvement in implementation. It also might be the case that some pre-service teachers may require more individualized support based on their performance scores (e.g., follow-up coaching focused on specific procedural elements; Rispoli et al., 2017; Wood et al., 2016). Course instructors will need to carefully monitor pre-service teacher performance to make informed data-based decisions concerning the additional support students might require to fully master the skill.

Limitations and Future Research

Although our findings provide important preliminary information concerning the effectiveness of video performance feedback and video self-monitoring in relation to teacher preparation in systematic instruction, there are a few limitations that should be considered that can inform future research directions. Interpretation of our findings may be limited given that the study was conducted with pre-service teachers in one teacher preparation program who were enrolled in two sections of the same special education course. An examination of these training practices across different teacher preparation programs and courses will be needed as programs and courses differ in a number of ways, including program size, program type (e.g., cross categorical vs. program focused on students with extensive support needs), course delivery format (e.g., in-person, online, hybrid), and available field-based experiences, among others.

Although we found that both training strategies generally were effective, the order in which the response prompting systems were introduced (see Figure 1) may have contributed to higher implementation scores for the SLP submissions. Because SLP shares several common procedural elements with CTD (e.g., attentional cue, instructional cue, reinforcement contingent on correct responding), it is possible that pre-service teachers mastered these procedural elements by the time they submitted the first SLP video, thereby inflating their initial performance scores. This might help to explain the slight drop in performance for the discrete skill using SLP in the self-monitoring group, as the original submission scores were already high at 94.68. It should be noted that, in order to randomly assign participants across course sections to each condition, we were unable to change the sequence of prompting systems, though this is an area that should be explored in future research. It also is possible that the didactic, lecture-based instruction, video models, and in-class practice affected performance during the initial video submissions; this might explain higher performance for some skills during the initial submissions.

Another limitation relates to our focus on two particular response prompting systems. Although SLP and CTD are commonly used prompting systems to teach skills to students with extensive support needs (Brown et al., 2020), it will be important to determine whether video performance feedback and video self-monitoring are effective in supporting implementation of other prompting systems (e.g., progressive time delay, simultaneous prompting). Finally, our study did not address generalization to applied settings. Pre-service teachers completed all study activities with a peer and, although students delivered systematic instruction for a student with expensive support needs in their practicum placement during the same semester, we were unable to include these data. Unfortunately, generalization programming is often limited in teacher preparation programs (Markelz et al., 2017). Future studies should examine whether supports like video performance feedback and video self-monitoring that are provided within a college course lead to generalized implementation in applied school settings, as this information can inform generalization programming efforts across courses and other program experiences. Opportunities for pre-service teachers to apply systematic instruction for students with extensive support needs in applied settings, including inclusive, general education classrooms, and to receive feedback from course instructors and qualified supervising teachers in these settings has been identified as critical for teacher preparation focused on students with extensive support needs (Kurth et al., 2021). Collaboration among faculty and staff, especially in cases where the course instructor does not oversee students in field-based settings, will be critical to ensure students have opportunities to implement systematic instruction and receive feedback (e.g., in-person or video feedback from instructor, supervising teacher) to transfer their skills to applied settings.

Conclusion

Special education teachers should be equipped with the knowledge and skills necessary to implement EBPs within the classroom, and teacher preparation programs can play an important role in preparing teachers for this responsibility. In particular, programs focused on preparation in the area of students with extensive support needs can support future special education teachers in developing their abilities to implement systematic instruction, including various response prompting systems. Our findings suggest that video performance feedback and video self-monitoring have the potential to prepare pre-service special education teachers to implement systematic instruction, though we encourage additional research across different programs and courses and that focuses on generalization to applied settings.

Supplemental Material

Supplemental Material—Video Performance Feedback and Video Self-Monitoring to Improve Systematic Instruction Implementation for Pre-Service Teachers

Supplemental Material for Video Performance Feedback and Video Self-Monitoring to Improve Systematic Instruction Implementation for Pre-Service Teachers by Virginia L. Walker, Kristin J. Lyon, Amy M. Clausen, Jie Chen and Heather H. Smith in Journal of Special Education Technology

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Virginia L. Walker

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Virginia L. Walker, Ph.D., BCBA-D, is an associate professor of special education at the University of North Carolina at Charlotte. Her research interests include positive behavioral interventions and supports, inclusive education, and effective training practices for school members who support students with extensive support needs.

Kristin Joannou Lyon, Ph.D., is an elementary special education teacher at Lawrence Public Schools. Her research interests include inclusive education, positive behavioral interventions and supports, and mathematics instruction.

Amy Clausen, M.Ed., is a doctoral candidate at the University of North Carolina at Charlotte. Her research interests include general curriculum access for students with extensive support needs and preparing general education teachers for inclusive classrooms.

Jie Chen, Ph.D., is a psychometrician and research associate at Center for Accessible Teaching, Learning, and Assessment Systems and Center for Montessori Research at the University of Kansas. She has more than 10 years of experience in large-scale assessment projects, research design and methodology, test development, test item analysis and data analysis.

Heather Haynes Smith, Ph.D., is an associate professor in the Department of Education at Trinity University. She teaches courses on special education, high incidence disabilities, and reading instruction. Her research interests include multi-tiered systems of supports, learning disabilities, inclusive education, and teacher preparation.

References

Agran

Spooner

Singer

G. H. S.

(2017). Evidence-based practices: The complexities of implementation. Research and Practice for Persons with Severe Disabilities, 42(1), 3−7. https://doi.org/10.1177/1540796916685050

Artman-Meeker

Rosenberg

Badgett

Yang

Penney

(2017). The effects of bug-in-ear coaching on pre-service behavior analysts’ use of functional communication training. Behavior Analysis in Practice, 10(3), 228−241. https://doi.org/10.1007/s40617-016-0166-4

Brock

M. E.

Cannella-Malone

H. I.

Seaman

R. L.

Andzik

N. R.

Schaefer

J. M.

Page

E. J.

Barczak

M. A.

Dueker

S. A.

(2017). Findings across practitioner training studies in special education: A comprehensive review and meta-analysis. Exceptional Children, 84(1), 7−26. https://doi.org/10.0014402917698008

Browder

D. M.

Spooner

Courtade

G. R.

(2020). Teaching students with moderate and severe disabilities (2nd ed.). Guilford Press.

Browder

D. M.

Wood

Thompson

Ribuffo

(2014). Evidence-based practices for students with severe disabilities (Document No. IC-3). University of Florida, Collaboration for Effective Educator, Development, Accountability, and Reform Center. http://ceedar.education.ufl.edu/tools/innovation-configurations/

Brown

McDonnell

Snell

M. E.

(2020). Instruction of students with severe disabilities: Meeting the needs of children and youth with intellectual disabilities, multiple disabilities, and autism spectrum disorder (9th ed.). Pearson.

Brown

Stephenson

Carter

(2014). Multicomponent training of teachers of students with severe disabilities. Teacher Education and Special Education, 37(4), 347−362. https://doi.org/10.1177/0888406414546873

Collins

(2012). Systematic instruction for students with moderate and severe disabilities (1st ed.). Brookes.

Collins

(2022). Systematic instruction for students with moderate and severe disabilities (2nd ed.). Brookes.

10.

Cornelius

K. E.

Nagro

S. A.

(2014). Evaluating the evidence base of performance feedback in preservice special education teacher training. Teacher Education and Special Education, 37(2), 133−146. https://doi.org/10.1177/0888406414521837

11.

Courtade

G. R.

Test

D. W.

Cook

B. G.

(2015). Evidence-based practices for learners with severe intellectual disability. Research and Practice for Persons with Severe Disabilities, 39(4), 305−318. https://doi.org/10.1177/1540796914566711

12.

Every Student (2015). Succeeds Act, 20 U.S.C. § 6301. https://legcounsel.house.gov/Comps/ElementaryAndSecondaryEducationActOf1965.pdf

13.

Fallon

L. M.

Collier-Meek

M. A.

Maggin

D. M.

Sanetti

L. M. H.

Johnson

A. H.

(2015). Is performance feedback for educators an evidence-based practice? A systematic review and evaluation based on single-case research. Exceptional Children, 81(2), 227−246. https://doi.org/10.1177/0014402914551738

14.

Hsiao

Y.-J.

Petersen

S. S.

(2019). Evidence-based practices provided in teacher education and in-service training programs for special education teachers of students with autism spectrum disorders. Teacher Education and Special Education, 42(3), 193−208. https://doi.org/10.1177/0888406418758464

15.

Hugh

M. L.

Johnson

L. D.

Fleury

V. P.

(2020). Dissemination and decision-making: Factors related to pre-service practitioners’ selection of practices for students with autism. Education and Training in Autism and Developmental Disabilities, 55(3), 332−347.

16.

Hunzicker

(2017). From teacher to teacher leader: A conceptual model. International Journal of Teacher Leadership, 8(2), 1−27.

17.

Kilbourn

(1991). Self-monitoring in teaching. American Educational Research Journal, 28(4), 721−736. https://doi.org/10.3102/00028312028004721

18.

Klingner

J. K.

Boardman

A. G.

McMaster

K. L.

(2013). What does it take to scale up and sustain evidence-based practices? Exceptional Children, 79(3), 195−211. https://doi.org/10.1177/001440291307900205

19.

Knight

V. F.

Huber

H. B.

Kuntz

E. M.

Carter

E. W.

Juarez

A. P.

(2019). Instructional practices, priorities, and preparedness for educating students with autism and intellectual disability. Focus on Autism and Other Developmental Disabilities, 34(1), 3−14. https://doi.org/10.1177/1088357618755694

20.

Kurth

J. A.

Allcock

Walker

Olson

Taub

(2021). Faculty perceptions of expertise for inclusive education for students with significant disabilities. Teacher Education and Special Education, 44(2), 117−133. https://doi.org/10.1177/0888406420921582

21.

Leko

M. M.

Brownell

M. T.

Sindelar

P. T.

Kiely

M. T.

(2015). Envisioning the future of special education personnel preparation in a standards-based era. Exceptional Children, 82(1), 25−43. https://doi.org/10.1177/0014402915598782

22.

Marder

deBettencourt

L. U.

(2015). Teaching students with ASD using evidence-based practices: Why is training critical now? Teacher Education and Special Education, 38(1), 5−12. https://doi.org/10.1177/0888406414565838

23.

Markelz

Riden

Scheeler

M. C.

(2017). Generalization training in special education teacher preparation: Does it exist? Teacher Education and Special Education, 40(3), 179−193. https://doi.org/10.1177/0888406417703752

24.

McLeod

R. H.

(2020). Supporting preservice teachers to implement systematic instruction through video review, reflection, and performance feedback. Early Childhood Education Journal, 48(3), 337−343. https://doi.org/10.1007/s10643-019-01001-y

25.

Morin

K. L.

Ganz

J. B.

Vannest

K. J.

Haas

A. N.

Nagro

S. A.

Peltier

C. J.

Fuller

M. C.

Ura

S. K.

(2019). A systematic review of single-case research on video analysis as professional development for special educators. The Journal of Special Education, 53(1), 3−14. https://doi.org/10.1177/002246691879861

26.

Nagro

S. A.

Raines

A. R.

Hooks

S. D.

Fraser

D. W.

Nagy

(2022). The connection between teacher candidate attitude and video-based self-evaluation accuracy. Journal of Special Education Technology, 37(1), 22−34. https://doi.org/10.1177/0162643420947827

27.

O’Reilly

M. F.

Renzaglia

Hutchins

Koterba-Buss

Clayton

Halle

J. W.

Izen

(1992). Teaching systematic instruction competencies to special education student teachers: An applied behavioral supervision model. Journal of the Association for the Severely Handicapped, 17(2), 104−111. https://doi.org/10.1177/154079699201700205

28.

O’Reilly

M. F.

Renzaglia

Lee

(1994). An analysis of acquisition, generalization and maintenance of systematic instruction competencies by preservice teachers using behavioral supervision techniques. Education and Training in Mental Retardation and Developmental Disabilities, 29(1), 22−33. https://doi.org/10.1177/088840649401700304

29.

Rispoli

Zaini

Mason

Brodhead

Burke

M. D.

Gregori

(2017). A systematic review of teacher self-monitoring on implementation of behavioral practices. Teaching and Teacher Education, 63, 58−72. https://doi.org/10.1016/j.tate.2016.12.007

30.

Ruppar

A. L.

Neeper

L. S.

Dalsen

(2016). Special education teachers’ perceptions of preparedness to teach students with severe disabilities. Research and Practice for Persons with Severe Disabilities, 41(4), 273−286. https://doi.org/10.1177/1540796916672843

31.

Ruppar

A. L.

Roberts

C. A.

Olson

A. J.

(2018). Developing expertise in teaching students with extensive support needs: A roadmap. Intellectual and Developmental Disabilities, 56(6), 412−426. https://doi.org/10.1352/1934-9556-56.6.412

32.

Sawyer

M. R.

Andzik

N. R.

Kranak

M. P.

Willke

C. P.

Curiel

E. S. L.

Hensley

L. E.

Neef

N. A.

(2017). Improving pre-service teachers’ performance skills through behavioral skills training. Behavior Analysis in Practice, 10(3), 296−300. https://doi.org/10.1007/s40617-017-0198-4

33.

Sayeski

K. L.

Hamilton-Jones

Cutler

Earle

G. A.

Husney

(2019). The role of practice and feedback for developing teacher candidate’s opportunities to respond expertise. Teacher Education and Special Education, 42(1), 18−35. https://doi.org/10.1177/0888406417735876

34.

Schles

R. A.

Robertson

R. E.

(2019). The role of performance feedback and implementation of evidence-based practices for preservice special education teachers and student outcomes: A review of the literature. Teacher Education and Special Education, 42(1), 36−48. https://doi.org/10.1177/0888406417736571

35.

Spooner

McKissick

B. R.

Knight

V. F.

(2017). Establishing the state of affairs for evidence-based practices in students with severe disabilities. Research and Practice for Persons with Severe Disabilities, 42(1), 8−18. https://doi.org/10.1177/1540796916684896

36.

Spooner

Root

J. R.

Saunders

A. F.

Browder

D. M.

(2019). An updated evidence-based practice review on teaching mathematics to students with moderate and severe developmental disabilities. Remedial and Special Education, 40(3), 150−165. https://doi.org/10.1177/0741932517751055

37.

Steinbrenner

J. R.

Hume

K. A.

Odom

S. L.

Morin

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yucesoy-Ozkan

Savage

M. N.

(2020). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. The national clearinghouse on autism evidence and practice review team.

38.

Taub

D. A.

McCord

J. A.

Ryndak

(2017). Opportunities to learn for students with extensive support needs: A context of research-support practices for all in general education classrooms. The Journal of Special Education, 5(3), 127−137. https://doi.org/10.1177/0022466917696263

39.

Wood

C. L.

Goodnight

C. I.

Bethune

K. S.

Preston

A. I.

Cleaver

S. L.

(2016). Role of professional development and multi-level coaching in promoting evidence-based practices in education. Learning Disabilities: A Contemporary Journal, 14(2), 159−170.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB